Screen projection control method, related equipment and system

Abstract

The embodiment of the invention provides a screen projection control method. The method can be applied to a screen projection control system comprising a source, a sink and a remote control device. In the method, when a key on the remote control equipment is pressed down by a user, the remote control equipment can send a broadcast signal outwards. Within the communicable distance, both the transmitting device and the receiving device can receive the broadcast signal. If the sending device and the receiving device are in the screen projection state, the sending device processes the broadcast signal, and the receiving device gives up to process the broadcast signal; and if the transmitting device and the receiving device are not in the screen projection state, the receiving device processes the broadcast signal, and the transmitting device filters or ignores the broadcast signal. Thus, in a screen projection scene, the sending device can respond to the key operation of the user on the remote control device, the receiving device does not need to relay, and time delay is not introduced.

Description

Technical Field

[0001] This application relates to the field of electronic technologies, and in particular to a screen mirroring control method, related devices and systems. Background Art

[0002] Mirror screen mirroring and application screen mirroring are two other data sharing technologies outside the Digital Living Network Alliance (DLNA). However, whether it is mirror screen mirroring or application screen mirroring, since the source is an electronic device such as a mobile phone and the sink is a large-screen device such as a TV, the remote control of the large-screen device cannot operate the display content on the large screen in the screen mirroring scenario. For example, it is impossible to pause a video or click on the desktop icon of an application to open the application, etc. The user needs to operate on the mobile phone to change the display content on the large screen. Summary of the Invention

[0003] In a first aspect, an embodiment of the present application provides a screen mirroring control method, which can be applied to a screen mirroring control system. The screen mirroring control system may include: a sending device, a receiving device, and a remote control device.

[0004] The method provided in the first aspect may include: detecting that the user operates a first button on the remote control device, and the remote control device emits a first broadcast signal outward. Within the communicable distance, both the sending device and the receiving device can receive the broadcast signal. If the sending device is mirroring to the receiving device, after receiving the first broadcast signal, the receiving device abandons processing the first broadcast signal, and the sending device processes the first broadcast signal. If the sending device is not mirroring to the receiving device, after receiving the first broadcast signal, the receiving device processes the first broadcast signal, and the sending device abandons processing the first broadcast signal.

[0005] By implementing the method of the first aspect, in the screen mirroring scenario, it is possible to implement the sending device (source) to respond to the button operation of the user on the remote control, without the need for the receiving device (sink) to relay.

[0006] In combination with the first aspect, in some embodiments, the sending device may be an electronic device that provides screen mirroring content. The sending device can be used to generate a screen mirroring stream for mirroring to the receiving device. The screen mirroring stream may include: the screen stream of the sending device, or the application stream of the application to be mirrored on the sending device. The former is the screen mirroring data during mirror screen mirroring, and the latter is the screen mirroring data during application screen mirroring. In the scenario of mirror screen mirroring, the sending device can record the screen while forming a screen stream based on the recorded content and sending it to the receiving device. In the scenario of application screen mirroring, when the application to be mirrored outputs content, the sending device can form an application stream based on the content and send it to the receiving device.

[0007] In combination with the first aspect, in some embodiments, the receiving device may be a display device for the screen mirroring content. In a screen mirroring scenario, the receiving device can be used to receive the screen mirroring stream while displaying it, so as to present a picture that is synchronized and consistent with the sending side on the receiving side. Compared with the sending device, the receiving device may have more powerful display capabilities, such as a larger display screen. Not limited to display, the receiving device can also be used to render the audio stream in the screen mirroring data and play the sound. Screen mirroring can include not only the picture but also the sound. The receiving device may include, but is not limited to, a tablet computer, a television, a smart screen, a vehicle-mounted device, or an electronic billboard, etc.

[0008] In combination with the first aspect, in some embodiments, both the sending device and the receiving device may have wireless communication modules such as WLAN modules to establish a screen mirroring communication connection suitable for transmitting the screen mirroring stream. Not limited to wireless communication connections, the screen mirroring communication connection can also be a wired communication connection, such as a High-Definition Multimedia Interface (HDMI) wired connection.

[0009] In combination with the first aspect, in some embodiments, the remote control may be configured with one or more buttons. The remote control can be paired with the receiving device. Here, being paired means that the receiving device may store the key mapping relationship for the remote control, and this key mapping relationship records the user instructions mapped by the buttons on the remote control in various application scenarios. For example, in a video playback scenario, the left arrow key on the remote control is mapped to the user instruction "rewind", the right arrow key is mapped to the user instruction "fast forward", and the confirmation key (OK) is mapped to the user instruction "play / pause". Another example is that in a slide show scenario, the left arrow key or the up arrow key is mapped to the user instruction "previous page", and the right arrow key or the down arrow key is mapped to "next page". In this way, when the user presses the physical button on the remote control, the receiving device can determine the corresponding user instruction based on the key mapping relationship, thereby understanding the user's operation intention, performing corresponding processing, and giving corresponding feedback.

[0010] In combination with the first aspect, in some embodiments, the remote control can be a Bluetooth remote control with a Bluetooth module. When the user operates the buttons on the remote control, the remote control can send out Bluetooth broadcast signals. The sending device and the receiving device can also have Bluetooth modules, so that both can receive the Bluetooth broadcast signals. In a non-screen mirroring scenario, the sending device does not process the Bluetooth broadcast signals sent by the remote control, and only the receiving device processes the Bluetooth broadcast signals. In this way, in a non-screen mirroring scenario, the user can directly control the receiving device with the remote control, such as playing the next episode. In a screen mirroring scenario, only the sending device processes the Bluetooth broadcast signals sent by the remote control 300, and the receiving device does not process the Bluetooth broadcast signals. In this way, in a screen mirroring scenario, when the user operates the remote control, the sending device can respond to the user operation and change the screen mirrored to the receiving side, so that the display screen on the receiving device is controlled by the user operation on the remote control, realizing reverse control. This reverse control does not require the receiving device to relay, does not introduce latency, and does not additionally require the receiving device to provide storage space to store the mapping relationship between the user input on the remote control and the coordinates on the touch screen of the sending device.

[0011] Not limited to Bluetooth communication, the remote control can also send broadcast messages to the sending device and the receiving device based on other communication technologies (such as Wi-Fi communication) to broadcast the event that the button on the remote control is pressed. The remote control and the sending device and the receiving device only need to have the same or corresponding communication modules that support broadcast communication.

[0012] In combination with the first aspect, in some embodiments, the sending device processes the first broadcast signal, which may include: the sending device extracts the key value of the first button from the first broadcast signal, and determines whether the first button exists in the button mapping relationship corresponding to the remote control device based on the key value; the sending device determines whether the first application that provides the screen mirroring content belongs to the white list; if the first button exists in the button mapping relationship and the first application belongs to the white list, the sending device executes the user instruction corresponding to the first button.

[0013] In combination with the first aspect, in some embodiments, the receiving device may store the button mapping relationship. The receiving device processes the first broadcast signal, which may include: the receiving device identifies the application scenario where the receiving device is located, determines the user instruction corresponding to the first button according to the button mapping relationship and the application scenario where the receiving device is located, and executes the user instruction.

[0014] In combination with the first aspect, in some embodiments, before the sending device mirrors the screen to the receiving device, the method may further include: the receiving device sends the button mapping relationship to the sending device.

[0015] In combination with the first aspect, in some embodiments, when a sending device performs screen mirroring to a receiving device, it may include: establishing a screen mirroring communication connection between the sending device and the receiving device, the sending device transmitting a screen mirroring stream to the receiving device through the screen mirroring communication connection, and the receiving device displaying a screen mirroring image based on the screen mirroring stream.

[0016] In combination with the first aspect, in some embodiments, when the sending device is not performing screen mirroring to the receiving device, it may include: the sending device and the receiving device do not establish a screen mirroring communication connection, or the screen mirroring communication connection is established but the sending device does not transmit a screen mirroring stream to the receiving device through the screen mirroring communication connection, or the screen mirroring communication connection is established but the screen mirroring image is not displayed on the receiving device.

[0017] In combination with the first aspect, in some embodiments, the screen mirroring may include mirror screen mirroring, and the screen mirroring stream may include a screen stream; the first application is a foreground application on the sending device, and the screen mirroring image displayed on the receiving device is the same as the image displayed on the sending device.

[0018] In combination with the first aspect, in some embodiments, the screen mirroring may include application screen mirroring, and the screen mirroring stream may include an application stream generated based on the screen mirroring content provided by the first application; the first application does not perform full-screen display on the sending device, and not performing full-screen display may include: displaying through a floating window, or not displaying.

[0019] In combination with the first aspect, in some embodiments, the screen mirroring control method may further include: the receiving device pairs with a remote control device, the receiving device receives a broadcast signal transmitted by the remote control device during pairing, and determines the broadcast signal as a trusted broadcast signal, and the broadcast signal carries a first parameter, and the first parameter is used to identify the trusted broadcast signal.

[0020] In combination with the first aspect, in some embodiments, before the receiving device processes the first broadcast signal, the screen mirroring control method may further include:

[0021] The receiving device determines that the first broadcast signal is a trusted broadcast signal.

[0022] In combination with the first aspect, in some embodiments, before the sending device processes the first broadcast signal, the screen mirroring control method may further include:

[0023] The sending device determines that the first broadcast signal is a trusted broadcast signal.

[0024] In combination with the first aspect, in some embodiments, after the sending device and the receiving device establish a screen mirroring communication connection, the screen mirroring control method may further include: the receiving device notifies the sending device of the first parameter.

[0025] In a second aspect, an embodiment of the present application provides a screen mirroring control method, which may be applied to a sending device in a screen mirroring control system, and the screen mirroring control system may include: a sending device, a receiving device, and a remote control device.

[0026] The screen projection control method provided in the second aspect may include: a transmitting device receiving a first broadcast signal transmitted by a remote control device, the first broadcast signal being transmitted by the remote control device when a user operates a first button on the remote control device; the transmitting device determining whether the transmitting device is projecting the screen to a receiving device; if so, processing the first broadcast signal; if not, abandoning processing the first broadcast signal; and the receiving device being configured to abandon processing the first broadcast signal when the transmitting device is projecting the screen to the receiving device, and to process the first broadcast signal when the transmitting device is not projecting the screen to the receiving device.

[0027] In combination with the second aspect, in some embodiments, the sending device processes the first broadcast signal, which may include: the sending device extracts the key value of the first button from the first broadcast signal, and determines whether the first button exists in the key mapping relationship corresponding to the remote control device based on the key value; the sending device determines whether the first application providing the projection content belongs to the whitelist; if the first button exists in the key mapping relationship, and the first application belongs to the whitelist, the sending device executes the user instruction corresponding to the first button; wherein, the key mapping relationship is used to record the user instructions mapped to the buttons on the remote control device in multiple application scenarios; the whitelist includes: identification information of one or more applications on the sending device, and the application scenarios provided by one or more applications exist in the key mapping relationship.

[0028] In combination with the second aspect, in some embodiments, the screen projection control method may further include: before the sending device projects the screen to the receiving device, the receiving device sends a key mapping relationship to the sending device.

[0029] In combination with the second aspect, in some embodiments, when a sending device projects a screen to a receiving device, it may include: establishing a screen projection communication connection between the sending device and the receiving device, and the sending device transmitting the screen projection stream to the receiving device through the screen projection communication connection.

[0030] In combination with the second aspect, in some embodiments, the sending device is not projecting the screen to the receiving device, which may include: no screen projection communication connection is established between the sending device and the receiving device, or the screen projection communication connection is established but the sending device does not transmit the screen projection stream to the receiving device through the screen projection communication connection.

[0031] In conjunction with the second aspect, in some embodiments, screen projection includes mirror projection, and the screen projection stream includes screen stream. The first application is a foreground application on the sending device, and the projection screen displayed on the receiving device is consistent with the screen displayed on the sending device.

[0032] In combination with the second aspect, in some embodiments, screen projection includes application projection, and the projection stream includes an application stream generated based on the projection content provided by the first application; the first application is not displayed in full screen on the sending device, and not displaying in full screen includes: displaying through a virtual floating window, or not displaying.

[0033] In combination with the second aspect, in some embodiments, the screen mirroring control method may further include: the sending device receives a first parameter notified by the receiving device, where the first parameter is used to identify a trusted broadcast signal, and the trusted broadcast signal carrying the first parameter is transmitted by a remote control device paired with the receiving device.

[0034] In combination with the second aspect, in some embodiments, before the sending device processes the first broadcast signal, the screen mirroring control method may further include:

[0035] The sending device determines that the first broadcast signal carries the first parameter.

[0036] In a third aspect, an embodiment of the present application provides a screen mirroring control method, which can be applied to a receiving device in a screen mirroring control system. The screen mirroring control system includes: a receiving device, a sending device, and a remote control device.

[0037] The screen mirroring control method provided in the second aspect may include: the receiving device receives a first broadcast signal sent by the remote control device; the first broadcast signal is transmitted by the remote control device when the user operates a first button on the remote control device; the receiving device determines whether the sending device is mirroring to the receiving device. If so, the receiving device abandons processing the first broadcast signal; if not, the receiving device processes the first broadcast signal; the sending device is used to process the first broadcast signal when the sending device is mirroring to the receiving device, and abandons processing the first broadcast signal when the sending device is not mirroring to the receiving device.

[0038] In combination with the third aspect, in some embodiments, a key mapping relationship may be stored on the receiving device, and the key mapping relationship can be used to record the user instructions mapped by the keys on the remote control device in multiple application scenarios.

[0039] The receiving device processes the first broadcast signal, which may include: the receiving device first identifies the application scenario where the receiving device is located, then determines the user instruction corresponding to the first button in the application scenario where the receiving device is located from the key mapping relationship, and executes the determined user instruction.

[0040] In combination with the third aspect, in some embodiments, before the sending device mirrors to the receiving device, the screen mirroring control method may further include: the receiving device sends the key mapping relationship to the sending device.

[0041] In combination with the third aspect, in some embodiments, the sending device mirroring to the receiving device may include: establishing a screen mirroring communication connection between the sending device and the receiving device, the sending device transmitting a screen mirroring stream to the receiving device through the screen mirroring communication connection, and the receiving device displaying a screen mirroring picture based on the screen mirroring stream.

[0042] In combination with the third aspect, in some embodiments, when the sending device is not casting the screen to the receiving device, it may include: no screen mirroring communication connection is established between the sending device and the receiving device, or although the screen mirroring communication connection is established, the sending device does not transmit the screen mirroring stream to the receiving device through the screen mirroring communication connection, or although the screen mirroring communication connection is established, the screen mirroring image is not displayed on the receiving device.

[0043] In combination with the third aspect, in some embodiments, screen mirroring may include mirror screen mirroring, and the screen mirroring stream may include a screen stream. The first application may be a foreground application on the sending device, and the screen mirroring image displayed on the receiving device is the same as the image displayed on the sending device.

[0044] In combination with the third aspect, in some embodiments, screen mirroring includes applicable screen mirroring, and the screen mirroring stream may include an application stream generated based on the screen mirroring content provided by the first application. The first application is not displayed full screen on the sending device. Not being displayed full screen includes: being displayed through a floating window, or not being displayed.

[0045] In combination with the third aspect, in some embodiments, the screen mirroring control method may further include: the receiving device is paired with a remote control device, the receiving device receives the broadcast signal emitted by the remote control device during pairing, and determines the broadcast signal as a trusted broadcast signal. The broadcast signal carries a first parameter, and the first parameter is used to identify the trusted broadcast signal.

[0046] In combination with the third aspect, in some embodiments, before the receiving device processes the first broadcast signal, the screen mirroring control method may further include:

[0047] The receiving device determines that the first broadcast signal is a trusted broadcast signal.

[0048] In combination with the third aspect, in some embodiments, the screen mirroring control method may further include: the receiving device notifies the sending device of the first parameter.

[0049] In a fourth aspect, an embodiment of the present application provides an electronic device, which is the sending device in the foregoing screen mirroring control system, and may include: a processor and a memory, where the memory is coupled to the processor; wherein, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the screen mirroring control method described in any one or more of the foregoing second aspect embodiments.

[0050] In a fifth aspect, an embodiment of the present application provides an electronic device, which is the receiving device in the foregoing screen mirroring control system, and may include: a processor and a memory, where the memory is coupled to the processor; wherein, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the screen mirroring control method described in any one or more of the foregoing third aspect embodiments.

[0051] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium including instructions, characterized in that when the instructions run on an electronic device, the electronic device is caused to execute the method described in any one or more of the embodiments of the foregoing second aspect.

[0052] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium including instructions, characterized in that when the instructions run on an electronic device, the electronic device is caused to execute the method described in any one or more of the embodiments of the foregoing third aspect.

[0053] In an eighth aspect, an embodiment of the present application provides a picture editing system, which may include: a remote control device, a sending device, and a receiving device, where the sending device is the sending device in the foregoing screen mirroring control system and is also the electronic device described in the fourth aspect; the receiving device is the receiving device in the foregoing screen mirroring control system and is also the electronic device described in the fifth aspect. Description of the Drawings

[0054] Figure 1A Exemplarily shows a screen mirroring scenario;

[0055] Figure 1B Exemplarily shows an application screen mirroring scenario;

[0056] Figure 2 Shows the screen mirroring control system 10 provided by the embodiment of the present application;

[0057] Figure 3 Exemplarily shows the electronic device 20 provided by the embodiment of the present application;

[0058] Figure 4 Exemplarily shows the remote control device 30 provided by the embodiment of the present application;

[0059] Figure 5 Shows the screen mirroring control method provided by the embodiment of the present application. Detailed Embodiments

[0060] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0061] The Digital Living Network Alliance (DLNA) is a set of protocols for interoperability between PCs, mobile devices, and consumer electronics initiated by Sony, Intel, Microsoft, etc. The goal of DLNA is to "enjoy music, photos, and videos anytime, anywhere". DLNA is file-based, and media files can have various encoding formats. The player must be able to handle such a variety of encoding formats. Usually, for a better playback experience, DLNA will cache for a short period of time. What DLNA has in common with mirror screen mirroring and app mirroring is that they all support users to project media files (such as audio / video files, pictures, etc.) in the mobile phone onto a large screen for playback; however, the difference is that DLNA does not support real-time projection of the screen content or app content on the mobile phone onto the large screen for display. It only supports projecting photos and audio / video files in the mobile phone onto the large screen for playback. Moreover, when using DLNA mirroring, the large screen is the source device (source), and users can use the remote control supporting the large screen to control the playback of photos and audio / video files on the large screen.

[0062] Mirror screen mirroring supports projecting the screen content on electronic devices such as mobile phones and tablets onto large screen devices such as televisions for display. Users can see the same screen content on the large screen device such as the TV as on the electronic device such as the mobile phone. For example, as Figure 1A shown, when using mirror screen mirroring, when the user opens a video playback screen on the mobile phone, the same video playback screen will also be displayed on the large screen device, and the screens displayed on the large screen device and the mobile phone are the same; when the user operates the mobile phone, the display content on the large screen device will change simultaneously while the display content on the mobile phone changes. App mirroring supports only projecting app content onto the large screen device for display. When mirroring, the mobile phone can switch to other interfaces, such as the desktop, and the display screens of the mobile phone and the large screen device do not interfere with each other. For example, as Figure 1B shown, when mirroring a video app, the video app can be displayed through a floating small window 11. In this way, the user can watch the video played by the video app displayed on the large screen device on one side, and switch to the desktop on the mobile phone on the other side, so as to perform other operations, such as opening a chat app to send chat messages, etc., achieving the goal of watching movies and chatting at the same time.

[0063] However, as described in the background art section, neither of them supports using the remote control of the large screen device to operate the display content on the large screen.

[0064] To solve this problem, the Wireless Fidelity (Wi-Fi) Alliance has launched the Miracast video reverse control function. Its main process may include: The sink receives user input through input devices such as remote controls, mice, and touch screens. The user input can be, for example, user operations such as pressing a remote control button, clicking a mouse, or touching the screen. Then, the sink transmits the user input received on the sink side to the source side through the user input back channel (UIBC) for processing. For example, the coordinates clicked by the user are transmitted to the source side to achieve reverse control on the source side. In this way, the input devices such as remote controls, mice, and touch screens connected to the sink can also be used as input devices for the source side. Users can then operate the input devices on the sink side to change the screen casted to the sink side without having to move to the source side to switch the screen cast, which is especially convenient for users to conduct content demonstrations on large-screen devices and other sink sides.

[0065] However, this reverse control function has the following problems: 1. It introduces latency because the sink does not process user input but acts as a relay to transmit user input to the source side for processing; 2. Coordinate mapping is required. For example, various types of user input received by input devices such as remote controls on the sink side need to be mapped to the coordinates on the touch screen of the source side, and the sink side is required to provide storage space in hardware to store this coordinate mapping.

[0066] The embodiments of the present application provide a screen casting control method, related devices, and a screen casting control system. While supporting users to use input devices such as remote controls on the sink side to operate the switching of the screen cast on the source side, it does not introduce relay latency, and the key mapping relationship can be stored on the source side devices such as mobile phones and personal computers, without requiring the sink side devices to provide additional storage, making it easier to maintain and expand.

[0067] The screen casting scenarios involved in the embodiments of the present application include mirror screen casting scenarios and application screen casting scenarios, excluding DLNA screen casting scenarios.

[0068] Figure 2 Figure 10 shows the screen casting control system 10 provided by the embodiments of the present application.

[0069] As Figure 2 shown, the screen casting control system 10 may include a sending device 100, a receiving device 200, and a remote control 300. The sending device 100, the receiving device 200, and the remote control 300 cooperate together to jointly achieve screen casting and the screen casting control method provided by the embodiments of the present application. Among them,

[0070] The transmitting device 100 is an electronic device that provides screen mirroring content. The transmitting device 100 can be used to generate a screen mirroring stream for mirroring to the receiving device 200. The screen mirroring stream can include: the screen stream of the transmitting device 100, or the application stream of the application to be mirrored on the transmitting device 100. The former is the screen mirroring data in the case of mirror screen mirroring, and the latter is the screen mirroring data in the case of application screen mirroring. In the scenario of mirror screen mirroring, the transmitting device 100 can record the screen while forming a screen stream based on the recorded screen content and sending it to the receiving device 200. In the scenario of application screen mirroring, when the application to be mirrored outputs content, the transmitting device 100 can form an application stream based on the content and send it to the receiving device 200. The transmitting device 100 can include, but is not limited to, a smart phone, a tablet computer, a personal computer (PC), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device (such as a smart watch, smart glasses), etc.

[0071] The receiving device 200 is a display device for screen mirroring content. In the screen mirroring scenario, the receiving device 200 can be used to receive the screen mirroring stream while displaying, so as to present a picture on the receiving side that is synchronized and consistent with the sending side, such as synchronously presenting the system desktop 13 displayed on the transmitting device 100. Often, compared with the transmitting device 100, the receiving device 200 can have more powerful display capabilities, such as having a larger display screen. Not limited to display, the receiving device can also be used to render the audio stream in the screen mirroring data and play the sound. Screen mirroring can be more than just the picture, and can also include sound. The receiving device 200 can include, but is not limited to, a tablet computer, a television, a smart screen, a vehicle-mounted device, or an electronic billboard, etc.

[0072] The transmitting device 100 may also be equipped with a display screen, which can display the local content of the transmitting device 100 or the content from the network. The transmitting device 100 can directly receive user input, such as various types of touch gestures on the screen, such as swipe gestures, click gestures, drag gestures, pinch gestures, etc., and in response to the user input, change the content displayed on the display screen.

[0073] The transmitting device 100 and the receiving device 200 may both be equipped with a wireless communication module such as a WLAN module to establish a screen mirroring communication connection 12 suitable for transmitting the screen mirroring stream. The WLAN module may provide a solution for WLAN communication including one or more of Wi-Fi direct, Wi-Fi LAN, or Wi-Fi soft AP. The transmitting device 100 and the receiving device 200 may be in the same local area network. For example, by connecting to the same Wi-Fi router, they are in the same Wi-Fi local area network. After discovering the receiving device 200, the transmitting device 100 may establish a wireless communication connection with the receiving device 200, such as a Wi-Fi direct communication connection. Not limited to wireless communication connections, the screen mirroring communication connection 12 may also be a wired communication connection, such as a high definition multimedia interface (HDMI) wired connection.

[0074] The remote control 300 may be configured with one or more buttons, such as a power button 301, an OK button 302, a back button 303, a home button 304, a menu button 305, a volume up button 306, a volume down button 307, and a left direction button 308, a right direction button 309, an up direction button 310, a down direction button 311, and so on. The remote control 300 may be paired with the receiving device 200. Here, being paired may mean that the receiving device 200 may store the key mapping relationship for the remote control 300, and this key mapping relationship records the user instructions mapped by the buttons on the remote control 300 in various application scenarios. For example, in the video playback scenario, the left direction button 308 on the remote control 300 is mapped to the user instruction "rewind", the right direction button 309 is mapped to the user instruction "fast forward", and the OK button 302 is mapped to the user instruction "play / pause". Another example is that in the slide presentation scenario, the left direction button 308 or the up direction button 310 is mapped to the user instruction "previous page", and the right direction button 309 or the down direction button 311 is mapped to "next page". In this way, when the user presses the physical buttons on the remote control 300, the receiving device 200 can determine the corresponding user instructions based on the key mapping relationship, thereby understanding the user's operation intention, performing corresponding processing, and giving corresponding feedback. For example, in the video playback scenario, when the user presses the right direction button 309 on the remote control 300, the receiving device 200 can determine that the corresponding user instruction is "fast forward", and in response to this, it can perform fast forward playback of the video to achieve the user's operation intention of fast forwarding the video. Here, the examples are only used to explain the embodiments of the present application and should not constitute a limitation.

[0075] In the embodiments of the present application, the remote controller 300 can be a Bluetooth remote controller with a Bluetooth module. When a user operates a button on the remote controller, the remote controller 300 can send out a Bluetooth broadcast signal. The sending device 100 and the receiving device 200 can also have Bluetooth modules, so that both can receive the Bluetooth broadcast signal. In a non-screen mirroring scenario, the sending device 100 does not process the Bluetooth broadcast signal sent by the remote controller 300, and only the receiving device 200 processes the Bluetooth broadcast signal. In this way, in a non-screen mirroring scenario, the user can directly control the receiving device 200 with the remote controller 300, such as playing the next episode. In a screen mirroring scenario, only the sending device 100 processes the Bluetooth broadcast signal sent by the remote controller 300, and the receiving device 200 does not process the Bluetooth broadcast signal. In this way, in a screen mirroring scenario, when the user operates the remote controller 300, the sending device 100 can respond to the user operation and change the screen mirrored to the receiving side, so that the display screen on the receiving device 200 is controlled by the user operation on the remote controller 300, realizing reverse control. This reverse control does not require the receiving device 200 to relay, does not introduce latency, and does not additionally require the receiving device 200 to provide storage space to store the mapping relationship between the user input on the remote controller 300 and the coordinates on the touch screen of the sending device 100.

[0076] Not limited to Bluetooth communication, the remote controller 300 can also send broadcast messages to the sending device 100 and the receiving device 200 based on other communication technologies (such as Wi-Fi communication) to broadcast the event that a button on the remote controller 300 is pressed. The remote controller 300 and the sending device 100, the receiving device 200 only need to have the same or corresponding communication modules that support broadcast communication.

[0077] Figure 3 Exemplarily shows the electronic device 20 provided by the embodiments of the present application.

[0078] The electronic device 20 can be implemented as the sending device 100 in the screen mirroring control system 10, and its device type can be a smart phone, a tablet computer, a personal computer (PC), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device (such as a smart watch, smart glasses), etc.

[0079] The electronic device 20 can also be implemented as the receiving device 200 in the screen mirroring control system 10, and its device type can be a tablet computer, a television, a smart screen, a vehicle-mounted device, or an electronic billboard, etc.

[0080] Such as Figure 3As shown, the electronic device 20 may include: a processor 110, a memory 120, a display screen 130, a display driver integrated circuit (DDIC) 140, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a gyroscope sensor 180B, an acceleration sensor 180E, a touch sensor 180K, etc. Each part in the electronic device 20 may be connected through a bus.

[0081] The processor 110 may be one or more, and they may be integrated within an integrated circuit of a system on chip (SOC). The SOC is a system-level chip. The processor 110 may include a central processing unit (CPU), a graphic processing unit (GPU), a neural-network processing unit (NPU), etc. Among them, the CPU may include an application processor (AP), a baseband processor chip (BP), etc. The AP may be responsible for running the operating system, user interface, and application programs on the terminal device; the BP may be responsible for transmitting and receiving wireless signals and managing radio frequency services. The GPU may be responsible for graphic rendering, performing coloring, material filling, rendering, output, etc. based on the rendering instructions and data from the CPU. By drawing on the structure of biological neural networks, such as the transmission mode between human brain neurons, the NPU can quickly process input information and can also continuously learn by itself. The NPU can be used to run artificial intelligence algorithms, such as an editing instruction recommendation algorithm, an image processing algorithm, an image understanding algorithm, etc. The CPU and GPU can be used to render and synthesize the picture to be sent to the display screen 130.

[0082] The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0083] A cache memory may be provided within the processor 110, which can be used to store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the cache memory, which can reduce the waiting time of the processor 110 and improve the program running efficiency.

[0084] Among them, the memory 120 may include a program storage area and a user data storage area. The program storage area can store an operating system and one or more application programs (such as game applications), and the data storage area can store data created by the user during the use of the electronic device 20 (such as photos, contacts). The display 130 can be a high-speed random access memory or a non-volatile memory, such as a disk, a flash memory, a universal flash storage (UFS), etc. The memory 120 can also be an external memory card, such as a Micro SD card.

[0085] The memory 120 may also store the code instructions of the screen mirroring control method provided in the embodiments of the present application. When the processor 110 reads the code instructions from the memory 120 and runs the code instructions, the electronic device 20 can execute the steps performed by the sending device or the receiving device in the screen mirroring control method provided in the embodiments of the present application.

[0086] The memory 120 may also be integrated with the processor 110 into an integrated circuit of the SOC.

[0087] The electronic device 20 can implement the display function through the SOC, the DDIC 140, and the display screen 130, etc.

[0088] Among them, the display screen 130 has multiple refresh rates. The refresh rate represents the number of times the display screen refreshes the display image within 1 second. For example, a refresh rate of 60 Hertz (Hz) means that the display screen refreshes the display image 60 times within 1 second. The display screen 130 can adopt an LTPO display panel, allowing the refresh rate to be reduced to a low refresh rate, such as 10Hz or 1Hz, thereby supporting the reduction of the power consumption of the display screen.

[0089] Among them, the display driver integrated circuit (DDIC) 140 can be used as the control core of the display screen 130, driving the display screen 130 to work and receiving data from the SOC (processor 110), such as image data and some instructions. The DDIC 140 can send driving signals and data to the display panel of the display screen 130 in the form of electrical signals, and then realize the control of the screen brightness and color, so that image information such as letters and pictures can be displayed on the screen to complete the screen refresh.

[0090] The image data of the picture to be displayed sent by the SOC to the DDIC 140 can be sent to the frame buffer for storage to complete the display (or picture sending). Then, the DDIC 140 extracts the image data from the frame buffer and drives the display screen 130 to display.

[0091] The wireless communication function of the electronic device 20 can be realized through antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0092] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 20 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: Antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0093] The mobile communication module 150 can provide wireless communication solutions such as 2G / 3G / 4G / 5G applied to the electronic device 20. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through antenna 1 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 can be arranged in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be arranged in the same device.

[0094] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 130. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0095] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 20, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive the signal to be transmitted from the processor 110, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through the antenna 2 for radiation.

[0096] In some embodiments, the antenna 1 of the electronic device 20 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 20 can communicate with the network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0097] The electronic device 20 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 130, and the application processor, etc.

[0098] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.

[0099] The camera 193 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard formats such as RGB and YUV. In some embodiments, the electronic device 20 may include one or N cameras 193, where N is a positive integer greater than 1.

[0100] The video codec is used to compress or decompress digital videos. The electronic device 20 can support one or more video codecs. In this way, the electronic device 20 can play or record videos in multiple encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0101] The electronic device 20 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0102] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0103] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 20 can listen to music or make hands-free calls through the speaker 170A.

[0104] The receiver 170B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 20 answers a call or a voice message, it can listen to the voice by placing the receiver 170B close to the ear.

[0105] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C with their mouth to input the sound signal into the microphone 170C. The electronic device 20 can be provided with at least one microphone 170C. In some other embodiments, the electronic device 20 can be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 20 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0106] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0107] The keys 190 include a power key, a volume key, etc. The keys 190 can be mechanical keys or touch keys. The electronic device 20 can receive key inputs and generate key signal inputs related to the user settings and function controls of the electronic device 20. The motor 191 can generate a vibration prompt. The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the electronic device 20.

[0108] Figure 3 The schematic structure does not specifically limit the electronic device 20. The electronic device 20 can include more or fewer parts than shown in the figure, or combine certain parts, or split certain parts, or have different arrangements of parts. Each part shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0109] Figure 4 Exemplarily shown is the remote control device 30 provided by the embodiments of the present application.

[0110] The remote control device 30 can be the remote controller 300 in the screen mirroring control system 10. As Figure 4 shown, the remote control device 30 can include: a processor 201, a memory 202, a wireless communication processing module 203, keys 204, a power management module 205, and so on. Among them:

[0111] The processor 201 can be used to read and execute computer-readable instructions. In a specific implementation, the processor 201 mainly includes a controller, an arithmetic unit, and registers. Among them, the controller is mainly responsible for instruction decoding and sending control signals for the operations corresponding to the instructions. The arithmetic unit is mainly responsible for storing the register operands and intermediate operation results temporarily stored during the execution of the instructions, etc. In a specific implementation, the hardware architecture of the processor 201 can be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

[0112] The memory 202 is coupled to the processor 201 and is used to store various software programs and / or multiple sets of instructions. In a specific implementation, the memory 202 can include a high-speed random access memory and can also include a non-volatile memory. The memory 202 can store a communication program, which can be used to communicate with other devices.

[0113] The processor 201 can be used to generate signals sent out by the wireless communication processing module 203, such as Bluetooth broadcast signals, etc. The processor 201 can also be used to parse the signals received by the wireless communication processing module 203, such as Bluetooth signals sent by the electronic device 20, etc.

[0114] The wireless communication processing module 203 can include a Bluetooth communication processing module 203A, a WLAN communication processing module 203B, etc. The wireless communication processing module 203 can transmit broadcast signals, such as Bluetooth broadcast signals. The wireless communication processing module 203 can also detect signals emitted by other devices, such as probe requests, scan signals, etc., and can send response signals, such as probe responses, scan responses, etc., so that other devices can discover the remote control device 30 and establish a wireless communication connection. Among them, the Bluetooth communication processing module 203A can provide a solution for one or more Bluetooth communications including classic Bluetooth or Bluetooth low energy (BLE). The WLAN communication processing module 203B can include a solution for one or more WLAN communications such as Wi-Fi direct, Wi-Fi LAN, or Wi-Fi softAP.

[0115] The button 204 can include one or more buttons, such as a power button, an OK button, a return button, a home button, a menu button, a volume up button, a volume down button, and a left direction button, a right direction button, an up direction button, a down direction button, etc. Not limited to physical buttons, the button 204 can also be implemented as a touch button. When the user clicks or presses the button 204, the remote control device 30 can send out a broadcast signal, and the key value and other information of the button can be carried in the broadcast signal to broadcast the button pressed by the user. In this way, the device that receives this broadcast signal, such as Figure 2In the transmitting device 100 or the receiving device 200 in the screen mirroring control system 10 shown, it can be known which button the user has pressed and corresponding processing can be performed.

[0116] The power management module 205 can be used to connect to a battery (not shown), a charging management module (not shown), and the processor 201. The power management module 205 receives inputs from the battery (not shown) and / or the charging management module (not shown) and supplies power to the processor 201, the memory 202, the wireless communication processing module 203, etc. The power management module 205 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance).

[0117] Figure 4 The schematic structure does not constitute a specific limitation on the remote control device 30. The remote control device 30 may include more or fewer parts than shown, or combine certain parts, or split certain parts, or have different part arrangements. For example, the remote control device 30 may include a rocker device, and in this case, the remote control device 30 can be implemented as a rocker handle.

[0118] Figure 5 The screen mirroring control method provided by the embodiment of the present application is shown.

[0119] This method can be applied to Figure 2 The screen mirroring control system 10 shown. In this method, when a button on the remote control is pressed by the user, the remote control 300 can send out a broadcast signal. Within the communicable distance, both the transmitting device 100 and the receiving device 200 can receive this broadcast signal. If the receiving device 200 is mirroring the screen to the transmitting device 100, the transmitting device 100 processes this broadcast signal and the receiving device 200 abandons processing this broadcast signal; if the transmitting device 100 is not mirroring the screen to the receiving device 200, the receiving device 200 processes this broadcast signal and the transmitting device 100 filters or ignores this broadcast signal. In this way, in the screen mirroring scenario, it can be realized that the transmitting device 100 responds to the button operation of the user on the remote control 300 without the need for the receiving device 200 to relay.

[0120] The following is an expansion:

[0121] Non-screen mirroring state

[0122] In the non-screen mirroring scenario, both the transmitting device 100 and the receiving device 200 can be located near the remote control 300 and can receive the broadcast signal of the remote control 300. However, the transmitting device 100 is not mirroring the screen to the receiving device 200, and the picture displayed on the receiving device 200 comes from the content provided by the application running on the receiving device 200 and is independent of the picture displayed on the transmitting device 100 and is not a synchronized picture.

[0123] S1. The receiving device 200 is paired with the remote control 300. During the pairing process, the receiving device 200 can obtain the broadcast signal sent by the remote control 300 and determine the broadcast signal transmitted by the remote control 300 as a trusted broadcast signal.

[0124] In addition, the receiving device 200 can also extract specific parameters from the broadcast signal, such as a specific check bit. This specific parameter can be used to identify the trusted broadcast signal of the receiving device 200 and can be called the first parameter. In other words, the first parameter is the identification information (or called a marker) of the trusted broadcast signal. Specifically, subsequently, after receiving any broadcast signal, the receiving device 200 can determine whether the received broadcast signal is a trusted broadcast signal according to whether it carries the first parameter. If it carries the first parameter, it can be determined that the received broadcast signal is a trusted broadcast signal and comes from the supporting remote control of the receiving device 200; if it does not carry the first parameter, it can be determined that the received broadcast signal is not a trusted broadcast signal. The trusted broadcast signal will be processed by the receiving device 200, and the untrusted broadcast signal will be discarded by the receiving device 200.

[0125] The pairing shown in S1 can be completed before the receiving device 200 leaves the factory. The receiving device 200 has already determined the first parameter for identifying the trusted broadcast signal when it leaves the factory, so that the user does not need to pair the receiving device 200 and the remote control 300 again.

[0126] S2. The remote control 300 receives the input from the user acting on the first button, such as pressing the first button. The first button can be any button on the remote control 300, such as the power button, the confirmation button, the left direction button, etc.

[0127] S3. In response to the input from the user acting on the first button, the remote control 300 can send out a first broadcast signal. Correspondingly, both the sending device 100 and the receiving device 200 can receive the first broadcast signal.

[0128] The first broadcast signal can carry the device identifier of the remote control 300, which is used to indicate that the first broadcast signal comes from the supporting remote control 300 of the receiving device 200. In this way, the receiving device 200 can compare the device identifier carried in the first broadcast signal with the device identifier obtained in the previous step S1. If they are the same, it is determined that the first broadcast signal comes from its own supporting remote control 300, and then it decides to respond to the first broadcast signal and perform corresponding processing.

[0129] Specifically, the remote control 300 can send the first broadcast signal through wireless communication technologies such as Bluetooth and WLAN direct connection.

[0130] The first broadcast signal can carry the key value of the first button. When the user presses the first button, the remote control 300 can first determine the key value corresponding to the first button from the key value library. Exemplarily, the key value library can be as shown in Table 1 below:

[0131] Table 1

[0132] Button Key value Power button 0x01 Confirm button 0x02 Back button 0x03 Home button 0x04 Menu button 0x05 Volume up button 0x06 Volume down button 0x07 …… ……

[0133] As can be seen from Table 1, the key value corresponding to the power key is "0x01", the key value corresponding to the confirmation key is "0x02", the key value corresponding to the return key is "0x03", the key value corresponding to the home key is "0x04", the key value corresponding to the menu key is "0x05", the key value corresponding to the volume up key is "0x06", and the key value corresponding to the volume down key is "0x07". Table 1 is only used to explain the embodiments of the present application and should not constitute a limitation. The key values of the respective buttons on the remote control can be determined according to actual application requirements.

[0134] S4. After receiving the first broadcast signal sent by the remote control 300, after the receiving device 200 determines that the sending device 100 is not casting a screen to the receiving device 200, the receiving device 200 processes the first broadcast signal.

[0135] The specific implementation of the receiving device 200 processing the first broadcast signal may include: extracting the key value of the first button from the first broadcast signal, determining the user instruction corresponding to the first button according to the button mapping relationship, and executing the user instruction. The button mapping relationship may record the user instructions mapped by the buttons on the remote control 300 in various application scenarios. Exemplarily, the button mapping relationship may be as shown in Table 2:

[0136] Table 2

[0137]

[0138] As shown in Table 2, the key value corresponding to the power button is "0x01", which can be mapped to the user instruction "turn off / turn on the display screen" in the "game scenario", "video playback scenario", and "desktop scenario". The key value corresponding to the confirmation button is "0x02", which can be mapped to the user instruction "launch game skills" in the "game scenario", "play / pause" in the "video playback scenario", and "click on the focused control" in the "desktop scenario". The key value corresponding to the back button is "0x03", which can be mapped to the user instruction "exit the game" in the "game scenario", "exit video playback" in the "video playback scenario", and may not be mapped to any user instruction (NULL) in the "desktop scenario". The key value corresponding to the home button is "0x04", which can be mapped to the user instruction "display the home page" in any game scenario. The key value corresponding to the menu button is "0x05", which can be mapped to the user instruction "display the game menu" in the "game scenario", "display the video playback menu" in the "video playback scenario", and may not be mapped to any user instruction (NULL) in the "desktop scenario". The key value corresponding to the volume up button is "0x06", which can be mapped to the user instruction "increase volume" in any game scenario. The key value corresponding to the volume down button is "0x07", which can be mapped to the user instruction "decrease volume" in any game scenario.

[0139] As can be seen from Table 2, the same button may be mapped to different user instructions in different application scenarios, reflecting different user operation intentions. Table 2 is only used to explain the embodiments of the present application and should not constitute a limitation. The button mapping relationship can be determined according to actual application requirements.

[0140] Taking Table 2 as an example, if the first button pressed by the user is the power button, then after receiving the first broadcast signal, the receiving device 200 can execute: turn on or turn off the display screen. Another example, if the first button pressed by the user is the back button and the application scenario is the "game scenario", then after receiving the first broadcast signal, the receiving device 200 can execute: exit the game. Another example, if the first button pressed by the user is the confirmation button and the application scenario is the "video playback scenario", then after receiving the first broadcast signal, the receiving device 200 can execute: play or pause the video. The examples are only used to explain the embodiments of the present application and should not constitute a limitation.

[0141] In the embodiments of the present application, the receiving device 200 may store the key mapping relationship of the supporting remote control 300. After receiving the first broadcast signal sent by the remote control 300, the receiving device 200 may first identify the application scenario where the receiving device 200 is located, and then determine the user instruction corresponding to the first key from the key mapping relationship according to the key value of the first key carried in the first broadcast signal and the identified application scenario, and then execute the user instruction. Specifically, after determining the user instruction corresponding to the first key according to the key mapping relationship, the receiving device 200 may allocate the user instruction to the foreground application on the receiving device 200, and the foreground application executes the user instruction. The execution steps for implementing the user instruction are predefined in the source code of the foreground application. Here, the foreground application may refer to the application displayed on the receiving device 200 when the user presses the first key on the remote control 300. In the split-screen scenario, the foreground application may more specifically refer to the application displayed in the split-screen window that obtains the focus.

[0142] Specifically, after receiving the first broadcast signal sent by the remote control 300, the receiving device 200 may first take a screenshot, and then perform image analysis on the screenshot image to identify the application scenario where the receiving device 200 is located when the first key is pressed. The multiple application scenarios in the key mapping relationship may include some or all of the application scenarios provided by the application programs on the receiving device 200.

[0143] S5. After receiving the first broadcast signal sent by the remote control 300, the transmitting device 100 determines that the transmitting device 100 is not casting a screen to the receiving device 200, and then abandons processing the first broadcast signal.

[0144] Specifically, the conditions for determining that the transmitting device 100 is not casting a screen to the receiving device 200 may include any one of the following: no screen mirroring communication connection is established between the transmitting device 100 and the receiving device 200, or although the screen mirroring communication connection is established, the transmitting device 100 does not transmit a screen mirroring stream to the receiving device 200 through the screen mirroring communication connection, or although the screen mirroring communication connection is established, no screen mirroring picture is displayed on the receiving device 200. The implementation method described in the last item is applicable to the receiving device 200 to determine that the transmitting device 100 is not casting a screen to the receiving device 200.

[0145] It can be seen from S1 - S5 that in the non-screen mirroring scenario, when the user presses a key on the remote control 300, the broadcast signal sent by the remote control 300 is processed by the receiving device 200 and ignored by the transmitting device 100.

[0146] Enter the screen mirroring state

[0147] To trigger the sending device 100 and the receiving device 200 to enter the screen mirroring state, the user can input a screen mirroring instruction on the sending device 100 or the receiving device 200. For example, the user clicks on the screen mirroring option displayed on the screen. In response to this, the sending device can activate the first communication module to discover the receiving device and authenticate with the receiving device, and finally establish a screen mirroring communication connection.

[0148] S6. The sending device 100 and the receiving device 200 establish a screen mirroring communication connection.

[0149] When the receiving device and the sending device establish screen mirroring, the receiving device 200 can notify the first parameter to the sending device 100, so that the sending device 100 can also trust the broadcast signal sent by the remote control 300, and complete the transfer of the trust relationship. After the sending device 100 enters the screen mirroring state and receives a trusted broadcast signal carrying the first parameter, the sending device 100 will process it and no longer filter it out.

[0150] Among them, one implementation of notifying the first parameter may include: when the receiving device 200 determines the first parameter carried in the trusted broadcast signal during pairing, it synchronizes it to a specific cloud server. The sending device 100 and the receiving device 200 can log in to the cloud server with the same user account, and then the cloud server pushes the first parameter to the sending device 100. Another implementation of notifying the first parameter may include: notifying the first parameter to the sending device 100 through the short-range communication connection established between the receiving device 200 and the sending device 100. The short-range communication connection can be, for example, a Bluetooth communication, a Wi-Fi communication, or other communication connections. Another implementation of notifying the first parameter may include: the receiving device 200 waits until it discovers the sending device 100 or is discovered by the sending device 100 before notifying the first parameter to the sending device 100.

[0151] In the embodiment of the present application, the receiving device 200 can be restricted to supporting only one remote control. When the remote control paired with the receiving device 200 is updated, the receiving device 200 can update the first parameter used to identify the trusted broadcast signal, determine the parameter carried by the broadcast signal emitted by the new remote control as the new first parameter, and can actively or passively synchronize the new first parameter to the sending device 100, so that the sending device 100 trusts the new remote control.

[0152] S7. The sending device 100 generates a screen mirroring stream.

[0153] In the scenario of mirror screen projection, the projection stream may include the screen stream of the sending device 100. The sending device 100 can record the screen while forming a screen stream based on the recorded screen content. In the scenario of application projection, the projection stream may include the application stream of the projected application on the sending device 100. When the projected application outputs content, the sending device 100 can form an application stream based on this content.

[0154] S8. Through this projection communication connection, the sending device 100 can send the projection stream to the receiving device 200. Correspondingly, the receiving device 200 can receive the projection stream sent by the sending device 100.

[0155] S9. The receiving device 200 displays the projection screen based on the received projection stream.

[0156] Specifically, the sending device 100 can generate the projection stream according to the projection content and send the projection stream to the receiving device 200 at the same time. Correspondingly, the receiving device 200 can receive the projection stream and display the projection screen at the same time. In the mirror screen projection scenario, the projection content may include the screen content, and the projection stream may refer to the screen stream. In the application projection scenario, the projection content may include the display content of a certain application, and the projection stream may refer to the application stream.

[0157] S7 - S9 are continuously executed during the projection to achieve real - time synchronization of the projection.

[0158] S10. The receiving device 200 sends the key mapping relationship to the sending device 100.

[0159] S11. The remote control 300 receives the input of the user acting on the first key, such as pressing the first key. The first key can be any key on the remote control 300, such as the power key, the confirmation key, the left arrow key, etc.

[0160] S12. In response to the input of the user acting on the first key, the remote control 300 can send out a first broadcast signal. Correspondingly, both the sending device 100 and the receiving device 200 can receive the first broadcast signal.

[0161] Regarding the implementation details of S11 to S12, reference can be made to the relevant content of S2 to S3 above, which will not be elaborated here. After the remote control 300 emits the first broadcast signal, the sending device 100 and the receiving device 200 can receive the first broadcast signal simultaneously or almost simultaneously.

[0162] After receiving the first broadcast signal, both the sending device 100 and the receiving device 200 can first determine that the first broadcast signal is a trusted broadcast signal. If a received broadcast signal is not a trusted broadcast signal, the sending device 100 and the receiving device 200 can directly abandon processing this broadcast signal.

[0163] After receiving the first broadcast signal sent by the remote controller 300, the receiving device 200 determines that after the sending device 10 casts the screen to the receiving device 200, it abandons processing the first broadcast signal.

[0164] S14. After receiving the first broadcast signal sent by the remote controller 300, the sending device 100 determines that after the sending device 100 casts the screen to the receiving device 200, it processes the first broadcast signal.

[0165] Specifically, when the sending device 100 casts the screen to the receiving device 200, it may include: a screen mirroring communication connection is established between the sending device 100 and the receiving device 200, the sending device 100 transmits a screen mirroring stream to the receiving device 200 through the screen mirroring communication connection, and the receiving device 200 displays a screen mirroring image based on the received screen mirroring stream.

[0166] Such as Figure 5 shown, the specific implementation of the sending device 100 processing the first broadcast signal may include the following steps:

[0167] A1. Extract the key value of the first button from the first broadcast signal, and determine whether the first button exists in the button mapping relationship based on the key value.

[0168] A2. Determine whether the first application providing the screen mirroring content belongs to the whitelist.

[0169] A3. If the first button exists in the button mapping relationship and the first application is a whitelist application, execute the user instruction corresponding to the first button.

[0170] Specifically, after receiving the first broadcast signal sent by the remote controller 300, the sending device 100 may first take a screenshot, and then perform image analysis on the screenshot image to identify the application scenario where the sending device 100 is located when the first button is pressed. Then, the sending device 100 may determine the user instruction corresponding to the first button from the button mapping relationship according to the key value of the first button carried in the first broadcast signal and the identified application scenario, and then execute the user instruction.

[0171] Not limited to performing image analysis on the screenshot image, the method of identifying the application scenario may also include: obtaining the application identifier of the first application providing the screen mirroring content, such as the application package name, and determining the application scenario according to the application identifier. The application identifier may indicate the type of the application program, thereby reflecting the application scenario it provides. For example, the application package name of the application program providing the screen mirroring content during screen mirroring indicates that the application program is a video application program, so the application scenario can be determined as a video playback scenario. Another example, the application package name of the application program providing the screen mirroring content during screen mirroring indicates that the application program is a game application, so the application scenario can be determined as a game scenario. The examples are only used to explain the embodiments of the present application and should not constitute a limitation.

[0172] After determining the user instruction corresponding to the first button according to the button mapping relationship, the sending device 100 may allocate the user instruction to the first application, and the first application executes the user instruction. The execution steps for implementing the user instruction are predefined in the source code of the first application. The first application may be an application that provides screen mirroring content on the sending device 100, and may include system applications and non-system applications. In the mirror screen mirroring scenario, the first application may be the foreground application on the sending device 100. Here, the foreground application may refer to the application displayed on the sending device 100 when the user presses the first button on the remote control 300. In the split screen scenario, the foreground application may more specifically refer to the application displayed in the split screen window that obtains the focus. In the application mirroring scenario, the first application is the application program that provides the mirroring content. For example, Figure 1B the video playback application in [], the display content of which is generally not full-screen displayed on the sending device, and can be displayed in a small window floating, or even can be not displayed and only the floating ball, etc. are retained to display the mirroring prompt information. The embodiments of the present application do not limit how the sending device 100 displays the mirrored application in the application mirroring scenario, and can be set according to actual application requirements.

[0173] The button mapping relationship stored in the sending device 100 may come from the receiving device 200 and can be used for the button mapping relationship used by the receiving device 200 for the remote control 300, as shown in Table 2 above. Specifically, as shown in S10, the receiving device 200 may send the button mapping relationship to the sending device 100 through the screen mirroring communication connection. The receiving device 200 may also send the button mapping relationship to the sending device 100 through other communication connections between the receiving device 200 and the sending device 100 before the screen mirroring communication connection is established. In this way, the receiving device 200 can send the button mapping relationship to the sending device 100 before screen mirroring, and can prepare the button mapping relationship in advance, so that the sending device 100 can promptly respond to the user operation on the remote control 300 at the initial stage when just entering the screen mirroring state, and improve the user's screen mirroring control experience.

[0174] In the above step A2, the whitelist may include: identification information of one or more application programs on the sending device 100, such as the application package name, application ID, etc. of the application program. In some embodiments, the applications in the whitelist may be the applications for which the provided application scenarios exist in the key mapping relationship. In this way, it can be ensured that the sending device 100 can find the user instruction mapped by the first key in the scenario provided by the whitelist application from the key mapping relationship, ensuring the realization of screen mirroring control. Because the application scenarios provided by some applications on the sending device 100 may not be covered by the key mapping relationship and do not belong to the whitelist applications, they need to be excluded. Furthermore, the whitelist applications may also be the applications that allow users to control screen mirroring through the remote control 300. In this way, the screen mirroring control permissions of different applications can be flexibly set to ensure that the screen mirroring of some application programs can only be controlled by the sending device 100, while the screen mirroring of some other application programs can also be controlled by the remote control 300.

[0175] It can be seen from S6 - S12 that in the screen mirroring scenario, the sending device 100 responds to the key operation of the user on the remote control 300 without the need for the receiving device 200 to relay.

[0176] Exit the screen mirroring state

[0177] To trigger the sending device 100 and the receiving device 200 to exit the screen mirroring state, the user can input an end screen mirroring instruction on the sending device 100 or the receiving device 200.

[0178] S15. The sending device 100 disconnects the screen mirroring communication connection with the receiving device 200.

[0179] When exiting the screen mirroring state, the sending device 100 stops sending the screen mirroring stream to the receiving device 200; and, the sending device 100 no longer regards the broadcast signal of the remote control 300 as a trusted broadcast signal. Of course, the sending device 100 can retain the mark of the trusted broadcast signal, that is, the aforementioned first parameter, in data storage. When entering the screen mirroring state with the receiving device 200 next time, it trusts the broadcast signal sent by the remote control 300 based on the retained first parameter.

[0180] Thereafter, as shown in S16 - S19, once the user operates the keys on the remote control 300 and the receiving device 200 emits a broadcast signal, the sending device 100 no longer processes the broadcast signal sent by the remote control 300, but instead the receiving device 200 processes the broadcast signal, enabling the user to control the receiving device 200 through the remote control 300. For details, reference can be made to the above S1 - S5, which will not be elaborated here.

[0181] The embodiments of the present application further provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps performed by the sending device or the receiving device in the above various method embodiments can be implemented.

[0182] The embodiments of the present application further provide a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps performed by the sending device or the receiving device in the above various method embodiments.

[0183] The embodiments of the present application further provide a chip system. The chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps performed by the sending device or the receiving device in any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0184] The term "user interface (UI), simply referred to as interface" in the specification and drawings of the present application is a media interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application program is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on a terminal device and finally presented as content recognizable by the user, such as controls like pictures, text, buttons, etc. A control (also known as a widget) is a basic element of the user interface. Typical controls include a toolbar, a menu bar, a text box, a button, a scrollbar, pictures, and text. The attributes and content of the controls in the interface are defined through tags or nodes. For example, XML passes through <textview> 、

[0185] <imgview> 、 <videoview>Nodes such as these are used to specify the controls included in the interface. One node corresponds to one control or property in the interface, and after being parsed and rendered, the node presents visible content to the user. In addition, in the interfaces of many applications, such as hybrid applications, there are usually also web pages. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hyper text markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as recognizable content to the user by a browser or a web page display component similar to the browser's function. The specific content included in the web page is also defined by tags or nodes in the web page source code. For example, HTML uses 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0186] A commonly used form of the user interface is the graphical user interface (GUI), which refers to the user interface related to computer operations displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include visible interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc.

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0188] Those of ordinary skill in the art can understand all or part of the processes in the above method embodiments. These processes can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disks, or optical discs, etc., which can store program codes of various types.

[0189] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A screen mirroring control method, characterized in that, The method is applied to a screen mirroring control system, which includes a sending device, a receiving device, and a remote control device; The method includes: Detecting that the user operates a first button on the remote control device, and the remote control device emits a first broadcast signal outward; If the sending device is mirroring to the receiving device, after receiving the first broadcast signal, the receiving device abandons processing the first broadcast signal, and the sending device processes the first broadcast signal; If the sending device is not mirroring to the receiving device, after receiving the first broadcast signal, the receiving device processes the first broadcast signal, and the sending device abandons processing the first broadcast signal.

2. The method according to claim 1, wherein The sending device processing the first broadcast signal includes: The sending device extracts the key value of the first button from the first broadcast signal, and determines whether the first button exists in the button mapping relationship corresponding to the remote control device based on the key value; The sending device determines whether a first application providing the screen mirroring content belongs to a whitelist; If the first button exists in the button mapping relationship and the first application belongs to the whitelist, the sending device executes the user instruction corresponding to the first button.

3. The method according to claim 2, wherein The button mapping relationship is stored on the receiving device; The receiving device processing the first broadcast signal includes: The receiving device identifies the application scenario where the receiving device is located, Determines the user instruction corresponding to the first button according to the button mapping relationship and the application scenario where the receiving device is located; Executes the user instruction.

4. The method according to claim 3, wherein Before the sending device mirrors to the receiving device, the method further includes: the receiving device sends the button mapping relationship to the sending device.

5. The method according to any one of claims 1-4, characterized in that The sending device mirroring to the receiving device includes: the sending device establishes a screen mirroring communication connection with the receiving device, the sending device transmits a screen mirroring stream to the receiving device through the screen mirroring communication connection, and the receiving device displays a screen mirroring picture based on the screen mirroring stream.

6. The method according to any one of claims 1-5, characterized in that, The sending device not mirroring to the receiving device includes: the sending device and the receiving device do not establish a screen mirroring communication connection, or the screen mirroring communication connection is established but the sending device does not transmit a screen mirroring stream to the receiving device through the screen mirroring communication connection, or the screen mirroring communication connection is established but the screen mirroring picture is not displayed on the receiving device.

7. The method according to claim 5 or 6, characterized in that, The screen mirroring includes mirror screen mirroring, and the screen mirroring stream includes a screen stream; the first application is the foreground application on the sending device, and the screen mirroring picture displayed on the receiving device is the same as the picture displayed on the sending device.

8. The method according to claim 5 or 6 or 7, characterized in that The screen mirroring includes application screen mirroring, and the screen mirroring stream includes an application stream generated based on the screen mirroring content provided by the first application; The first application does not perform full-screen display on the sending device, and the non-full-screen display includes: displaying through a floating window, or not displaying.

9. The method according to any one of claims 1-8, characterized in that It further includes: The receiving device is paired with the remote control device. The receiving device receives the broadcast signal transmitted by the remote control device during the pairing and determines the broadcast signal as a trusted broadcast signal. The broadcast signal carries a first parameter, and the first parameter is used to identify the trusted broadcast signal.

10. The method according to claim 9, wherein Before the receiving device processes the first broadcast signal, it further includes: the receiving device determines that the first broadcast signal is the trusted broadcast signal.

11. The method according to claim 9 or 10, characterized in that, Before the transmitting device processes the first broadcast signal, it further includes: the transmitting device determines that the first broadcast signal is the trusted broadcast signal.

12. The method according to claim 11, wherein After the transmitting device and the receiving device establish a screen mirroring communication connection, the method further includes: the receiving device notifies the transmitting device of the first parameter.

13. A screen mirroring control method, characterized in that, The method is applied to a transmitting device in a screen mirroring control system. The screen mirroring control system includes: the transmitting device, the receiving device, and the remote control device; The method includes: The transmitting device receives a first broadcast signal sent by the remote control device; the first broadcast signal is transmitted outward by the remote control device when the user operates a first button on the remote control device. The transmitting device determines whether the transmitting device is screen mirroring to the receiving device. If so, the transmitting device processes the first broadcast signal; if not, the transmitting device abandons processing the first broadcast signal; the receiving device is used to abandon processing the first broadcast signal when the transmitting device is screen mirroring to the receiving device, and process the first broadcast signal when the transmitting device is not screen mirroring to the receiving device.

14. The method according to claim 13, wherein The transmitting device processes the first broadcast signal, including: The transmitting device extracts the key value of the first button from the first broadcast signal and determines whether the first button exists in the key mapping relationship corresponding to the remote control device based on the key value. The transmitting device determines whether the first application providing the screen mirroring content belongs to the whitelist. If the first button exists in the key mapping relationship and the first application belongs to the whitelist, the transmitting device executes the user instruction corresponding to the first button. Wherein, the key mapping relationship is used to record the user instructions mapped by the buttons on the remote control device in multiple application scenarios; the whitelist includes: the identification information of one or more application programs on the transmitting device, and the application scenarios provided by the one or more applications exist in the key mapping relationship.

15. The method according to claim 14, characterized in that, The method further includes: before the transmitting device is screen mirroring to the receiving device, the receiving device sends the key mapping relationship to the transmitting device.

16. The method according to any one of claims 13-15, characterized in that, The transmitting device is screen mirroring to the receiving device, including: the transmitting device and the receiving device establish a screen mirroring communication connection, and the transmitting device transmits a screen mirroring stream to the receiving device through the screen mirroring communication connection.

17. The method according to any one of claims 13 - 16, characterized in that, The transmitting device is not screen mirroring to the receiving device, including: the transmitting device and the receiving device do not establish a screen mirroring communication connection, or the screen mirroring communication connection is established but the transmitting device does not transmit a screen mirroring stream to the receiving device through the screen mirroring communication connection.

18. The method according to claim 16 or 17, characterized in that, The screen mirroring includes mirror screen mirroring, and the screen mirroring stream includes a screen stream; the first application is the foreground application on the sending device, and the screen mirroring image displayed on the receiving device is the same as the image displayed on the sending device.

19. The method according to claim 16 or 17 or 18, characterized in that The screen mirroring includes application screen mirroring, and the screen mirroring stream includes an application stream generated based on the screen mirroring content provided by the first application; The first application is not displayed full screen on the sending device, and the non-full-screen display includes: being displayed through a floating window, or not being displayed.

20. The method according to any one of claims 13-19, characterized in that, It further includes: The sending device receives a first parameter notified by the receiving device, where the first parameter is used to identify the trusted broadcast signal, and the trusted broadcast signal carrying the first parameter is transmitted by the remote control device paired with the receiving device.

21. The method according to claim 20, wherein, Before the sending device processes the first broadcast signal, it further includes: the sending device determines that the first broadcast signal carries the first parameter.

22. A screen mirroring control method, characterized in that, The method is applied to a receiving device in a screen mirroring control system, and the screen mirroring control system includes: the receiving device, the sending device, and the remote control device; The method includes: The receiving device receives a first broadcast signal sent by the remote control device; the first broadcast signal is transmitted by the remote control device when the user operates the first button on the remote control device. The receiving device determines whether the sending device is mirroring to the receiving device. If so, the receiving device abandons processing the first broadcast signal; if not, the receiving device processes the first broadcast signal; the sending device is used to process the first broadcast signal when the sending device is mirroring to the receiving device and abandon processing the first broadcast signal when the sending device is not mirroring to the receiving device.

23. The method according to claim 22, wherein A key mapping relationship is stored on the receiving device, and the key mapping relationship is used to record the user instructions mapped by the keys on the remote control device in multiple application scenarios. The receiving device processes the first broadcast signal, including: the receiving device first identifies the application scenario where the receiving device is located, then determines the user instruction corresponding to the first button in the application scenario where the receiving device is located from the key mapping relationship, and executes the determined user instruction.

24. The method according to claim 23, wherein Before the sending device mirrors to the receiving device, the method further includes: the receiving device sends the key mapping relationship to the sending device.

25. The method according to any one of claims 22-24, characterized in that, The sending device mirroring to the receiving device includes: establishing a screen mirroring communication connection between the sending device and the receiving device, the sending device transmitting a screen mirroring stream to the receiving device through the screen mirroring communication connection, and the receiving device displaying a screen mirroring image based on the screen mirroring stream.

26. The method according to any one of claims 22-25, characterized in that, The sending device not mirroring to the receiving device includes: no screen mirroring communication connection is established between the sending device and the receiving device, or the screen mirroring communication connection is established but the sending device does not transmit a screen mirroring stream to the receiving device through the screen mirroring communication connection, or the screen mirroring communication connection is established but no screen mirroring image is displayed on the receiving device.

27. The method according to claim 25 or 26, characterized in that, The screen mirroring includes mirror screen mirroring, and the screen mirroring stream includes a screen stream; the first application is the foreground application on the sending device, and the screen mirroring image displayed on the receiving device is the same as the image displayed on the sending device.

28. The method according to claim 25 or 26 or 27, characterized in that, The screen mirroring includes application screen mirroring, and the screen mirroring stream includes an application stream generated based on the screen mirroring content provided by the first application; The first application does not perform full-screen display on the sending device, and the non-full-screen display includes: displaying through a floating window or not displaying.

29. The method according to any one of claims 22-28, characterized in that, It further includes: The receiving device is paired with the remote control device. The receiving device receives the broadcast signal transmitted by the remote control device during the pairing and determines the broadcast signal as a trusted broadcast signal. The broadcast signal carries a first parameter, and the first parameter is used to identify the trusted broadcast signal.

30. The method according to claim 29, characterized in that, Before the receiving device processes the first broadcast signal, it further includes: the receiving device determines that the first broadcast signal is the trusted broadcast signal.

31. The method according to claim 29 or 30, characterized in that, It further includes: The receiving device notifies the first parameter to the sending device.

32. An electronic device, characterized in that, It includes: A communication module, a display screen, a processor, and a memory, where the communication module, the display screen are coupled to the processor, and the memory is coupled to the processor; Among them, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device is caused to execute the method described in any one of claims 12-21 or claims 22-31.

33. A computer-readable storage medium, comprising instructions, characterized in that, When the instructions run on the electronic device, the electronic device is caused to execute the method described in any one of claims 12-21 or claims 22-31.

34. A screen mirroring control system, characterized in that, It includes: A remote control device, a sending device, and a receiving device. The sending device is the sending device in the method described in any one of claims 12-21 or claims 22-31, and the receiving device is the receiving device in the method described in any one of claims 12-21 or claims 22-31. The remote control device is the paired remote control device of the receiving device.

Images