Cross-platform instruction switching method and intelligent device

Abstract

The invention discloses a cross-platform instruction switching method and intelligent equipment applying the method, the intelligent equipment, such as an intelligent screen with a display screen, is connected with a data source through a communication module and displays display content received from the data source, and the intelligent equipment further comprises an input module for providing an input interface, and the control device can be connected with intelligent equipment through the input interface. In the cross-platform instruction switching method, an operation system in the intelligent equipment receives an instruction which is generated by a control device and triggers a menu interface, namely, the menu interface is started, options of at least one data source are provided, and then received content is displayed on a display screen according to the selected data source. At the moment, a signal channel is established between the intelligent equipment and the data source, and the instruction generated by the control device is transmitted to the data source through the signal channel.

Description

Technical Field

[0001] The specification discloses a technology for switching input control between devices, and in particular, refers to a method for implementing cross-platform switching of input control instructions through a menu and a smart device applying the method. Background Art

[0002] An intelligent device can be defined as a smart screen with audio and video processing, image display, input and output control, and networking functions. An operating system, such as Android, runs on the intelligent device. TM OS) and has an input interface that can be connected to a control device such as a remote control, keyboard or mouse by wire or wirelessly, so that the user can operate the control device to control the smart device. Smart devices also have communication interfaces for transmitting audio and video data, such as the universal serial bus (USB), high-definition multimedia interface (HDMI) TM ) and DisplayPort (DP), and can also be equipped with wireless communication technology to connect to other external devices, such as audio and video devices or computer devices, and can also receive audio and video data transmitted by external audio and video sources.

[0003] However, in the prior art, the control device connected to the smart device is driven by a driver executed by the operating system in the smart device and can only be used to control the smart device itself, but cannot control other external devices connected to the smart device.

[0004] On the other hand, on the smart device, through the operation of the operating system, a user interface is provided on the smart device to provide options for multiple audio and video sources, and the options may include TV channels (TV), HDMI TM , USB and DP, or other wireless communication protocols (such as Airplay TM 、Miracast TM When the smart device displays the image received via one of the communication interfaces according to the selection, the control device connected to the smart device has not yet provided a method to switch from the smart device to operate the display device to display the image provided by the connected external device. Summary of the invention

[0005] In order to achieve the purpose of controlling another device using only a control device connected to a smart device, the present application proposes a cross-platform switch command method and a smart device, one of which is to use a user interface to switch the input data source. At this time, a signal channel is established so that the control device originally connected to the smart device can also smoothly control the content of another data source.

[0006] The main components of the smart device, in addition to the processing circuit for processing image data, include one or more communication modules that provide a communication interface to connect to the data source, a display screen for displaying the display content received from the data source, and an input module that provides an input interface so that the control device can connect to the smart device through this input interface. The control device can be a remote control, mouse or keyboard for controlling the smart device.

[0007] A method for executing a cross-platform switch command in an operating system running in a smart device, wherein the process includes receiving a command generated by a control device to trigger a menu interface, that is, opening the menu interface, the menu interface provides at least one data source option, such as a TV channel, a streaming audio and video source, and an image source, and then receiving content according to the selected data source and displaying it on a display screen. At this time, a signal channel is established between the smart device and the data source, and the command generated by the control device can be transferred to the data source through the signal channel.

[0008] Preferably, the data source is an external device, and the display screen of the external device is projected onto the display screen of the smart device through one of the communication interfaces. The external device can use wireless audio and video transmission technology to project the display screen onto the display screen of the smart device through the communication interface.

[0009] Furthermore, when the external device projects the display image to the smart device, a signal channel is established between the operating system of the external device and the operating system of the smart device.

[0010] Furthermore, a driver program of the control device is executed in the smart device to drive the operation of the control device and interpret the instructions generated by the control device.

[0011] Furthermore, in the method of the cross-platform switch instruction, when the signal channel is turned on, the control device that originally controls the smart device controls the content projected on the display screen from the data source through the signal channel; when the signal channel is closed, the instructions of the control device are only for the control of the smart device.

[0012] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram showing an embodiment of a smart device and its peripheral devices for implementing a method for implementing a cross-platform switch instruction;

[0014] Figure 2 Functional block diagram of an intelligent device and its connecting device showing a method for implementing a cross-platform switch instruction;

[0015] Figure 3 A flowchart of an embodiment of a method for displaying a cross-platform switch instruction; and

[0016] FIG. 4A to FIG. 4C A schematic diagram showing an embodiment of a method for implementing a cross-platform switch instruction using a user interface. DETAILED DESCRIPTION

[0017] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0018] It should be understood that, although the terms "first", "second", "third", etc. may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are mainly used to distinguish one component from another component, or one signal from another signal. In addition, the term "or" used herein may include any one or more combinations of the associated listed items depending on the actual situation.

[0019] The present application proposes a method for cross-platform switch instructions and a smart device using this method, in particular, this is a smart device with a display, which can be defined as a smart screen with audio and video processing, image display, input and output control and networking functions, in which an operating system runs and provides various audio and video programs, such as streaming media programs, and executes a user interface program, providing a menu for the user to select one of the data sources, so that the content displayed by the smart device can be switched to the content input through this data source.

[0020] In addition to processing circuits for processing image data, network data, etc., smart devices are equipped with one or more wired or wireless communication interfaces. The data source connected to the smart device can be an external device through one of the communication interfaces, such as a computer device or a mobile device. In addition to the function of displaying TV channels by itself, the smart device also provides an external device that can project the display screen on the smart device. For example, the external device can image (mirroring) the display screen of the external device on the display of the smart device through a specific audio and video connection port (USB, HDMI, AVI, DVI, DP, etc.) or wireless audio and video transmission technologies such as Miracast, Chromecast and Airplay. This type of wireless audio and video transmission technology realizes the transmission of audio and video content to the smart device using wireless communication technology, among which Miracast uses WiFi Direct to implement the wireless HDMI protocol; the data source can also be an audio and video program executed in the operating system of the smart device, such as Netflix TM ,Youtube TM The display of smart devices can be switched to the content broadcast by such video streaming programs.

[0021] In particular, a control device such as a remote controller, keyboard or mouse is connected to the smart device via an input interface provided on the smart device in a wired or wireless manner, and the control device is operated through a driver under the operating system of the smart device, and a user can input content on the smart device or control the operation of the smart device through these control devices. According to one of the main purposes of the cross-platform switch instruction method and smart device proposed in this application, a user interface is provided so that the control device can cross the smart device and work with an external device connected to the smart device. In the user's experience, the smart device and the external device can be switched across platforms by using only one mouse, keyboard or remote controller.

[0022] Figure 1 A schematic diagram of an embodiment of a smart device and its peripheral devices showing a method for implementing a cross-platform switch instruction.

[0023] The figure shows a smart device 100, which may be a smart TV or display with data processing and communication capabilities, having a display screen for displaying TV content, internal audio and video programs (such as Netflix TM ,Youtube TMThe smart device is provided with a communication interface that can transmit audio and video signals, such as the Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI) and Display Port (DP). Through these communication interfaces, an external device is provided with a corresponding audio and video port to connect to the smart device, so that the smart device can display the display content of the external device in an image manner, or receive audio and video data provided by the external device and then display it on the display of the smart device.

[0024] The smart device 100 can be connected to one or more control devices via wireless or wired means, such as the keyboard 101, mouse 103 and remote control device 105 shown in the figure. In general, the control device is used to control the operation of the smart device 100. When an external device 12 is connected to the smart device 100 via wired or wireless communication means, and then transmits image data through a specific communication interface, the smart device 100 can use the above-mentioned imaging technology to project the display content transmitted by the specific data source (such as the external device 12 in the figure) onto the display screen.

[0025] What is special is that when the user originally operates the control device (101, 103 or 105) to control the smart device 100, when the user wants to switch to the screen of operating the image of the external device 12, for example, using the control device to generate control instructions to control functions such as volume and screen settings, as well as start playing, fast forward and backward, pause, stop playing audio and video content, or switch to other audio and video content, etc., it is necessary to transmit the control instructions through the signal channel established between the operating system in the smart device 100 and the operating system of the external device 12, and transmit the instructions of the control device to the operating system of the external device 12, so as to successfully control the content played by the image.

[0026] According to an embodiment of the method for cross-platform switch command, a user interface program is installed in the operating system of the smart device 100, and a menu interface 110 is started on the smart device 100, which becomes a user interface for realizing cross-platform switch between the smart device 100 and the external device 12. At the beginning, the user operates the control device to control the smart device 100. When the screen imaged by the external device 12 is to be controlled, the menu interface 110 can be opened by a pre-specified command. For example, the key code of the middle button of the mouse 103 can be set as the command to open the menu interface 110. When the operating system of the smart device 100 receives this key code, the menu interface 110 is opened, and the data source options provided therein may include TV channels, at least one streaming audio and video source, and at least one image source, etc., so that the user can select one of the data sources through the control device, and the control right can be smoothly switched to the external device 12.

[0027] For further reference Figure 2 The functional block embodiment diagram of the smart device and its connected devices for implementing the method of cross-platform switch instructions shown in the figure respectively shows the functional modules required for implementing the method of cross-platform switch instructions in the smart device (first device 21) and one or more external devices connected (represented by the second device 22), wherein the functional modules include functional modules implemented by software means, and may also include functional modules implemented by the collaboration of software and hardware circuits.

[0028] The first device 21 is a smart device with a display, which has data processing and communication capabilities, including a processor, a memory (not shown in the figure) and a communication circuit, wherein a first operating system 211 is run, and software services such as a user interface 212, an application 213 and a driver 214 are executed through the software program therein, wherein the user interface 212 provides a menu including multiple data source options, and the application 213 can be a TV playback program or a specific streaming audio and video playback program, etc. The driver 214 is used to drive the operation of the control device 23, to interpret the information and instructions generated by the control device 23, and to execute cross-platform switch instructions.

[0029] The first device 21 is also provided with a communication module 215 and an input module 216. The communication module 215 includes a communication interface and a communication circuit for processing communication protocols and communication data, and is connected to at least one data source in a wireless or wired manner via the communication interface, which may be a specific video streaming platform or an external device as shown in the attached figure; the input module 216 includes a wired or wireless interface for connecting to various control devices 23 and a circuit for processing input signals.

[0030] The first device 21 is provided with a control device 23, such as a remote control, a mouse, a keyboard or other input device, and multiple control devices can be configured at the same time. The control device 23 is connected to the first device 21 by wireless or wired means to control the operation of the first device 21. Taking the playing of TV channel content as an example, the remote control can be used to control the volume, picture setting, channel switching, etc. If a specific audio and video file is played, it can also control various functions of audio and video playback. Further, by Figure 1 In the displayed menu interface, the user can operate the control device 23 to select one of the data sources to switch to the second device 22 projected on the display screen of the first device 21, wherein a signal channel is established so that the control device 23 connected to the first device 21 can smoothly control the audio and video content played by the second device 22.

[0031] The second device 22 is connected to the communication module 215 of the first device 21 by wireless or wired means through the communication module 225. The second device 22 is a computer device with data processing and communication capabilities, in which a second operating system 221 is running. The second operating system 221 executes application programs 223 through software programs therein, such as audio and video programs, presentation programs, or any application programs that need to project images to the first device 21.

[0032] In order to enable the instructions generated by the control device 23 that originally controls the first device 21 (referring to a smart device with a display screen) to be transferred to control the second device 22 (referring to a data source, such as an external device) to achieve the technical purpose of cross-platform switch instructions, the first operating system 211 can establish the above-mentioned signal channel with the second operating system 221 of the second device 22, and transmit instructions or information in a specific encoding manner in the signal channel, and can be interpreted by the operating systems of the two devices under a specific communication protocol. The purpose of the signal channel is to enable the instructions that originally control the first device 21 to control the second device 22 that projects the display content on the first device 21 through a specific communication protocol. The control instructions are mainly transmitted to the application in the second device 22 to control the displayed content.

[0033] For example, in the first device 21, the user originally operates the control device 23 to operate the first device 21. When the user wants to control the second device 22 to project the content on the first device 21, a signal channel can be established between the first device 21 and the second device 22, so that the control instruction generated by the control device 23 can be transmitted to the second operating system 221 of the second device 22 through the signal channel, so that the second operating system 221 processes the control instruction and generates an instruction to control the content projected on the first device 21.

[0034] For related processes, please refer to Figure 3 A flow chart of an embodiment of a method for executing a cross-platform switch instruction is shown.

[0035] Before the method is run, the relevant system settings are as follows Figure 2In the embodiment shown, a control device is connected to a first device 21, and the first device 21 runs a first operating system 211, wherein a driver 214 for driving the control device 23 is run, and a user interface 212 is started by a software program, which is displayed on the display screen of the first device 21. The user interface 212 implements a display interface for displaying a picture and a menu interface for providing data source options, wherein at least one data source option is provided, and the actual implementation may include multiple data source options. An application 213 is also provided, which may be a TV playback program, a streaming video playback program, etc. The first device 21 is connected to a second device 22 by wireless or wired communication means, and the second device 22 runs a second operating system 221, wherein an application 223 such as a video playback program, a presentation program, or others, and the screen after executing the application 223 can be projected on the display screen of the first device 21 in an image manner.

[0036] describe Figure 3 The displayed process can also refer to FIG. 4A to FIG. 4C The diagram is a schematic diagram of an embodiment of a method for implementing a cross-platform switch instruction using a user interface.

[0037] Figure 4A The smart device 40 is shown. The operating system is running in the smart device 40. A user interface is provided for displaying to the user, including a display interface and a menu interface. In addition to providing options for data sources, the user interface program running the menu interface can also be used to intercept instructions generated by the control device and determine the channel for forwarding the control instructions according to the menu selection. The smart device 40 shown in the icon is connected to a control device 403. The user can operate the control device 403 to control the operation of the smart device 40. For example, when playing TV channel content, it can be used to switch channels, control volume, adjust display settings, etc. At the same time, the method of implementing cross-platform switch instructions in the smart device 40 is implemented by software, firmware running in the circuit, or hardware.

[0038] In particular, the operating system of the smart device 40 sets an instruction for opening the menu interface 401. For example, a button on the control device 403 can be set as a trigger button for opening the menu interface 401, such as the middle button of the mouse, one of the function buttons on the keyboard, or a combination key formed by a combination of multiple buttons (such as Alt+0), or a button on the remote control. In another way, the menu interface 401 can also be opened by moving the mouse cursor to a certain position on the display screen.

[0039] When the smart device 40 is playing content from a specific data source, the menu interface 401 can be stored in a position by default to avoid affecting the displayed content. If the user wants to switch to another data source, the input signal for switching the data source can be generated by the method set by the system in the smart device 40, such as the key code generated by the control device. The operating system of the smart device 40 receives this input signal (step S301), determines it as a pre-set instruction, and triggers the user interface (step S303). In addition to displaying the screen, the menu interface is also opened, in which at least one data source option is displayed (step S305). For example, Figure 4B As shown, the originally hidden menu interface 401 is opened, that is, displayed as a pop-up menu interface 402.

[0040] The pop-up menu interface 402 may provide multiple data source options, including returning to the home page (Home) of the operating system of the smart device 40 itself, which has native functions of the smart device 40; providing a link to play a TV channel (TV), so that the smart device 40 broadcasts the content of the TV channel; providing a link to a first streaming audio and video program (such as Netflix) and a link to a second streaming audio and video program (Youtube), and the streaming audio and video program may be an application installed in the operating system of the smart device 40; providing a link to a first imaging function (such as AirPlay) and a link to a second imaging function (such as Miracast) of the external device 400, etc. The smart device proposed in this application is not limited to the embodiment shown in the embodiment diagram.

[0041] Then, if Figure 4B As shown, the user operates the external device 400 to connect to the smart device 40, and is ready to project the display screen of the external device 400 onto the display screen of the smart device 40 in an image manner. When the user operates the control device 403 to select one of the data sources from the pop-up menu interface 402, an option signal is generated, and the operating system in the smart device 40 receives an option information (step S307), that is, switches to the screen of the selected data source (step S309). Figure 4C As shown, the display screen of the smart device 40 displays the image data projected by the external device 400, and after the imaging process is completed, the pop-up menu interface 402 can be retracted and hidden at a certain position on the display screen.

[0042] At the same time, when the operating system of the smart device 40 receives the option signal for switching the data source, the data source may be the connected external device 400, that is, a signal channel is established with the data source. According to the embodiment, a signal channel is established between the operating system of the smart device 40 and the operating system of the external device (step S311), and the control instruction generated by the control device 403 can be transmitted to the external device 400 through the signal channel (step S313), so that the control device 403 that originally controls the smart device 40 can control the application on the external device 400 through the signal channel.

[0043] According to an embodiment, the data source may be an external device 400, and the smart device 40 is used to project the display screen imaged by the external device 400; on the other hand, the data source may be an application program executed in the operating system of the smart device 40. Taking the connection to a specific streaming audio and video source as an example, the streaming audio and video playback program is executed in the smart device 40, and the streaming audio and video source is connected.

[0044] It is worth mentioning that in the method of cross-platform switch command, one of the main methods is to establish a signal channel for transmitting control commands between two devices on different platforms, and use the user interface as a program running behind the control command generated by the control device. When the signal channel between the devices is established and opened, the command of the control device can be transmitted across the platform. Conversely, when the signal channel is closed, the command of the control device does not need to be transmitted across the platform. Figure 2 In the schematic diagram of the embodiment shown, the first device is a smart device, and the second device is an external device to project content to the first device, such as a desktop computer, a laptop computer, a mobile phone or a tablet computer, etc. When the second device projects content to the first device using a specific transmission technology (such as Miracast, Chromecast or Airplay, etc.), a signal channel can be established between the operating system in the first device and the operating system of the second device.

[0045] Furthermore, when the signal channel is turned on, the control device that originally controls the smart device can control the content of the data source of the projected content on the smart device through this signal channel; conversely, when the signal channel is turned off, the instructions of the control device are only for the control of the smart device. The connection between the first device and the second device, the connection method between the control device and the first device, and the transmission protocol between the devices are not limited to a specific method.

[0046] To summarize, according to the above-mentioned cross-platform switch instruction method and the embodiment of the smart device, the proposed smart device can be a smart screen that provides various data sources to project display images. For example, another external device can be provided to project the display image on this smart screen. At the same time, the operating system in the smart device runs the cross-platform switch instruction method. The smart device is connected to one or more control devices, which can be a remote control, a mouse or a keyboard. When controlling the smart device, you can choose to switch to other data sources through the user interface, and you can also control the external device through the cross-platform switch instruction method.

[0047] The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the scope of the claims of the present invention.

Explanation of symbols

Claims

1. A method for a cross-platform switch command, executed in a smart device, characterized in that: include: An operating system running on the smart device receives an instruction from a control device connected to the smart device to trigger a menu interface; Opening the menu interface, wherein the menu interface provides at least one data source option; According to the selected one of the data sources, displaying the content received from the data source on the smart device; Establishing a signal channel between the smart device and the data source; and The command generated by the control device is transferred to the data source through the signal channel.

2. The method for cross-platform switch instructions according to claim 1, characterized in that: A user interface program running the menu interface is used to provide options for the at least one data source and is also used to intercept instructions generated by the control device.

3. The method for cross-platform switch instructions as described in claim 1, characterized in that: The data source is an external device, and a display image of the external device is projected onto a display screen of the smart device through a communication interface of the smart device.

4. The method for cross-platform switch instructions according to claim 3, characterized in that: The signal channel is established between the operating system of the smart device and an operating system of the external device, and instructions or information are transmitted in the signal channel and interpreted by the two operating systems of the smart device and the external device.

5. The method for cross-platform switch instructions according to claim 1, characterized in that: A driver program of the control device is executed in the operating system of the smart device to drive the operation of the control device and interpret the instructions generated by the control device.

6. The method for cross-platform switch instructions according to claim 5, characterized in that: When the data source receives the instruction generated by the control device through the signal channel, the instruction is used to control the content projected by the data source on the smart device.

7. The method for cross-platform switch instructions according to any one of claims 1 to 6, characterized in that: When the signal channel is turned on, the control device that originally controls the smart device controls the content projected on the display screen by the data source through the signal channel; when the signal channel is closed, the instructions of the control device are only for the control of the smart device.

8. A smart device, characterized in that: include: One or more communication modules connected to a data source via a communication interface; a display screen for displaying display content received from the data source; an input module, providing an input interface, so that a control device is connected to the smart device through the input interface; The smart device runs an operating system, wherein the method for executing a cross-platform switch instruction includes: receiving an instruction generated by the control device to trigger a menu interface; Opening the menu interface, wherein the menu interface provides at least one data source option; According to the selected data source, the content received from the data source is displayed on the display screen; Establishing a signal channel between the smart device and the data source; and The command generated by the control device is transferred to the data source through the signal channel.

9. The smart device according to claim 8, characterized in that: The data source is an external device, which uses a wireless audio and video transmission technology through one of the communication interfaces, and establishes a signal channel between the operating system of the external device and the operating system of the smart device, projecting the display screen of the external device onto the display screen of the smart device.

10. The smart device according to claim 8 or 9, characterized in that: In the cross-platform switch instruction method, when the signal channel is turned on, the control device that originally controls the smart device controls the content projected on the display screen by the data source through the signal channel; when the signal channel is closed, the instructions of the control device are only for the control of the smart device.

Images