Set top box control method and device, equipment, storage medium and program product

Abstract

The invention provides a set top box control method, device and equipment, a storage medium and a program product, and relates to the technical field of smart home, and the method comprises the steps: transmitting a media data stream to a display device; receiving a control instruction sent by the display device; the control instruction is generated according to a control mode of the display device; and updating the media data stream according to the control instruction, and sending the updated media data stream to the display device. The set top box is wirelessly connected with the display device, and the set top box can be controlled based on the original control mode of the display device, so that the control complexity of the set top box is reduced, and the user experience and the control flexibility of the set top box are improved.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a control method, device, equipment, storage medium, and program product for a set-top box. Background Technology

[0002] Currently, set-top boxes can only display their images when connected to smart TVs or other displays via wired connections (such as HDMI (High-Definition Multimedia Interface) or VGA (Video Graphics Array) cables). Alternatively, the set-top box itself may have a display, but the screen is bulky and difficult to hold, and the existing operating methods of the display device are incompatible with the set-top box. Users need an additional remote control or device to operate the set-top box, increasing operational complexity. Therefore, when the set-top box and display device are connected wirelessly, the complexity of set-top box operation becomes a pressing issue that needs to be addressed. Summary of the Invention

[0003] This invention provides a control method, device, equipment, storage medium, and program product for a set-top box, which solves the problem of complex operation of the set-top box when the set-top box and the display device are connected wirelessly in the prior art. It enables control of the set-top box based on the original control method of the display device, thereby reducing the operation complexity of the set-top box, improving the user experience, and enhancing the control flexibility of the set-top box.

[0004] This invention provides a control method for a set-top box, applied to a set-top box, wherein the set-top box and a display device are wirelessly connected, comprising the following steps: Transmit media data streams to the display device; The system receives control commands sent by the display device; the control commands are generated based on the operation mode of the display device; the system updates the media data stream according to the control commands, and sends the updated media data stream to the display device.

[0005] According to a control method for a set-top box provided by the present invention, updating the media data stream according to the control command includes: When the control command is a button command, the media data stream is updated according to the button code and press information in the button command; When the control command is a touch command, the media data stream is updated according to the touch point coordinates in the touch command and the coordinate system of the display device; When the control command is a voice command, the media data stream is updated according to the voice stream in the voice command.

[0006] According to a control method for a set-top box provided by the present invention, the method further includes: When multiple display devices are connected, a master control display device for controlling the set-top box is determined from among the multiple display devices. Alternatively, the display devices can be grouped according to their operation type, and a master control display device for controlling the set-top box can be determined from each group of display devices.

[0007] According to a control method for a set-top box provided by the present invention, the method further includes: When multiple display devices simultaneously open the same-screen application, multiple copies of the media data stream are made; A copy of the media data stream is synchronized to each of the display devices so that multiple display devices can display the media data stream simultaneously.

[0008] According to a control method for a set-top box provided by the present invention, the step of transmitting media data streams to the display device includes: If the display device is successfully bound, send a data packet of the application to the display device. Receive interactive data sent by the display device based on the screen mirroring application; The interactive data is verified, and if the verification is successful, the media data stream is transmitted to the display device.

[0009] This invention provides a control method for a set-top box, applied to a display device, wherein the display device and the set-top box are wirelessly connected, comprising the following steps: Receive the media data stream transmitted by the set-top box; Based on the operation method of the display device, control commands for the set-top box are generated; The control command is sent to the set-top box so that the set-top box updates the media data stream based on the control command and sends the updated media data stream to the display device.

[0010] According to a control method for a set-top box provided by the present invention, the step of generating control commands for the set-top box based on the operation mode of the display device includes: In response to the detected control event of the display device, determine whether the currently running application on the same screen is in the foreground of the display device; If the screen-sharing application is in the foreground of the display device, then control commands for the set-top box are generated based on the detected control events. The control event is generated based on the control method; the screen-sharing application is in the foreground of the display device, and the page representing the current user operation is the set-top box page.

[0011] According to a control method for a set-top box provided by the present invention, the step of generating control commands for the set-top box based on the detected manipulation event includes: When the control event is a button event, the button event is intercepted and the key value code and press information of the button event are extracted. According to the set-top box communication protocol, the key value code and the press information are encapsulated to generate the button command of the set-top box. When the control event is a touch screen event, the coordinate information of the touch point is determined based on the user's touch operation information, and the touch command of the set-top box is generated based on the coordinate information of the touch point and the coordinate system of the display device. When the control event is a voice event, the user's voice stream is collected, the voice stream is encoded and compressed, and voice commands for the set-top box are generated.

[0012] The present invention also provides a control device for a set-top box, which is applied to a set-top box and includes the following modules: The transmission module is used to transmit media data streams to the display device; A control command receiving module is used to receive control commands sent by the display device; the control commands are generated according to the operation mode of the display device. An update module is used to update the media data stream according to the control command and send the updated media data stream to the display device.

[0013] The present invention also provides a control device for a set-top box, applied to a display device, comprising the following modules: The media data stream receiving module is used to receive the media data stream transmitted by the set-top box; A control command generation module is used to generate control commands for the set-top box based on the operation mode of the display device. A control command sending module is used to send the control command to the set-top box so that the set-top box updates the media data stream based on the control command and sends the updated media data stream to the display device.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the control method of any of the set-top boxes described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method of the set-top box as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the control method of any of the set-top boxes described above.

[0017] The present invention provides a control method, apparatus, device, storage medium, and program product for a set-top box. This involves transmitting a media data stream to a display device; receiving control commands sent by the display device; generating the control commands based on the operation mode of the display device; updating the media data stream according to the control commands; and sending the updated media data stream to the display device. The set-top box and display device in this invention are wirelessly connected, allowing control of the set-top box based on the display device's existing operation mode. This reduces the complexity of set-top box operation, improves user experience, and enhances the control flexibility of the set-top box. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the control method for a set-top box provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the set-top box and display device provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the operation method of the display device provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the display device binding process provided by the present invention.

[0023] Figure 5 This is the second flowchart illustrating the control method for the set-top box provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the button control process provided by the present invention.

[0025] Figure 7 This is a schematic diagram of the touch control process provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the voice control process provided by the present invention.

[0027] Figure 9 This is one of the structural schematic diagrams of the control device for the set-top box provided by the present invention.

[0028] Figure 10 This is the second schematic diagram of the control device for the set-top box provided by the present invention.

[0029] Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] In related technologies, set-top boxes have the following disadvantages: 1. The set-top box and the display device need to be connected via a wired connection, such as an HDMI cable or a VGA cable; 2. After the set-top box is connected to the display device, the original control methods of the set-top box and the display device are not compatible. For example, if a smart TV is used as the display device, the smart TV remote control can only control the smart TV and cannot control the set-top box, nor can it be controlled by voice. If the connected display device has a touch screen function, the display device can only be controlled by touch screen and cannot control the set-top box. 3. Connecting the set-top box to the display device using screen mirroring also fails to solve the problem in point 2; 4. The screen mirroring method can only mirror to one device, which cannot meet the need for simultaneous viewing on multiple display devices.

[0032] Based on the above problems, this invention proposes a control method for a set-top box, mainly used to solve the following problems: 1. The set-top box connects wirelessly to display devices (such as smart TVs, smartphones, computers, etc.) and displays media data streams; 2. Existing control methods on both the set-top box and display device are universally applicable. For example, a smart TV remote can control the set-top box (e.g., buttons, voice); a smartphone can control the set-top box via touch or its own voice assistant; a computer display can control the set-top box via mouse, greatly enhancing the user experience; the set-top box remote can also control the display device, etc. 3. Multiple display devices can simultaneously connect to the set-top box and watch the set-top box content in sync, thus expanding the usage scenarios of the set-top box.

[0033] The following is combined Figures 1-11 The present invention describes a control method, apparatus, device, storage medium, and program product for a set-top box.

[0034] Figure 1 This is one of the flowcharts illustrating the control method for a set-top box provided by the present invention, such as... Figure 1 As shown, this method is applied to a set-top box, which is wirelessly connected to a display device, and includes the following steps: Step 101: Transmit a media data stream to the display device.

[0035] The set-top box transmits media data streams to the display device wirelessly. These media data streams are data streams related to audio and video content, mainly including audio streams and video streams.

[0036] It should be understood that the set-top box can also wirelessly transmit firmware data, screen mirroring APP data, status information, log information, and other data to the display device. Among them, firmware data is the firmware file used to update the device (such as the display device, home appliances, etc.); screen mirroring APP data is the relevant data used to install and run screen mirroring applications on the display device, including the APP installation package, configuration file, etc.

[0037] It should be understood that the set-top box and the display device can communicate wirelessly via the following methods: 1. Wi-Fi Connection: The set-top box has a built-in Wi-Fi module, which can be directly connected to a home wireless router to communicate with display devices (such as smart TVs, tablets, and mobile phones). Alternatively, it can be bridged with a wireless router to connect the optical modem and the set-top box to the same wireless network for wireless communication.

[0038] 2. NearLink technology: Supports high-bandwidth, low-latency inter-device communication, suitable for audio and video streaming between set-top boxes and display devices.

[0039] refer to Figure 2The main modules in the set-top box include: Wireless module: Employing Wi-Fi, StarFlash, and other wireless protocols; responsible for data communication with the display device, including transmitting audio streams, video streams, display device firmware, and home device firmware; responsible for communication with the display device and distributing screen-sharing apps. Screen-sharing module: Generates the set-top box's page audio and video streams and transmits them to the display device via the wireless module. It also receives user operation information (such as press, release, short press, long press, drag, and swipe actions) and voice streams, and performs information parsing, speech recognition, and semantic understanding. Display module: Responsible for multi-device display on the set-top box. Communication module: Communicates with cloud services. Storage module: Stores screen-sharing app information, main control display device information, and display device information.

[0040] Display devices can include smart TVs, smartphones, computers, tablets, and other devices with display functions, capable of receiving and displaying images, videos, or other visual content. The main modules within the display device include: a wireless module: communicating with the set-top box using wireless protocols such as Wi-Fi or StarFlash; responsible for receiving audio streams, video streams, display device firmware, and home device firmware transmitted from the set-top box; and a screen mirroring module: collecting operation information from the display device and the user (such as press, release, short press, long press, drag, and swipe operations), requesting and receiving voice stream capabilities, and transmitting this information to the set-top box via the wireless module; and decoding and displaying the audio and video streams transmitted from the set-top box via the wireless module.

[0041] Once a connection or network has been established and authentication has been completed, the set-top box can send data streams to the display device, receive synchronization instructions from the display device, and perform corresponding operations.

[0042] Step 102: Receive control commands sent by the display device.

[0043] Specifically, the set-top box receives control commands sent by the display device, which are generated based on the operation method of the display device.

[0044] It should be understood that the operation method of a display device refers to the method by which a user operates the display device through its interface or hardware. (Refer to...) Figure 3 This control method may include: Button operation: Operation is performed using physical buttons or virtual buttons on the touchpad. For example, remote control buttons: using the buttons on the TV remote to control the set-top box, such as turning it on / off, adjusting the volume, and changing channels. Mobile phone buttons: using the physical or virtual buttons on the mobile phone to perform operations, such as adjusting the volume and changing channels.

[0045] Touchscreen operation: Directly operating on the device's touchscreen, such as swiping, clicking, and long-pressing. For example, touchscreen device operation: controlling a set-top box by touching the screen on a smartphone or tablet. Mouse operation: Using a mouse on a computer to click, drag, etc. Although not directly touching the screen, it is still indirect touchscreen operation.

[0046] Voice operation: Control the set-top box via voice commands. For example, remote control voice button: Use the voice button on the remote control to issue commands such as play, pause, fast forward, rewind, change channels, adjust volume, etc. Smart device built-in voice assistant: Use the voice assistant on your mobile phone to control the set-top box.

[0047] Optionally, the control method may also include gesture operation, whereby users can interact with the device through specific gestures, such as using a camera to capture user movements to complete the operation, thereby controlling the set-top box.

[0048] When a user controls the set-top box via the display device, the display device generates corresponding control commands based on the user's actions. Simultaneously, these generated control commands are transmitted to the set-top box via wireless communication methods (such as Wi-Fi, Bluetooth, or Wi-Fi).

[0049] Step 103: Update the media data stream according to the control command, and send the updated media data stream to the display device.

[0050] The set-top box parses the received control commands and then performs corresponding operations based on the command content. Specifically, it updates the media data stream according to the command content and sends the updated media data stream to the display device.

[0051] It should be understood that updating the media data stream refers to adjusting or replacing the audio and video content, which may include: 1) Switching audio and video sources: The set-top box can switch to different audio and video sources according to the control instructions of the display device, such as switching from live TV to on-demand content or from one media application to another. 2) Adjusting playback parameters: The set-top box can adjust playback parameters according to control instructions, such as playback speed (fast forward, rewind), volume, or picture quality. 3) Dynamic content adjustment: The set-top box can dynamically adjust the content according to control instructions, such as adjusting the resolution or frame rate during a video call to adapt to changes in network conditions. 4) Data stream reassembly: After receiving control instructions, the set-top box may need to reassemble the data stream to ensure the continuity and synchronization of the audio and video content. This involves reordering or repackaging data packets to adapt to new playback parameters or content sources. 5) Error recovery: If errors or data loss occur during transmission, the set-top box needs to update the media data stream to resume playback. This may include requesting the retransmission of lost data packets or recovering from an error recovery mechanism.

[0052] In one embodiment, it is assumed that the user is using a smartphone to control the set-top box: the user clicks the "Switch audio and video source" button on the phone screen, the phone generates a "Switch audio and video source" command based on the user's click operation, the phone sends this "Switch audio and video source" command to the set-top box via Wi-Fi, the set-top box receives the command and switches the audio and video source, and sends the media data stream after switching the audio and video source to the display device for display.

[0053] In one embodiment, it is assumed that the user controls the set-top box using the remote control of the smart TV: the user presses the "volume +" button on the remote control, the smart TV generates a "volume increase" command based on the button operation of the remote control, the smart TV sends this command to the set-top box via Bluetooth, the set-top box receives the command, increases the volume, and sends the increased media data stream to the display device for display.

[0054] The set-top box control method provided in this invention involves transmitting a media data stream to a display device; receiving control commands sent by the display device; generating the control commands based on the operation mode of the display device; updating the media data stream according to the control commands; and sending the updated media data stream to the display device. In this invention, the set-top box and display device are wirelessly connected, allowing control of the set-top box based on the display device's existing operation mode. This reduces the complexity of set-top box operation, improves user experience, and enhances the control flexibility of the set-top box.

[0055] In one embodiment, updating the media data stream according to the control command includes: Step 210: If the control command is a button command, update the media data stream according to the button code and press information in the button command; Step 220: If the control command is a touch command, update the media data stream according to the touch point coordinates in the touch command and the coordinate system of the display device; Step 230: If the control command is a voice command, update the media data stream according to the voice stream in the voice command.

[0056] Specifically, after receiving a button command, the set-top box parses the button code and press information (such as whether it is a long press, the duration of the long press, etc.), and then simulates different system commands based on different codes. For example, the code can correspond to operations such as play, pause, and volume adjustment. The set-top box also determines whether the button is a long press and the duration of the long press to make corresponding responses, and updates the audio and video streams in real time, transmitting them to the screen-sharing app via the wireless module.

[0057] It should be understood that during the key code parsing process, the set-top box can use standard or predefined encoding tables to identify specific key commands. For example, in the Android system, KEYCODE_BACK corresponds to the back button, KEYCODE_MENU corresponds to the menu button, and so on. The set-top box uses these codes to identify the user's intent and execute the corresponding operation.

[0058] After receiving a touch command, the set-top box parses the touch point coordinates and the coordinate system of the display device, mapping them to its own coordinate system. It then parses the control information (such as images, buttons, text, lists, etc.) within the coordinate system and interprets the actions. The system's underlying control then performs the corresponding actions and updates the audio and video streams, which are then transmitted to the display device. For example, the set-top box can process touch commands through the following steps: (1) Parsing touch point coordinates: After receiving a touch command, the set-top box first parses the touch point coordinates in the touch command. These coordinates are relative to the display device screen.

[0059] (2) Coordinate system mapping: Since the coordinate system of the display device is different from that of the set-top box, the set-top box needs to map these coordinates into its own coordinate system to ensure that touch actions can be correctly recognized and processed.

[0060] (3) Control information parsing: Within the coordinate system, the set-top box identifies the control information (such as images, buttons, text, lists, etc.) corresponding to the touch point. These controls are part of the user interface and can respond to touch actions.

[0061] (4) Action parsing and execution: The set-top box parses the user's actions (such as clicking, swiping, long pressing, etc.) based on the coordinates of the touch point and the control information, and triggers the corresponding system-level control to make the corresponding control execute the preset action.

[0062] (5) Update audio and video streams: The set-top box updates the media data stream based on the actions of the controls. For example, if the user touches a play button, the set-top box will start playing the video; if the user touches a fast forward button, the set-top box will fast forward the video.

[0063] After receiving a voice command, the set-top box performs speech recognition and semantic understanding on the voice stream. Based on the instructions returned from the cloud, it responds accordingly, updates the audio and video streams, and transmits them wirelessly to a display device for display in a screen-sharing app. Specifically, upon receiving the voice stream, the set-top box first performs speech recognition, converting the voice signal into text. Then, it performs semantic understanding on the recognized text to determine the user's intent and the required action. This process involves cloud services, utilizing powerful natural language processing technology to parse the user's voice commands. Once the set-top box understands the user's instructions, it responds accordingly based on the instructions returned from the cloud, which may include changing channels, adjusting volume, searching for content, or performing other media playback-related operations. Finally, the set-top box updates the audio and video streams to reflect the results of these operations and transmits the updated audio and video streams wirelessly to a display device, such as a smart TV or tablet, for display in a screen-sharing app.

[0064] The embodiments of the present invention control the set-top box through the operation of the display device, which improves the convenience and flexibility of operation, while increasing the frequency of use and interaction of the set-top box in the home setting, and reducing the user's cost.

[0065] In one embodiment, the method further includes: Step 310: In the case of connecting multiple display devices, determine one of the multiple display devices as the main control display device for controlling the set-top box; Step 320: Group the display devices according to their operation type, and determine a master control display device from each group of display devices for controlling the set-top box.

[0066] When a set-top box is connected to multiple display devices simultaneously, it can select one as the master display device. This master display device has the authority to control the set-top box, such as switching channels, adjusting volume, and playing or pausing content. The set-top box also stores information about the master display device, such as device type, connection status, and supported control methods (touch, buttons, voice, etc.).

[0067] It should be understood that the screen mirroring apps on other display devices will not intercept button presses, touch events, or voice capability requests. In other words, other display devices can receive user input normally, but will not directly affect the operation of the set-top box.

[0068] Set-top boxes can group display devices into different groups based on their control method (e.g., button operation, touchscreen operation, voice operation). Within each control type group, a master control display device can be designated; that is, each operation type has a corresponding master control display device. For example, a smart TV can be selected as the master control display device for button operation, while a mobile phone can be selected as the master control display device for touchscreen and voice operation. Each operation type can independently select its master control display device, ensuring that devices with different operation types do not interfere with each other, while also providing more flexible control options.

[0069] It should be understood that users can dynamically adjust / switch the main control display device based on the current usage scenario and personal preferences. For example, if a user is conducting a voice search, they can choose their mobile phone as the main control display device; if a user prefers to use physical buttons, they can choose a smart TV remote control.

[0070] This invention allows users to flexibly select and switch between multiple display devices as the main control display device, while ensuring that other display devices do not interfere with the control operation of the main control display device. Furthermore, by grouping by operation type, control permissions for different devices can be managed and allocated more effectively, thereby improving user experience and system flexibility.

[0071] In one embodiment, the method further includes: Step 410: When multiple display devices simultaneously open the same-screen application, copy multiple copies of the media data stream; Step 420: Synchronize one copy of the media data stream to each display device so that multiple display devices can simultaneously display the media data stream.

[0072] In scenarios where multiple display devices play content simultaneously, the set-top box's media stream distribution mechanism involves stream copying and synchronous transmission. Specifically, when multiple display devices (such as smart TVs, mobile phones, and tablets) simultaneously open a shared-screen application, the set-top box needs to generate an independent media data stream for each display device. This means the set-top box copies the original audio and video streams and creates a copy for each connected display device. The set-top box then synchronizes the copied media data stream to each display device to ensure that each device receives complete audio and video information and can start playing simultaneously, thus achieving the effect of multi-screen simultaneous display.

[0073] It should be understood that in order to ensure the synchronization of content on multiple display devices, the set-top box needs to implement a synchronization mechanism to ensure that all display devices play the same content at the same time.

[0074] For example, during family gatherings or get-togethers with friends, multiple display devices can simultaneously play the same movie or TV program, increasing viewing convenience. In classrooms or conference rooms, teachers or speakers can use multi-screen display technology to ensure that all participants can clearly see the presentation content.

[0075] This invention provides an efficient, flexible, and user-friendly content sharing method by synchronizing media data streams on multiple display devices, which can improve user experience and meet the needs of different users.

[0076] In one embodiment, transmitting the media data stream to the display device includes: Step 510: If the display device is successfully bound, send a data packet of the screen mirroring application to the display device; Step 520: Receive interactive data sent by the display device based on the screen mirroring application; Step 530: Verify the interactive data. If the verification is successful, transmit the media data stream to the display device.

[0077] When a display device and a set-top box are bound together, a screen mirroring module (such as a screen mirroring app or service) will be downloaded and installed. The following explanation will use a screen mirroring app as an example.

[0078] After the set-top box and the display device are successfully bound together, the set-top box will send the data package required for the screen mirroring application to the display device. This data package may include the installation file of the screen mirroring application, such as an APK (Android application package) file or an EXE executable file, as well as the required configuration files or resource files.

[0079] After the display device installs and runs the screen mirroring application, it interacts with the set-top box based on the screen mirroring application. For example, it transmits interactive data such as display device information and screen mirroring APP information to the set-top box based on the screen mirroring application.

[0080] After receiving interactive data from the display device, the set-top box verifies it. This verification process includes checking data integrity, verifying the data's source and format, and ensuring data security. Once the interactive data passes verification, the set-top box generates or selects the appropriate media data stream (such as audio and video content) and transmits it to the display device via a wireless module (such as Wi-Fi, Bluetooth, or Wi-Fi). The screen mirroring application on the display device receives the media data stream, decodes it, and displays it.

[0081] like Figure 4 As shown, the process of binding a display device to a set-top box may include: In step S1, the display device enters a pairing mode, which allows the set-top box to recognize the device and establish a connection. The user selects the display device to be paired (such as a smart TV, mobile phone, computer, etc.) on the set-top box's user interface; for example, by browsing the list of available devices on the set-top box and selecting a specific device. After selecting a device, the pairing process begins, including exchanging pairing information and verifying the device's identity. Once paired successfully, a stable connection is established between the set-top box and the display device, which can be achieved through Wi-Fi networking or wireless technologies such as StarFlash.

[0082] In step S2, after the set-top box detects successful binding, it transmits the screen-sharing app to the display device. The app's format depends on the display device's operating system; for example, Android devices receive APK files, and Windows devices receive EXE files. When the display device receives the screen-sharing app, it simultaneously verifies its legitimacy, such as checking the app's signature and version information, to ensure the app is secure and reliable. After successful verification, the display device installs the screen-sharing app, which will be used to interact with the set-top box and receive and display audio and video content. After installation, the screen-sharing app can request the display device's voice capabilities to listen to, capture, and interpret user actions.

[0083] In step S3, when the set-top box receives the data transmitted by the screen mirroring APP of the display device, the set-top box will verify the transmitted information (such as display device information, screen mirroring APP information, etc.). If the verification is successful, an audio and video stream will be generated and transmitted to the screen mirroring APP through the wireless module. The screen mirroring APP will then decode and display the data.

[0084] This invention supports display devices with different operating systems, such as Android and Windows. By sending application data packages (APK or EXE) in the appropriate formats, the versatility and compatibility of the solution are improved. Furthermore, content synchronization and interactive control between the set-top box and multiple display devices are achieved wirelessly, enhancing the user experience, increasing device flexibility and compatibility, while also providing good security and cost-effectiveness.

[0085] Figure 5 This is a second flowchart illustrating the control method for a set-top box provided by the present invention, as shown below. Figure 5 As shown, this method is applied to a display device, which is wirelessly connected to a set-top box, and includes the following steps: Step 501: Receive the media data stream transmitted by the set-top box.

[0086] The display device receives media data streams transmitted from the set-top box wirelessly.

[0087] Step 502: Generate control commands for the set-top box according to the operation mode of the display device.

[0088] Users can control the set-top box through the control methods provided by the display device itself. Specifically, control commands for the set-top box are generated based on the control methods of the display device.

[0089] In one embodiment, in response to a detected control event on the display device, it is determined whether the currently running screen-sharing application is in the foreground of the display device; if the screen-sharing application is in the foreground of the display device, control instructions for the set-top box are generated based on the detected control event.

[0090] Control events are generated based on the control method; these include button events, touch events, and voice events. When the mirrored application is in the foreground of the display device, the page currently being operated by the user is the set-top box page. This set-top box page can be understood as a graphical user interface rendered by the mirrored application, with content sourced from the set-top box. It should be understood that if the mirrored app is in the foreground, the displayed content comes from the set-top box, such as an OTT interface or a set-top box live stream, and the user's intention is clearly to control the set-top box. For example, when the TV displays the Tencent Video app from the set-top box in full screen, the remote control buttons should control video playback (e.g., pause / fast forward). If the mirrored app is in the background, the displayed content is the TV's native system interface, such as the HDMI input source or TV settings menu, and the user's intention is to control the TV itself. For example, when returning to the TV's home screen, the same remote control should control TV app switching, not the set-top box. Specifically, when the application being mirrored is in the foreground of the display device, the current displayed content is determined to be the set-top box output page, and the input command is routed to the set-top box; when the application being mirrored is not in the foreground, the current displayed content is determined to be the display device's native page, and the display device processes the input command.

[0091] Specifically, after detecting user interaction events on the display device, it determines whether the currently running application is in the foreground. It should be understood that in a multitasking operating system, the foreground application refers to the application the user is currently directly interacting with. If the application is in the foreground, it means the user's current intention is related to the set-top box's content or functions. Then, based on the detected interaction events, corresponding set-top box control commands are generated. These commands are generated according to the user's interaction method (such as buttons, touch, voice, etc.) and aim to instruct the set-top box to perform specific operations, such as play, pause, fast forward, etc.

[0092] In one embodiment, when the control event is a button event, the button event is intercepted and the key value code and press information of the button event are extracted. According to the set-top box communication protocol, the key value code and press information are encapsulated to generate button commands for the set-top box. For example, the button event is intercepted and the key value code and press duration are extracted. Then, according to the set-top box communication protocol, the key value code and press duration are encapsulated into a structured control command. Finally, the structured control command (i.e., the button command) is sent to the set-top box via a wireless transmission module. The structured control command may include: an operation type field to distinguish between single trigger, long press, and combination key; a timestamp field to mark the absolute occurrence time of the button event; and a device identifier field to identify the display device ID from which the command originates. The calculation of the press duration includes starting the timer from when the button is pressed and ending when the button is released or a timeout threshold is reached.

[0093] When the control event is a touch event, the coordinates of the touch point are determined based on the user's touch operation information. Based on the touch point coordinates and the coordinate system of the display device, touch commands for the set-top box are generated. For example, the physical coordinates of the touch event are converted to standard coordinates in the set-top box's UI logical coordinate system. Then, a predefined gesture type is identified based on the temporal-spatial characteristics of the touch trajectory. The above information is encapsulated, including a control command packet containing the following fields: gesture type identifier, start / end coordinates, touch duration, and trajectory key point sequence. Finally, the command packet (containing the touch command) is transmitted to the set-top box via a wireless communication module.

[0094] It should be understood that coordinate transformation includes: obtaining the element layout information of the current interface of the set-top box, and then establishing a mapping matrix between screen pixel coordinates and set-top box UI controls. Gesture type recognition includes: calculating the Fourier descriptor of the touch trajectory, and then matching it with a pre-stored gesture template through a convolutional neural network.

[0095] When the control event is a voice event, the user's voice stream is acquired, encoded, and compressed to generate voice commands for the set-top box. For example, the system receives raw voice stream transmitted from a display device, which is raw audio data in PCM format. The raw voice stream undergoes the following processing: noise reduction and endpoint detection; compression of the PCM format into a low-bit-rate encoding format; and encapsulation into a voice command packet containing a timestamp and device identifier. Finally, the voice command packet is transmitted to the set-top box via a wireless communication module, where it decodes the packet and performs semantic recognition and control operations. Optionally, the parameters of the raw voice stream include: a sampling rate of 16kHz, a sampling depth of 16bit, mono, and metadata including the voice remote control device ID.

[0096] This invention allows users to control the set-top box through the display device's own control method, enabling them to operate the set-top box conveniently without direct interaction with it, thus improving the user experience and making set-top box control more intuitive and convenient.

[0097] Step 503: Send the control command to the set-top box so that the set-top box updates the media data stream based on the control command and sends the updated media data stream to the display device.

[0098] The display device sends control commands to the set-top box, causing the set-top box to update the media data stream based on the control commands, and then sends the updated media data stream back to the display device.

[0099] For example, if a user finds the volume too loud while watching live TV and wants to turn it down, they can do so by following these steps: User operation: The user adjusts the volume by scrolling down the mouse wheel in the TV application interface on the computer screen; Sending control commands: The computer converts the request to decrease the volume into a control command and sends it to the set-top box; Set-top box processing: After receiving a volume adjustment command, the set-top box adjusts the output volume; Media stream update: The set-top box continues to transmit audio data streams, but the volume has been adjusted; Sending to display device: The set-top box sends the adjusted audio data stream to the computer; Display device demonstration: After the computer receives the adjusted audio data stream, it plays it through the speakers, and the volume has been reduced.

[0100] The set-top box control method provided in this invention involves receiving media data streams transmitted from the set-top box; generating control commands for the set-top box based on the operation mode of the display device; sending the control commands to the set-top box to update the media data stream based on the control commands; and then sending the updated media data stream to the display device. In this invention, the set-top box and display device are wirelessly connected, allowing control of the set-top box based on the display device's existing operation mode. This reduces the complexity of set-top box operation, improves user experience, and enhances the control flexibility of the set-top box.

[0101] like Figure 6 As shown, Figure 6 This is a schematic diagram of the button control process provided by the present invention. Button operations include remote control button operations (such as number keys, channel keys, settings keys, menu keys, etc.) and mobile phone built-in buttons (such as volume keys, virtual keys for back, home settings keys, etc.). The button control process may include: In step S1, when the user controls the device using buttons, it is first determined whether the screen-sharing app is in the foreground of the display device (the rule is: if the user sees the set-top box page, the operation controls the set-top box; if it is the display device's own page, the operation controls the display device itself). If it is in the foreground, the screen-sharing app intercepts the button events and assembles data according to the protocol (button code, whether it is a long press, long press duration, etc.), and transmits it to the set-top box via the wireless module. If it is in the background, it is considered to be controlling the display device itself, and the display device processes its own logic.

[0102] In step S2, after receiving the button command, the set-top box parses the button code, simulates different system commands based on different codes, determines whether to press and hold and the duration of the press, and responds accordingly. It also updates the audio and video streams in real time and transmits them to the screen mirroring APP via the wireless module.

[0103] like Figure 7 As shown, Figure 7 This is a schematic diagram of the touch control process provided by the present invention. Similarly, for touch screen operation, it will first determine whether the APP on the same screen is in the foreground of the display device. Only when it is in the foreground will the operation be intercepted. The touch control process may include: In step S1, when the user performs touch screen control, the screen senses and collects the user's actions (such as: pressing, releasing, short press, long press, dragging, sliding, sliding selection, etc.), determines the coordinates of the user's touch point and its position in its own coordinate system, and transmits it to the set-top box through a wireless protocol.

[0104] In step S2, when the set-top box receives data from the display device, if it is a touch action, it parses the touch point coordinates and coordinate system and maps them to its own coordinate system. It then parses the control information (such as images, buttons, text, lists, etc.) within the coordinate system and the action. The system's underlying layer controls the corresponding controls to perform the appropriate action and updates the audio and video streams, which are then transmitted to the display screen. If the received instruction is a simple button operation, the system's underlying layer directly executes the system instruction and updates the audio and video streams, transmitting them to the display device. When the display device receives the audio and video streams, it updates the display.

[0105] like Figure 8 As shown, Figure 8 This is a schematic diagram of the voice control process provided by the present invention. The set-top box has permission to broadcast live TV, and the user can use voice control to open the live TV broadcast on the set-top box. After the display device detects the live TV broadcast, it can open the screen-sharing app. The voice control process may include: In step S1, when the screen mirroring APP is in the foreground, it will request voice stream capability from the display device. When the display device system receives the request, it will transmit the raw voice stream (PCM format 16k, 16bit, mono) of the voice input (such as voice remote control, mobile phone microphone) to the screen mirroring APP. The screen mirroring APP encodes and compresses the audio and transmits it to the set-top box through the wireless module.

[0106] In step S2, when the set-top box receives the voice stream, it performs processes such as voice recognition and semantic understanding, responds accordingly based on the instructions returned from the cloud, updates the audio and video streams, and transmits them to the display device via the wireless module for display on the screen-sharing APP.

[0107] The control device for the set-top box provided by the present invention is described below. The control device for the set-top box described below can be referred to in correspondence with the control method for the set-top box described above.

[0108] refer to Figure 9 , Figure 9 This is one of the structural schematic diagrams of the control device for a set-top box provided by the present invention. The control device for a set-top box provided by the present invention includes a transmission module 901, a control command receiving module 902, and an update module 903.

[0109] Transmission module 901 is used to transmit media data streams to the display device; The control command receiving module 902 is used to receive control commands sent by the display device; the control commands are generated according to the operation mode of the display device. The update module 903 is used to update the media data stream according to the control command and send the updated media data stream to the display device.

[0110] The control device for a set-top box provided in this invention transmits a media data stream to a display device; receives control commands sent by the display device; the control commands are generated based on the operation mode of the display device; updates the media data stream according to the control commands; and sends the updated media data stream to the display device. The set-top box and display device in this invention are wirelessly connected, allowing control of the set-top box based on the original operation mode of the display device, thereby reducing the complexity of set-top box operation and improving user experience and control flexibility.

[0111] In one embodiment, the update module 903 is specifically configured to: update the media data stream according to the key code and press information in the key instruction when the control instruction is a button instruction; update the media data stream according to the touch point coordinates in the touch instruction and the coordinate system of the display device when the control instruction is a touch instruction; and update the media data stream according to the voice stream in the voice instruction when the control instruction is a voice instruction.

[0112] In one embodiment, the control device of the set-top box further includes a master control display device determination module, configured to: determine a master control display device for controlling the set-top box from among the multiple display devices when multiple display devices are connected; or, group the display devices according to their control types and determine a master control display device for controlling the set-top box from each group of display devices.

[0113] In one embodiment, the transmission module 901 is further configured to: copy multiple copies of the media data stream when multiple display devices simultaneously open a screen-sharing application; and synchronize one copy of the media data stream to each display device so that multiple display devices simultaneously display the media data stream.

[0114] In one embodiment, the transmission module 901 is specifically configured to: send a data packet of the screen-sharing application to the display device when the display device is successfully bound; receive interactive data sent by the display device based on the screen-sharing application; verify the interactive data; and transmit a media data stream to the display device after successful verification.

[0115] refer to Figure 10 , Figure 10 This is a second schematic diagram of the control device for the set-top box provided by the present invention. The control device for the set-top box provided by the present invention includes a media data stream receiving module 1001, a control command generation module 1002, and a control command sending module 1003.

[0116] The media data stream receiving module 1001 is used to receive the media data stream transmitted by the set-top box; The control command generation module 1002 is used to generate control commands for the set-top box according to the operation mode of the display device; the control command sending module 1003 is used to send the control commands to the set-top box so that the set-top box updates the media data stream based on the control commands and sends the updated media data stream to the display device.

[0117] The set-top box control method provided in this invention involves receiving media data streams transmitted from the set-top box; generating control commands for the set-top box based on the operation mode of the display device; sending the control commands to the set-top box to update the media data stream based on the control commands; and then sending the updated media data stream to the display device. In this invention, the set-top box and display device are wirelessly connected, allowing control of the set-top box based on the display device's existing operation mode. This reduces the complexity of set-top box operation, improves user experience, and enhances the control flexibility of the set-top box.

[0118] In one embodiment, the control instruction generation module 1002 is specifically configured to: respond to a detected control event of the display device, determine whether the currently running screen-sharing application is in the foreground of the display device; if the screen-sharing application is in the foreground of the display device, generate control instructions for the set-top box based on the detected control event; wherein the control event is generated based on the control method; the screen-sharing application being in the foreground of the display device indicates that the page currently operated by the user is the set-top box page.

[0119] In one embodiment, the control command generation module 1002 is specifically configured to: intercept the key event and extract the key value code and press information of the key event when the control event is a button event; encapsulate the key value code and press information according to the set-top box communication protocol to generate the button command of the set-top box; determine the coordinate information of the touch point based on the user's touch operation information when the control event is a touch screen event; generate the touch command of the set-top box based on the coordinate information of the touch point and the coordinate system of the display device; and collect the user's voice stream and encode and compress the voice stream to generate the voice command of the set-top box when the control event is a voice event.

[0120] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute a control method for the set-top box. This method includes: transmitting a media data stream to the display device; receiving control instructions sent by the display device; the control instructions being generated according to the operation mode of the display device; updating the media data stream according to the control instructions; and sending the updated media data stream to the display device.

[0121] The processor 1110 can call logic instructions in the memory 1130 to execute a control method for the set-top box. The method includes: receiving a media data stream transmitted by the set-top box; generating control instructions for the set-top box according to the operation mode of the display device; sending the control instructions to the set-top box so that the set-top box updates the media data stream based on the control instructions, and sending the updated media data stream to the display device.

[0122] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the set-top box control method provided by the above methods. The method includes: transmitting a media data stream to the display device; receiving a control command sent by the display device; the control command being generated according to the operation mode of the display device; updating the media data stream according to the control command; and sending the updated media data stream to the display device.

[0124] When the computer program is executed by the processor, the computer can execute the set-top box control method provided by the above methods. The method includes: receiving a media data stream transmitted by the set-top box; generating a control command for the set-top box according to the operation mode of the display device; sending the control command to the set-top box so that the set-top box updates the media data stream based on the control command, and sending the updated media data stream to the display device.

[0125] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a control method for a set-top box provided by the methods described above. The method includes: transmitting a media data stream to the display device; receiving a control command sent by the display device; the control command being generated according to the operation mode of the display device; updating the media data stream according to the control command; and sending the updated media data stream to the display device.

[0126] When executed by a processor, the computer program implements a control method for a set-top box provided by the methods described above. The method includes: receiving a media data stream transmitted by the set-top box; generating a control command for the set-top box according to the operation mode of the display device; sending the control command to the set-top box to cause the set-top box to update the media data stream based on the control command; and sending the updated media data stream to the display device.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for a set-top box, characterized in that, Applied to a set-top box, wherein the set-top box and a display device are wirelessly connected, the method includes: Transmit media data streams to the display device; The system receives control commands sent by the display device; the control commands are generated based on the operation mode of the display device; the system updates the media data stream according to the control commands, and sends the updated media data stream to the display device.

2. The control method for a set-top box according to claim 1, characterized in that, Updating the media data stream according to the control command includes: When the control command is a button command, the media data stream is updated according to the button code and press information in the button command; When the control command is a touch command, the media data stream is updated according to the touch point coordinates in the touch command and the coordinate system of the display device; When the control command is a voice command, the media data stream is updated according to the voice stream in the voice command.

3. The control method for a set-top box according to claim 1, characterized in that, The method further includes: When multiple display devices are connected, a master control display device for controlling the set-top box is determined from among the multiple display devices. Alternatively, the display devices can be grouped according to their operation type, and a master control display device for controlling the set-top box can be determined from each group of display devices.

4. The control method for a set-top box according to claim 1, characterized in that, The method further includes: When multiple display devices simultaneously open the same-screen application, multiple copies of the media data stream are made; A copy of the media data stream is synchronized to each of the display devices so that multiple display devices can display the media data stream simultaneously.

5. The control method for a set-top box according to claim 1, characterized in that, The transmission of media data stream to the display device includes: If the display device is successfully bound, send a data packet of the application to the display device. Receive interactive data sent by the display device based on the screen mirroring application; The interactive data is verified, and if the verification is successful, the media data stream is transmitted to the display device.

6. A control method for a set-top box, characterized in that, Applied to a display device, wherein the display device and a set-top box are wirelessly connected, the method includes: Receive the media data stream transmitted by the set-top box; Based on the operation method of the display device, control commands for the set-top box are generated; The control command is sent to the set-top box so that the set-top box updates the media data stream based on the control command and sends the updated media data stream to the display device.

7. The control method for a set-top box according to claim 1, characterized in that, The step of generating control commands for the set-top box based on the operation mode of the display device includes: In response to the detected control event of the display device, determine whether the currently running application on the same screen is in the foreground of the display device; If the screen-sharing application is in the foreground of the display device, then control commands for the set-top box are generated based on the detected control events. The control event is generated based on the control method; the screen-sharing application is in the foreground of the display device, and the page representing the current user operation is the set-top box page.

8. The control method for a set-top box according to claim 7, characterized in that, The generation of control commands for the set-top box based on the detected manipulation events includes: When the control event is a button event, the button event is intercepted and the key value code and press information of the button event are extracted. According to the set-top box communication protocol, the key value code and the press information are encapsulated to generate the button command of the set-top box. When the control event is a touch screen event, the coordinate information of the touch point is determined based on the user's touch operation information, and the touch command of the set-top box is generated based on the coordinate information of the touch point and the coordinate system of the display device. When the control event is a voice event, the user's voice stream is collected, the voice stream is encoded and compressed, and voice commands for the set-top box are generated.

9. A control device for a set-top box, characterized in that, Applied to set-top boxes, including: The transmission module is used to transmit media data streams to the display device; A control command receiving module is used to receive control commands sent by the display device; the control commands are generated according to the operation mode of the display device. An update module is used to update the media data stream according to the control command and send the updated media data stream to the display device.

10. A control device for a set-top box, characterized in that, Applied to display devices, including: The media data stream receiving module is used to receive the media data stream transmitted by the set-top box; The control command generation module is used to generate control commands for the set-top box according to the operation mode of the display device; the control command sending module is used to send the control commands to the set-top box so that the set-top box updates the media data stream based on the control commands and sends the updated media data stream to the display device.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the control method for the set-top box as described in any one of claims 1 to 8.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the control method of the set-top box as described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the control method of the set-top box as described in any one of claims 1 to 8.

Images