Display device, remote control device, and display device control method based on motion trajectory

Abstract

Embodiments of the present disclosure relate to the technical field of smart terminals, and disclosed are a display device, a remote control device, and a display device control method based on a motion trajectory. The method comprises: displaying a first interface; on the basis of motion information of a remote control device at the current moment, determining a motion trajectory of the remote control device from the previous moment to the current moment; matching the motion trajectory of the remote control device from the previous moment to the current moment with multiple preset motion trajectories; and if a motion trajectory of the remote control device from an initial moment to the current moment matches a target motion trajectory and the distance travelled by the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, on the basis of the target motion trajectory, generating a first control instruction corresponding to the first interface, wherein the first control instruction is used for opening a second interface corresponding to a target control. Thus, the convenience of controlling a display device by means of a remote control device can be improved.

Description

Display device, remote control device, and display device control method based on motion trajectory

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese application No. 202410353556.2, filed on March 26, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the technical field of intelligent terminals, and in particular to a display device, a remote control device, and a display device control method based on motion trajectory. Background Art

[0004] In the application scenario of smart TV, users can move the focus position of the TV through remote control devices such as remote controllers and handles, so as to select and open controls such as application software and menu options configured in the TV.

[0005] However, using a remote control to move the focus position on a TV to select a target control is rather cumbersome. Users often need to move the focus multiple times to reach the target control, and then press the confirm button on the remote control to open the target control. Therefore, the current solution for controlling a display device to open related applications using a remote control device needs to be optimized. Summary of the Invention

[0006] In a first aspect, embodiments of the present disclosure provide a display device, which may include: a display configured to display a user interface; a first communication device configured to receive motion information of a remote control device sent by the remote control device at multiple moments, wherein the motion information of the remote control device includes acceleration of the remote control device; at least one first processor connected to the display and the first communication device, and configured to execute computer instructions to cause the display device to: control the display to display a first interface; wherein the user interface includes the first interface, and the first interface includes multiple controls; and determine, based on the motion information of the remote control device at a current moment, a motion trajectory of the remote control device from a previous moment to the current moment; Wherein, the multiple moments include the current moment and the previous moment; the motion trajectory of the remote control device from the previous moment to the current moment is matched with multiple preset motion trajectories; if the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory in the multiple preset motion trajectories, and the moving distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, then a first control instruction corresponding to the first interface is generated according to the target motion trajectory; wherein, the first control instruction is used to open a second interface corresponding to the target control in the first interface, the multiple controls include the target control, and the user interface includes the second interface.

[0007] In a second aspect, an embodiment of the present disclosure provides a remote control device, which may include: a communication device configured to send a trajectory signal to the display device; at least one processor connected to the communication device, configured to execute computer instructions to enable the remote control device to execute: obtaining motion information of the remote control device; wherein the motion information of the remote control device includes the acceleration of the remote control device; determining the motion trajectory of the remote control device based on the motion information of the remote control device; determining the trajectory signal corresponding to the motion trajectory of the remote control device; controlling the communication device to send the trajectory signal to the display device so that the display device receives the trajectory signal and generates a second control instruction corresponding to the first interface according to the trajectory signal; wherein the second control instruction is used to open a second interface corresponding to a target control in the first interface, and the display device displays a user interface, the user interface includes the first interface and the second interface, the first interface includes multiple controls, and the multiple controls include the target control.

[0008] In a third aspect, an embodiment of the present disclosure also provides a display device control method based on motion trajectory, which may include: displaying a first interface; wherein the first interface includes multiple controls; receiving motion information of the remote control device sent by the remote control device at multiple moments; wherein the motion information of the remote control device includes the acceleration of the remote control device; based on the motion information of the remote control device at the current moment, determining the motion trajectory of the remote control device from the previous moment to the current moment; wherein the multiple moments include the current moment and the previous moment; matching the motion trajectory of the remote control device from the previous moment to the current moment with multiple preset motion trajectories; if the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among the multiple preset motion trajectories, and the moving distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, then generating a first control instruction corresponding to the first interface according to the target motion trajectory; wherein the first control instruction is used to open a second interface corresponding to the target control in the first interface, and the multiple controls include the target control.

[0009] In the fourth aspect, the embodiment of the present disclosure also provides a display device control method based on motion trajectory, which may include: obtaining motion information of a remote control device; wherein the motion information of the remote control device includes the acceleration of the remote control device; based on the motion information of the remote control device, determining the motion trajectory of the remote control device; determining the trajectory signal corresponding to the motion trajectory of the remote control device; sending the trajectory signal to the display device so that the display device receives the trajectory signal and generates a second control instruction corresponding to the first interface according to the trajectory signal; wherein the second control instruction is used to open a second interface corresponding to a target control in the first interface, and the display device is used to display the first interface and the second interface, and the first interface includes multiple controls, and the multiple controls include the target control. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of interaction between a display device and a remote control device according to some embodiments;

[0011] FIG2 is a block diagram illustrating a configuration of a remote control device according to some embodiments;

[0012] FIG3 is a block diagram of a hardware configuration of a display device according to some embodiments;

[0013] FIG4 is an interactive flow chart of a method for controlling a display device based on motion trajectory according to some embodiments;

[0014] FIG5 is a schematic diagram of a first interface according to some embodiments;

[0015] FIG6 is another schematic diagram of the first interface according to some embodiments;

[0016] FIG7 is another schematic diagram of the first interface according to some embodiments;

[0017] FIG8 is another schematic diagram of the first interface according to some embodiments;

[0018] FIG9 is another schematic diagram of the first interface according to some embodiments;

[0019] FIG10 is a schematic diagram of a remote control device according to some embodiments;

[0020] FIG11 is a schematic diagram of a motion trajectory of a remote control device according to some embodiments;

[0021] FIG12 is another schematic diagram of a motion trajectory of a remote control device according to some embodiments;

[0022] 13 is a flowchart of determining a motion trajectory of a remote control device from a previous moment to a current moment according to some embodiments;

[0023] FIG14 is another interactive flowchart of a method for controlling a display device based on motion trajectories according to some embodiments. DETAILED DESCRIPTION

[0024] The motion trajectory-based display device control method provided by the present disclosure is described in detail below with reference to the accompanying drawings.

[0025] The display device provided in the embodiments of the present disclosure may have various implementation forms, for example, it may be a mobile terminal, a tablet computer, a computer, a laptop computer, a television, an electronic bulletin board, an electronic table, etc. The specific form of the display device is not limited in the embodiments of the present disclosure.

[0026] Figure 1 is a schematic diagram illustrating the interaction between a display device and a remote control device according to some embodiments. As shown in Figure 1 , a user can operate a display device 200 via a mobile terminal 300 or a remote control device 100. The remote control device 100 can be a remote controller, a handle, or the like. The remote control device 100 and the display device 200 can communicate via infrared or Bluetooth protocols, or the remote control device 100 can control the display device 200 wirelessly or via other wired means.

[0027] The user can control the display device 200 by inputting user commands through buttons on the remote control device 100, voice input, a control panel, etc. For example, the user can control the display device 200 to switch displayed pages by using the up and down keys on the remote control device 100, control the video played by the display device 200 to play or pause by using the play and pause keys, and input voice commands by using the voice input key to control the display device 200 to perform corresponding operations.

[0028] In some embodiments, the user may also use a mobile terminal, tablet computer, computer, laptop computer, or other smart device to control the display device 200. For example, the user may control the display device 200 through an application installed on the smart device, which may be configured to provide the user with various controls in an intuitive user interface on a screen associated with the smart device.

[0029] In some embodiments, the mobile terminal 300 can connect and communicate with the software application installed on the display device 200 via a network communication protocol, achieving one-to-one control operations and data communication. For example, a control command protocol can be established between the mobile terminal 300 and the display device 200, and a remote control keyboard can be synchronized with the mobile terminal 300. By controlling the user interface on the mobile terminal 300, the display device 200 can be controlled. Alternatively, content displayed on the mobile terminal 300 can be transmitted to the display device 200 for synchronized display.

[0030] As shown in FIG1 , the display device 200 and the server 400 can communicate data through a variety of communication methods, allowing the display device 200 to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving messages and interacting with an electronic program guide (EPG). The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0031] The display device 200 may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection display device, a smart terminal such as a mobile phone, a tablet computer, a smart TV, a laser projection device, an electronic table, etc. The specific display device type, size, and resolution are not limited.

[0032] Figure 2 is a block diagram of a remote control device according to some embodiments. As shown in Figure 2 , the remote control device 100 may include a second processor 110, a second communication device 130, a user input / output interface 140, a memory, and a power supply. The remote control device 100 can receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200.

[0033] The second processor 110 may include an acceleration sensor (also known as an accelerometer, gravity acceleration sensor, etc.). For example, the remote control device 100 may measure one-dimensional, two-dimensional, or three-dimensional (i.e., X / Y / Z axis, XY / XZ / YZ axis, or XYZ axis) acceleration during its movement using a built-in acceleration sensor.

[0034] The second communication device 130 may include a Bluetooth module, an infrared module, a WiFi chip, a near-field communication (NFC) protocol chip, etc. The second communication device 130 may communicate with the display device 200 via the second communication device 130 .

[0035] For example, the second communication device 130 can send the motion information, trajectory detection signal, and termination detection signal of the remote control device 100 to the display device 200 via the Bluetooth module. The motion information of the remote control device 100 includes the acceleration of the remote control device 100; the trajectory detection signal can be a control signal generated by the remote control device 100 when the user presses a control button of the remote control device 100; and the termination detection signal can be a control signal generated by the remote control device 100 when the user releases the control button of the remote control device 100.

[0036] For another example, the second communication device 130 can send a trajectory signal, a trajectory detection signal, a termination detection signal, etc. to the display device 200 via the infrared signal transmitter in the infrared module. The trajectory signal is determined by the remote control device 100 according to the motion trajectory of the remote control device 100.

[0037] Taking a television as an example, Figure 3 is a hardware configuration block diagram of a display device according to some embodiments. As shown in Figure 3, the display device 200 may include: a tuner / demodulator 210, a first communication device 220, a detector 230, an external device interface 240, a first processor 250, a display 260, an audio output interface 270, and at least one of a memory, a power supply, and a user interface.

[0038] The tuner / demodulator 210 can receive broadcast television signals via wired or wireless reception, and demodulate audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The detector 230 can be used to collect signals from the external environment or external interactions.

[0039] In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the first processor 250, which can send a control signal based on user selection to enable the modem to respond to the TV signal frequency selected by the user and modulate and demodulate the TV signal carried by the frequency.

[0040] Broadcast television signals can be categorized as terrestrial, cable, satellite, or internet based on the broadcast format. They can also be categorized as digital or analog based on the modulation type. Furthermore, they can be categorized as digital or analog based on the type of signal.

[0041] In some embodiments, the first processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the first processor 250 is located, such as an external set-top box.

[0042] In some embodiments, the first communication device 220 may be a component for communicating with an external device or an external server according to various communication protocol types.

[0043] The first communication device 220 may include at least one of a Wifi module, a Bluetooth module, an infrared module, a wired Ethernet communication protocol chip, or other network communication protocol chips or near field communication protocol chips. The second communication device 130 may communicate with the remote control device 100 through the first communication device 220.

[0044] For example, the first communication device 220 may receive the motion information, trajectory detection signal, termination detection signal, etc. of the remote control device 100 sent by the remote control device 100 through the Bluetooth module.

[0045] For another example, the first communication device 220 may receive the track signal, track detection signal, termination detection signal, etc. sent by the remote control device 100 through the infrared signal receiver in the infrared module.

[0046] In some embodiments, the detector 230 may be used to collect signals from the external environment or interact with the external environment, and may include a light receiver and a temperature sensor.

[0047] Among them, the light receiver can be used as a sensor to collect the intensity of ambient light, and to adaptively adjust display parameters according to the intensity of ambient light; the temperature sensor can be used to sense the ambient temperature, so that the display device 200 can adaptively adjust the display color temperature of the image. For example, when the ambient temperature is high, the color temperature of the image displayed by the display device 200 can be adjusted to a cooler color, or when the ambient temperature is low, the color temperature of the image displayed by the display device 200 can be adjusted to a warmer color.

[0048] In some embodiments, the detector 230 may also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, as well as to collect user attributes or gestures for interacting with the user. It can adaptively change display parameters and recognize user gestures to realize the function of interacting with the user.

[0049] In some embodiments, the detector 230 may further include a sound collector, such as a microphone, for receiving the user's voice, for example, a voice signal including a user's control command for controlling the display device 200, or collecting ambient sound for identifying the type of ambient scene, so that the display device 200 can adapt to the ambient noise.

[0050] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more interfaces such as a High Definition Multimedia Interface (HDMI), an analog or digital high-definition component input interface, a composite video input interface, a Universal Serial Bus (USB) input interface, an RGB port, or a composite input / output interface formed by multiple interfaces mentioned above.

[0051] As shown in FIG3 , the first processor 250 may include a central processing unit (CPU), a video processor, an audio processor, a graphics processor, a random access memory (RAM), a read-only memory (ROM), and at least one of a first interface to an nth interface for input / output. A communication bus connects the various components.

[0052] In some embodiments, the first processor 250 can control the operation of the display device and respond to user operations through various software control programs stored in the external memory. For example, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface recognizes the sounds or gestures through sensors and receives the user input commands.

[0053] A "user interface" is the medium interface for interaction and information exchange between an application or operating system and a user. It converts information between its internal form and a form acceptable to the user. A graphical user interface (GUI) is a common form of user interface, which refers to a graphical user interface related to computer operations. It can be an icon, window, control, or other interface element displayed on a display device. A control can include at least one of the following visual interface elements: icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets (web widgets).

[0054] In some embodiments, RAM can be used to store temporary data for the operating system or other running programs; ROM can be used to store various system startup instructions, such as instructions for starting a basic input / output system (BIOS). ROM can be used to perform a power-on self-test on the system, initialize various functional modules in the system, run basic input / output drivers for the system, and boot the operating system.

[0055] In some embodiments, upon receiving a power-on signal, the display device 200 starts up, and the central processing unit executes system startup instructions in the ROM, copying temporary operating system data stored in the memory to the RAM to facilitate booting or running the operating system. After the operating system boot is complete, the central processing unit copies temporary data for various application programs in the memory to the RAM to facilitate booting or running the various application programs.

[0056] In some embodiments, the central processing unit can be used to execute operating system and application instructions stored in the memory, and execute various applications, data and content based on various interactive instructions received from external input, so as to ultimately display and play various audio and video content.

[0057] In some embodiments, the central processing unit may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform certain operations of the display device 200 in pre-power-on mode and / or display images in normal mode. The one or more sub-processors are used to perform certain operations in states such as standby mode.

[0058] In some embodiments, the video processor can be configured to receive an external video signal and perform decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, transparency setting, image synthesis and other video processing according to the standard codec protocol of the input signal to obtain a signal that can be directly displayed or played on the display device 200.

[0059] In some embodiments, the video processor may include a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

[0060] Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if the input Moving Picture Experts Group-2 (MPEG-2) standard is input, the demultiplexing module demultiplexes it into video signals and audio signals, etc.; the video decoding module is used to process the demultiplexed video signal, including decoding and scaling processing, transparency setting, etc.

[0061] An image synthesis module, such as an image synthesizer, can be used to overlay and mix the GUI signal generated by the graphics generator based on user input or itself with the scaled video image to generate an image signal for display. A frame rate conversion module is used to convert the input video frame rate, such as converting a 60Hz frame rate to a 120Hz or 240Hz frame rate. This is typically achieved using a format such as interpolation. A display formatting module is used to convert the received frame rate-converted video output signal to conform to the display format, such as outputting an RGB data signal.

[0062] In some embodiments, the audio processor can be used to receive external audio signals, perform decompression and decoding according to the standard codec protocol of the input signal, as well as noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

[0063] In some embodiments, the video processor may include one or more chips. The audio processor may also include one or more chips. Furthermore, the video processor and audio processor may be separate chips or integrated with the processor into one or more chips.

[0064] In some embodiments, the input / output interface can be used for audio output, that is, receiving the sound signal output by the audio processor under the control of the first processor 250 and outputting it to an external device such as a speaker. In addition to the speaker carried by the display device 200 itself, it can also be output to an external audio output terminal of a generating device of an external device, such as an external audio interface or a headphone jack. The audio output can also include a short-range communication module in the communication interface, such as a Bluetooth module for outputting sound to a Bluetooth-connected speaker.

[0065] In some embodiments, a graphics processor can be used to generate various graphical objects, such as icons, operation menus, and graphics displayed in response to user input commands. The graphics processor can include an arithmetic unit that receives various interactive commands from the user, performs calculations, and displays various objects based on display attributes. It can also include a renderer that renders the various objects generated by the arithmetic unit. The rendered objects are then displayed on a display.

[0066] In some embodiments, the graphics processor and the video processor can be integrated or separately set. When integrated, they can perform processing of graphic signals output to the display. When separately set, they can perform different functions respectively, such as a graphics processor (GPU) + frame rate conversion technology (FRC) architecture.

[0067] The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen.

[0068] In some embodiments, the display 260 can be used to display a user interface, such as an interface corresponding to a display device. For example, the display interface can be a channel search interface in a display device, or a display interface of an application.

[0069] In some embodiments, the display 260 may be configured to receive audio and video signals output by the audio processor and the video processor, display video content and images, play audio of the video content, and display components of a menu control interface.

[0070] In some embodiments, the display 260 may be used to present a user operation UI interface generated in the display device 200 and used to control the display device 200 .

[0071] In some embodiments, the display device 200 can establish transmission and reception of control signals and data signals with the remote control device 100 through the first communication means 220 .

[0072] In some embodiments, the memory may include various software modules for driving the display device 200. For example, the various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0073] The basic module is a low-level software module used to communicate signals between various hardware components in the display device 200 and send processing and control signals to upper-level modules. The detection module is a management module used to collect various information from various sensors or user input interfaces, perform digital-to-analog conversion, and perform analysis and management.

[0074] The display control module controls the display to display image content, playing multimedia image content and UI interface information. The communication module controls and communicates data with external devices. The browser module performs data communication between browsers and servers. The service module provides various services and applications. The memory can also store and receive external data, user data, images of various user interface items, and visual effects of focused objects.

[0075] In some embodiments, the user interface may be configured to receive infrared control signals sent by a remote control device 100 , such as an infrared remote controller.

[0076] The power supply can provide power to the display device 200 through power input from an external power source under the control of the first processor 250 .

[0077] In some embodiments, the display device 200 may receive a query command input by a user through the first communication device 220. For example, when the first communication device 220 is a touch-sensitive component, the touch-sensitive component and the display 260 may together form a touch screen. On the touch screen, a user may input different control commands through touch operations, such as click, slide, long press, double-click, and other touch commands, and different touch commands may represent different control functions.

[0078] To implement the above-mentioned different touch actions, the touch component can generate different electrical signals when the user inputs different touch actions, and send the generated electrical signals to the first processor 250. The first processor 250 can extract features from the received electrical signals and determine the control function to be executed by the user based on the extracted features.

[0079] For example, when the user inputs a click touch action at the search position in the display interface, the touch component will sense the touch action and generate an electrical signal. After receiving the electrical signal, the first processor 250 can first judge the duration of the level corresponding to the touch action in the electrical signal. When the duration is less than the preset time threshold, it recognizes that the user input is a click touch instruction. The first processor 250 then extracts the position features generated by the electrical signal to determine the touch position. When the touch position is within the search position range, it is determined that the user has input a click touch instruction at the search position. Then, the first processor 250 can start the media resource search function and receive search instructions input by the user, such as search keywords, voice search instructions, etc.

[0080] In some embodiments, the user can trigger a query operation through a specific gesture operation on the touch screen. For example, when the user performs two consecutive double-click operations on the display interface, the first processor 250 can judge the interval time between the two consecutive double-clicks. When the interval time is less than a preset time threshold, it is recognized that the user input is a continuous double-click operation, and it is determined that the user triggered the media resource search operation.

[0081] In some embodiments, a user may input a voice command through a touch operation on a touch screen, for example, a user may trigger a voice query operation on the display 260 through a voice trigger gesture.

[0082] In some embodiments, the first communication device 220 may also be an external control component, such as a mouse or remote control that establishes a communication connection with the display device. When the user performs different control operations on the external control component, the external control component may generate different control signals in response to the user's control operations and send the generated control signals to the first processor 250. The first processor 250 may extract features from the received control signals and determine the control function the user intends to perform based on the extracted features.

[0083] For example, when a user clicks the left mouse button anywhere on the channel display interface using an external control component, the external control component can sense the control action and generate a control signal. Upon receiving the control signal, the first processor 250 can identify the user input via the external control component as a click command based on the duration of the control action at the location in the control signal. If the duration is less than a preset time threshold, the processor can determine that the user input via the external control component is a click command. In the current scenario, a click command is used to trigger the input function of a query command or to switch media resource pages.

[0084] For another example, when the user presses the voice button on the remote control, the remote control can start the voice input function. During the process of the user inputting the voice command, the remote control can synchronize the voice command to the display 260. At this time, the display 260 can display the voice input mark to indicate that the user is inputting the voice command.

[0085] In some embodiments, the first communication device 220 may also be a control component connected to the display 260. For example, in a desktop computer, the control component may be a keyboard connected to the display. The user may input various control commands through the keyboard, such as media resource switching commands, query commands, etc.

[0086] In some embodiments, the user may input click commands, voice commands, etc. through corresponding shortcut keys. For example, the user may trigger a sliding operation by selecting the "Tab" key and the direction keys. That is, when the user simultaneously selects the "Tab" key and the direction keys on the keyboard, the first processor 250 may receive the key signal and determine that the user has triggered an operation to switch in the direction corresponding to the direction key. Then, the first processor 250 may control the display interface to turn pages or scroll in the media resource display page to display the corresponding media resource options.

[0087] Correspondingly, the user can also input voice commands through corresponding shortcut keys. For example, when the user selects the "Ctrl" key and the "V" key, the first processor 250 can receive the key signal and determine that the user has triggered a voice search operation. Then, the first processor 250 can receive the voice command input by the user and control the display 260 to perform corresponding operations according to the voice command, such as displaying the query result page corresponding to the voice command.

[0088] Figure 4 is an interactive flow chart of a method for controlling a display device based on motion trajectory according to some embodiments, which can be applied to the display device and remote control device shown in Figure 1. The display device and the remote control device can communicate via the Bluetooth protocol.

[0089] The display device may include a display, a first communication device, and at least one first processor connected to the display and the first communication device respectively.

[0090] In some embodiments, the display can be used to display a user interface. A user interface is an interactive interface between a user and a display device. The user can send instructions to the display device through control operations, such as touch operations and gesture operations, to perform certain tasks. The user interface has a comprehensive interactive design that allows users to easily complete tasks.

[0091] In some embodiments, the first communication device can be configured to receive motion information of the remote control device sent by the remote control device at multiple times, wherein the motion information of the remote control device includes the acceleration of the remote control device; the first processor can receive the motion information of the remote control device sent by the remote control device at multiple times.

[0092] The remote control device may include a second communication device and at least one second processor connected to the second communication device.

[0093] In some embodiments, the second communication device may be configured to send the motion information of the remote control device to the display device at multiple times, and the second processor may obtain the motion information of the remote control device at multiple times.

[0094] In a motion trajectory-based display device control method provided by an embodiment of the present disclosure, the display can be controlled to display a user interface; wherein the user interface includes a first interface and a second interface, and multiple controls are displayed in the first interface; motion information of the remote control device sent by the remote control device at multiple moments is received; wherein the motion information of the remote control device includes the acceleration of the remote control device; based on the motion information of the remote control device at the current moment, the motion trajectory of the remote control device from the previous moment to the current moment is determined; wherein the multiple moments include the current moment and the previous moment; and the motion trajectory of the remote control device from the previous moment to the current moment is matched with multiple preset motion trajectories; if the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory in the multiple preset motion trajectories, and the moving distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, then a first control instruction corresponding to the first interface is generated according to the target motion trajectory, thereby opening the second interface corresponding to the target control in the first interface based on the first control instruction; wherein the multiple controls include the target control.

[0095] The display device can determine the motion trajectory of the remote control device at the current moment in real time based on the acceleration of the remote control device, and match the motion trajectory of the remote control device at the current moment with the preset motion trajectory in real time, so that when the match is successful and the movement distance of the remote control device meets certain conditions, a corresponding first control instruction is generated to open the second interface corresponding to the target control in the first interface. There is no need to move the focus position in the display device interface. The user can directly control the remote control device to move according to a certain motion trajectory to open the corresponding control in the display device, thereby improving the convenience of controlling the display device through the remote control device. To facilitate understanding of the above process, the following steps can be included as shown in Figure 4:

[0096] S410: The display device controls the display to display a first interface.

[0097] The first interface includes multiple controls.

[0098] In some embodiments, the display device can control the display to display a first interface. The first interface can display multiple controls, including applications installed on the display device, menu options, program columns, etc. The user can open the first interface of the display device by controlling a remote control device or other means.

[0099] In some embodiments, the first interface of the display device may be as shown in FIG5 , where a first page 501 displays multiple controls, such as control A, control B, control C, control D, control E, control F, control G, control H, and so on. A user may open the first interface 501 by controlling a remote control device, for example. For example, the user may press a button on the remote control device to open the first interface 501 displaying the multiple controls.

[0100] In addition, the user can customize the controls displayed in the first interface 510. For example, the user can set controls such as Bluetooth configuration, Wi-Fi connection, application A, application B, etc. to be displayed in the first interface 510, so that the above controls can be quickly opened in the first interface 510.

[0101] S420: The remote control device obtains motion information of the remote control device at every preset sampling interval.

[0102] The motion information of the remote control device includes the acceleration of the remote control device.

[0103] In some embodiments, a control button may be provided in the remote control device, so that when the user operates the control button, the function of the remote control device to obtain its own motion information is triggered.

[0104] In some embodiments, the control button can be configured to: trigger the display device to open the first interface when the user presses or long presses the control button; and trigger the remote control device to obtain its own motion information and send a trajectory detection signal to the display device.

[0105] The motion information of the remote control device may include its acceleration, and the remote control device may obtain its own acceleration information via a built-in accelerometer. Specifically, when a user presses or long-presses a control button, the display device opens the first interface, and the remote control device responds to the user's operation of the control button and begins to obtain its own motion information and send a trajectory detection signal to the display device.

[0106] In some embodiments, as shown in FIG10 , a trajectory detection button 10 (i.e., the aforementioned control button) may be provided in the remote control device. The user may press or long-press the trajectory detection button 10 to trigger the remote control device to obtain its own acceleration information through a built-in acceleration sensor.

[0107] In some embodiments, after pressing a control button or while long pressing the control button, the user can control the remote control device to move in various directions in a straight line or arc trajectory, and the remote control device can obtain the acceleration corresponding to each moment of the remote control device during the movement through the built-in acceleration sensor.

[0108] In some embodiments, as shown in FIG11 , the user can control the remote control device to move in a straight line within the same plane. For example, the user can control the remote control device to move horizontally to the left or right, vertically up or down, or diagonally to the upper right, etc.

[0109] It should be noted that the straight line directions shown in FIG11 are only examples, and the actual straight line directions in which the remote control device can move are not limited to the directions shown in FIG11 , and the starting point of the movement of the remote control device is not a fixed point.

[0110] As shown in Figure 12, the user can control the remote control device to move along regular or irregular arcs. As shown in Figure 12 (a) and (b), the user can control the remote control device to move along an arc in any direction within the same plane. As shown in Figure 12 (c) and (d), the user can also control the remote control device to move along an arc in any direction within different planes.

[0111] It should be noted that the arc direction shown in FIG12 is only an example, and the arc direction in which the remote control device can actually move is not limited to the direction shown in FIG12 , and the starting point of the movement of the remote control device is not a fixed point.

[0112] In some embodiments, since the communication mode between the remote control device and the display device is Bluetooth mode, the remote control device can obtain its own motion information in real time, and thus can send its own motion information to the display device in real time.

[0113] In some embodiments, the remote control device may obtain its own motion information at certain sampling intervals (eg, 0.05s, 0.1s, 0.2s, etc.) The embodiment of the present disclosure does not limit the value of the sampling interval.

[0114] In some embodiments, the acceleration acquired by the remote control device can be a single-dimensional acceleration (for example, acceleration in any dimension of the X-axis, Y-axis, or Z-axis), or a three-dimensional acceleration (for example, acceleration in the X-axis, Y-axis, and Z-axis), etc., and the embodiments of the present disclosure are not limited to this.

[0115] S430: The remote control device sends movement information of the remote control device to the display device at multiple times.

[0116] In some embodiments, the remote control device can transmit the acquired motion information to the display device via the Bluetooth protocol at various times. Each time can be a time corresponding to a preset time interval, or a time corresponding to each time the remote control device moves a predetermined distance, etc. This disclosure is not limited to this.

[0117] The remote control device may send real-time motion information to the display device at regular intervals, where the intervals may be greater than the sampling intervals. For example, the intervals may be 0.1s, 0.3s, 0.5s, and so on. The remote control device may also send real-time motion information to the display device every time the remote control device moves a preset distance, where the preset distances may be 0.5cm, 0.8cm, 1cm, and so on. The embodiments of this disclosure do not limit the values ​​of the time intervals and preset distances.

[0118] In some embodiments, in a Bluetooth communication mode, the remote control device may use JSON (JavaScript Object Notation) data format to send its own motion information to the display device.

[0119] For example, a possible JSON data format for sports information is as follows:

[0120] Here, "acceleration" is the acceleration of the remote control device at a given moment, and "x," "y," and "z" are the acceleration of the remote control device in the X, Y, and Z axes, respectively. As can be seen, the acceleration of the remote control device in the X axis is 0.5, the acceleration in the Y axis is 0.2, and the acceleration in the Z axis is 0.8.

[0121] S440: The display device receives motion information of the remote control device sent by the remote control device at multiple times.

[0122] In some embodiments, the display device may receive the motion information of the remote control device sent at the aforementioned moments in real time. That is, the display device may receive the motion information of the remote control device sent at the aforementioned moments.

[0123] S450: The display device determines a motion trajectory of the remote control device from a previous moment to the current moment based on the motion information of the remote control device at the current moment.

[0124] The multiple moments include the current moment and the previous moment.

[0125] In some embodiments, when the user presses or long presses the above-mentioned control button in the remote control device, the remote control device can send a trajectory detection signal sent by the user through the control button to the display device, so that the display device receives and responds to the trajectory detection signal, thereby receiving the motion information of the remote control device sent by the remote control device at multiple times, and based on the motion information of the remote control device at the current moment, determining the motion trajectory of the remote control device from the previous moment to the current moment.

[0126] In some embodiments, as shown in FIG13 , the processor may determine the motion trajectory of the remote control device from the previous moment to the current moment by:

[0127] S1310: Determine the speed of the remote control device at the current moment based on the acceleration of the remote control device at the current moment.

[0128] In some embodiments, after responding to the trajectory detection signal, the display device may first initialize preset variables, which may include speed (including speed in each dimension), movement distance (including movement distance in each dimension), previous moment, etc.

[0129] The initialization of the speed of each dimension can be: velocityX=0 (that is, the speed of the X axis is initialized to 0); velocityY=0 (that is, the speed of the Y axis is initialized to 0); velocityZ=0 (that is, the speed of the Z axis is initialized to 0).

[0130] The initialization of the moving distance of each dimension can be: distanceX=0 (ie, the moving distance of the X axis is initialized to 0); distanceY=0 (ie, the moving distance of the Y axis is initialized to 0); distanceZ=0 (ie, the moving distance of the Z axis is initialized to 0).

[0131] Initialization of the previous moment can be: lastTime = System.currentTimeMillis(), which calls the system time in the display device to initialize the current time. It can be understood that the previous moment after initialization is also the initial moment, which can be the moment when the display device first receives motion information sent by the remote control device after receiving the trajectory detection signal.

[0132] In some embodiments, after initializing the preset variables, the display device receives the acceleration of the remote control device at the current moment, and determines the speed of the remote control device at the current moment based on the acceleration at the current moment.

[0133] The display device may first determine the current moment when the acceleration is received, and determine the time interval between the current moment and the corresponding previous moment.

[0134] For example, when the display device receives acceleration, it can call the system time to determine the current time (currentTime), that is, currentTime = System.currentTimeMillis(); and determine the time interval (deltaTime, dt) based on the difference between the current time and the previous time (lastTime), that is, dt = (currentTime-lastTime).

[0135] Furthermore, after determining the current time interval, the display device may assign the value of the current moment currentTime to the previous moment lastTime, ie, lastTime=currentTime, so as to calculate the corresponding time interval dt when receiving the next acceleration.

[0136] Next, the display device determines the speed of the remote control device at the current moment based on the acceleration of the remote control device at the current moment and the current time interval.

[0137] For example, the display device may determine the velocity (currentVelocity) at the current moment by multiplying the product of the current acceleration (currentAcceleration) and the current time interval, and the velocity (lastVelocity) at the previous moment, ie, currentVelocity=lastVelocity+currentAcceleration*dt.

[0138] In some embodiments, the current velocity of the remote control device on the X, Y, and Z axes can be determined by: velocityX+=ax*dt; velocityY+=ay*dt; velocityZ+=az*dt;

[0139] Where velocityX is the current velocity of the remote control device on the X axis, ax is the current acceleration of the remote control device on the X axis; velocityY is the current velocity of the remote control device on the Y axis, ay is the current acceleration of the remote control device on the Y axis; velocityZ is the current velocity of the remote control device on the Z axis, az is the current acceleration of the remote control device on the Z axis.

[0140] S1320: Determine a moving distance of the remote control device from the initial moment to the current moment based on the speed of the remote control device at the current moment.

[0141] In some embodiments, after determining the speed of the remote control device at the current moment, the display device may determine the movement distance of the remote control device from the initial moment to the current moment based on the speed at the current moment.

[0142] The display device can determine the movement distance (currentDistance) of the remote control device from the initial moment to the current moment based on the movement distance of the remote control device from the initial moment to the previous moment, the product of the velocity at the current moment (currentVelocity) and the current time interval, and the product of the acceleration at the current moment and the current time interval, that is, currentDistance = lastDistance + currentVelocity*deltaTime + 0.5*currentAcceleration*dt*dt.

[0143] For example, a possible way to determine the current movement distance of the remote control device on the X axis, Y axis, and Z axis is as follows: distanceX+=velocityX*dt+0.5*ax*dt*dt; distanceY+=velocityY*dt+0.5*ay*dt*dt; distanceZ+=velocityZ*dt+0.5*az*dt*dt

[0144] Wherein, distanceX is the moving distance of the remote control device on the X axis at the current moment, distanceY is the moving distance of the remote control device on the Y axis at the current moment, and distanceZ is the moving distance of the remote control device on the Z axis at the current moment.

[0145] In a case where the determined movement distance is a movement distance in multiple directions, the display device may determine the overall movement distance of the remote control device based on the movement distances in the multiple directions.

[0146] Taking the determined movement distance as the movement distance along the X-axis, Y-axis, and Z-axis from the initial moment to the current moment as an example, a possible way to determine the total movement distance of the remote control device from the initial moment to the current moment is as follows: totalDistance = Math.sqrt(distanceX*distanceX+distanceY*distanceY+distanceZ*distanceZ)

[0147] Here, totalDistance is the total movement distance of the remote control device from the initial moment to the current moment.

[0148] S1330: Determine a movement direction of the remote control device at the current moment based on a movement distance of the remote control device from the initial moment to the current moment.

[0149] In some embodiments, the display device may then determine the direction of movement of the remote control device at the current moment based on the movement distance of the remote control device from the initial moment to the current moment, thereby further determining the movement trajectory of the remote control device from the previous moment to the current moment.

[0150] The display device can determine the arc of motion of the remote control device at the current moment based on the movement distance of the remote control device from the initial moment to the current moment. It is understandable that the direction of motion of the remote control device can be determined by determining the arc of motion of the remote control device.

[0151] For example, a possible way to determine the arc of motion of the remote control device at the current moment is as follows: radianXY = Math.atan2(distanceY, distanceX); radianXZ = Math.atan2(distanceZ, distanceX); radianYZ = Math.atan2(distanceZ, distanceY)

[0152] Among them, radianXY is the movement arc of the remote control device on the XY plane at the current moment, radianXZ is the movement arc of the remote control device on the XZ plane at the current moment, and radianYZ is the movement arc of the remote control device on the YZ plane at the current moment.

[0153] S1340: Determine a movement trajectory of the remote control device from a previous moment to a current moment based on the movement distance and movement direction of the remote control device.

[0154] In some embodiments, after determining the movement distance and movement direction of the remote control device, the display device can determine the movement trajectory of the remote control device from the previous moment to the current moment based on the movement distance and movement direction of the remote control device.

[0155] S460: The display device matches the motion trajectory of the remote control device from the previous moment to the current moment with a plurality of preset motion trajectories.

[0156] In some embodiments, the display device may be configured with multiple preset motion trajectories, so that after determining the motion trajectory of the remote control device from the previous moment to the current moment, the motion trajectory is matched with the multiple preset motion trajectories. In other words, the display device can match the motion trajectory of the remote control device with the multiple preset motion trajectories in real time.

[0157] Each preset motion track is used to open a corresponding control in the display device. That is, the user controls the remote control device to move according to the preset motion track, and when the motion track of the remote control device meets certain conditions, the corresponding control in the display device can be opened.

[0158] In some embodiments, the method for determining whether a match exists may be to determine whether the motion trajectory of the remote control device overlaps with a preset motion trajectory to a certain extent. In other words, the motion trajectory of the remote control device may be determined to match the preset motion trajectory when the degree of overlap between the motion trajectory of the remote control device and the preset motion trajectory is greater than or equal to a certain matching threshold; or, the motion trajectory of the remote control device may be determined to match the preset motion trajectory when the motion trajectory of the remote control device completely overlaps with the preset motion trajectory. The disclosed embodiments do not limit the method for determining whether the motion trajectory of the remote control device matches the preset motion trajectory.

[0159] In some embodiments, the display device may display the plurality of preset motion trajectories in the first interface, so that the user can clearly know the motion trajectory according to which the remote control device should be controlled to move, and open the corresponding control in the first interface.

[0160] In some embodiments, as shown in Figure 6, the display device can display multiple preset motion trajectories in the first interface 501, for example: preset motion trajectory a (for opening control A), preset motion trajectory b (for opening control B), preset motion trajectory c (for opening control C), preset motion trajectory d (for opening control D), preset motion trajectory e (for opening control E), preset motion trajectory f (for opening control F), preset motion trajectory g (for opening control G) and preset motion trajectory h (for opening control H).

[0161] It should be noted that the preset motion trajectories shown in FIG6 are merely examples, and the starting point of each preset motion trajectory may also be located at other positions in the first interface. The embodiment of the present disclosure does not limit the shape and direction of the preset motion trajectory.

[0162] S470: If the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among multiple preset motion trajectories, and the movement distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, the display device generates a first control instruction corresponding to the first interface according to the target motion trajectory.

[0163] In some embodiments, if the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among multiple preset motion trajectories, the display device can display the motion trajectory in real time in the first interface based on the determined motion trajectory of the remote control device from the previous moment to the current moment, thereby facilitating the user to check whether the current motion trajectory of the remote control device matches the preset motion trajectory, and guiding the user to control the remote control device to move according to the displayed preset motion trajectory, so as to improve the success rate of matching the motion trajectory of the remote control device with the preset motion trajectory.

[0164] In some embodiments, as shown in Figure 7, the display device may display the motion trajectory a of the remote control device in real time on the first interface 501. At the current moment, the motion trajectory a matches the preset motion trajectory e.

[0165] In some embodiments, the user can control the remote control device to stop acquiring its own motion information by lifting or pressing a control button on the remote control device again, thereby controlling the display device to stop detecting the motion trajectory of the remote control device. For example, when the user lifts or presses the control button on the remote control device again, the remote control device can respond to the user's operation of the control button and stop acquiring its own motion information, and send a detection termination signal to the display device.

[0166] In some embodiments, when the display device receives a termination detection signal sent by the remote control device, or the movement distance of the remote control device from the initial moment to the current moment is greater than a first preset distance, if the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory, and the movement distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, the display device generates a first control instruction corresponding to the first interface according to the target motion trajectory.

[0167] The first preset distance may be close to the length of the preset motion trajectory. For example, if the length of the preset motion trajectory is 15 cm, the first preset distance may be 14 cm to 16 cm, etc. The embodiment of the present disclosure does not limit the specific value of the first preset distance.

[0168] That is, when the display device receives a termination detection signal, or when the motion trajectory of the remote control device reaches a sufficient length, if the motion trajectory of the remote control device consistently matches the target motion trajectory, and the length of the motion trajectory of the remote control device is greater than or equal to the length of the target motion trajectory, then it can be determined that the motion trajectory of the remote control device satisfies the conditions for opening the corresponding control in the first interface. Thus, the display device can generate a first control instruction corresponding to the first interface based on the target motion trajectory, and open the target control corresponding to the target motion trajectory according to the first control instruction.

[0169] In some embodiments, as shown in Figure 8, the motion trajectory a of the remote control device always matches the preset motion trajectory e (i.e., the target motion trajectory), and the display device can generate a first control instruction corresponding to the first interface 501 according to the preset motion trajectory e to open the target control E according to the first control instruction.

[0170] S480: The display device controls the display to display the second interface corresponding to the target control in the first interface according to the first control instruction.

[0171] In some embodiments, after the display device generates the first control instruction, it can control the display to display the second interface corresponding to the above-mentioned target control according to the first control instruction, that is, open the second interface corresponding to the target control according to the first control instruction.

[0172] In the embodiment of the present disclosure, in the Bluetooth communication mode, the display device can determine the motion trajectory of the remote control device at the current moment in real time based on the acceleration of the remote control device, and match the motion trajectory of the remote control device with multiple preset motion trajectories in real time, so that when the target motion trajectory among the multiple preset motion trajectories is successfully matched and the movement distance of the remote control device is greater than the first preset distance, a corresponding first control instruction is generated to open the second interface corresponding to the target control in the first interface. Therefore, the present disclosure does not require the user to move the focus position in the display device interface, nor does it require the user to exit the interface and then open the corresponding control by moving the focus position when the display device displays other interfaces. The user can directly open the first interface through the control button in the remote control device and control the remote control device to move according to a certain motion trajectory to open the corresponding control in the first interface, thereby improving the convenience of controlling the display device through the remote control device.

[0173] In addition, the display device can also display the above-mentioned multiple preset motion trajectories in the first interface, so that the user can clearly know according to which motion trajectory the remote control device should be controlled to move, and open the corresponding controls in the first interface; in addition, the display device can also display the motion trajectory of the remote control device in real time in the first interface when the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory, so as to facilitate the user to check whether the current motion trajectory of the remote control device matches the preset motion trajectory, and can guide the user to control the remote control device to move according to the displayed preset motion trajectory, so as to improve the success rate of matching the motion trajectory of the remote control device with the preset motion trajectory.

[0174] Figure 14 shows another interactive flow chart of a motion trajectory-based display device control method provided by an embodiment of the present disclosure, which can be applied to the display device and remote control device shown in Figure 1. The display device and the remote control device communicate via infrared protocol.

[0175] The display device may include a display, a first communication device, and at least one first processor connected to the display and the first communication device respectively.

[0176] In some embodiments, the first communication device may be configured to receive a trajectory signal transmitted by a remote control device, and the first processor may receive the trajectory signal and generate a second control instruction corresponding to the first interface based on the trajectory signal. The second control instruction is used to open a second interface corresponding to a target control in the first interface, and the display device displays a user interface, the user interface including the first interface and the second interface, the first interface including multiple controls, and the multiple controls including the target control.

[0177] The remote control device may include a second communication device and a second processor connected to the second communication device.

[0178] In some embodiments, the second communication device may be configured to send a trajectory signal to the display device, and the second processor may obtain motion information of the remote control device and determine a trajectory signal corresponding to the motion trajectory of the remote control device based on the motion information of the remote control device. The motion information of the remote control device includes the acceleration of the remote control device.

[0179] Through the above process, the motion information of the remote control device can be obtained, and based on the motion information of the remote control device, the motion trajectory of the remote control device can be determined; then, the trajectory signal corresponding to the motion trajectory of the remote control device is determined, and the second communication device is controlled to send the trajectory signal to the display device, so that the display device receives the trajectory signal and generates a second control instruction corresponding to the first interface according to the trajectory signal, thereby opening the second interface corresponding to the target control in the first interface according to the second control instruction.

[0180] The remote control device of the disclosed embodiment can determine the motion trajectory of the remote control device and the corresponding trajectory signal based on the acquired motion information of the remote control device; and transmit the trajectory signal to the display device, so that the display device generates a corresponding second control instruction based on the trajectory signal to open the second interface corresponding to the target control in the first interface. As a result, the disclosed embodiment does not require the user to press the relevant button of the remote control device to move the focus position in the display device interface. Instead, the remote control device can be directly controlled to move according to a certain motion trajectory to open the corresponding control in the display device, thereby improving the convenience of the user when controlling the display device through the remote control device.

[0181] As shown in FIG14 , the display device and the remote control device are configured to perform the following steps S1410 to S1460:

[0182] S1410: The display device controls the display to display the first interface.

[0183] In some embodiments, the display device can control the display to display a first interface. The first interface can display multiple controls, which may include applications installed on the display device, menu options, program columns, etc. The user can open the multiple controls on the display device by controlling a remote control device or other means.

[0184] It can be understood that the relevant implementation of step 910 can refer to the relevant embodiment of step 410, and will not be repeated here.

[0185] S1420: The remote control device obtains motion information of the remote control device.

[0186] In some embodiments, a control button may be provided in the remote control device, so that when the user operates the control button, the function of the remote control device to obtain its own motion information is triggered.

[0187] In some embodiments, the control button can be configured to trigger the remote control device to acquire its own motion information when the user presses or long-presses the control button. The motion information of the remote control device may include its acceleration, which the remote control device can acquire via a built-in accelerometer. In other words, when the user presses or long-presses the control button, the remote control device responds to the user's operation and begins acquiring its own motion information.

[0188] In some embodiments, after pressing a control button or while long pressing the control button, the user can control the remote control device to move in various directions in a straight line or arc trajectory, and the remote control device can obtain the acceleration corresponding to each moment of the remote control device during the movement through the built-in acceleration sensor.

[0189] In some embodiments, the remote control device may obtain its own motion information at certain sampling intervals (eg, 0.05s, 0.1s, 0.2s, etc.) The embodiment of the present disclosure does not limit the value of the sampling interval.

[0190] In some embodiments, the acceleration acquired by the remote control device can be a single-dimensional acceleration (for example, acceleration in any dimension of the X-axis, Y-axis, or Z-axis), or a three-dimensional acceleration (for example, acceleration in the X-axis, Y-axis, and Z-axis), etc., and the embodiments of the present disclosure are not limited to this.

[0191] It can be understood that the relevant implementation of step 920 can refer to the relevant embodiment in step 420, which will not be repeated here.

[0192] S1430: The remote control device determines a motion trajectory of the remote control device based on the motion information of the remote control device.

[0193] In some embodiments, when the user releases or presses the control button on the remote control device again, the remote control device may respond to the user's operation of the control button and stop acquiring its own motion information. The remote control device may then determine the motion trajectory of the remote control device based on the acquired motion information.

[0194] Alternatively, the remote control device may also determine the motion trajectory of the remote control device based on the motion information obtained by the remote control device when the moving distance of the remote control device is greater than the second preset distance.

[0195] The second preset distance may be close to the length of the preset motion trajectory. For example, if the length of the preset motion trajectory is 15 cm, the second preset distance may be 14 cm to 16 cm, etc. The embodiment of the present disclosure does not limit the specific value of the second preset distance.

[0196] In some embodiments, the way in which the remote control device determines its motion trajectory based on motion information is the same as the principle in which the display device determines the motion trajectory of the remote control device based on motion information, that is, the relevant implementation methods of step 930 can refer to the relevant embodiments in step 450 and will not be repeated here.

[0197] S1440: The remote control device determines a trajectory signal corresponding to the motion trajectory of the remote control device.

[0198] In some embodiments, the remote control device may be pre-set with multiple trajectory keys (i.e., preset trajectory keys). For example, the horizontal leftward motion trajectory in Figure 11 corresponds to the preset trajectory key KEY_1, and the vertical upward motion trajectory corresponds to the preset trajectory key KEY_2; for another example, the motion trajectory (a) in Figure 12 corresponds to the preset trajectory key KEY_3, the motion trajectory (c) corresponds to the preset trajectory key KEY_4, and so on.

[0199] In some embodiments, after the remote control device determines its motion trajectory based on the motion information, it can determine a target trajectory key value that matches its motion trajectory from the plurality of preset trajectory key values. For example, if the motion trajectory of the remote control device is a horizontal leftward motion trajectory, the target trajectory key value that matches its motion trajectory is the preset trajectory key value KEY_1.

[0200] In some embodiments, after the remote control device determines the target trajectory key value that matches its motion trajectory, it can further determine a trajectory signal corresponding to the target trajectory key value, wherein the trajectory signal is an infrared signal.

[0201] S1450: The remote control device sends a track signal to the display device.

[0202] In some implementations, after the remote control device determines the trajectory signal, it can send the trajectory signal to the display device via an infrared protocol.

[0203] S1460: The display device receives the trajectory signal.

[0204] In some implementations, the display device may receive the track signal sent by the remote control device via an infrared protocol.

[0205] S1470: The display device generates a second control instruction corresponding to the first interface according to the trajectory signal.

[0206] In some embodiments, the display device may display multiple preset motion trajectories in the first interface, each of which corresponds to a corresponding preset trajectory key value. This allows the user to clearly understand the motion trajectory to control the remote control device and open the corresponding control in the first interface.

[0207] In some embodiments, as shown in Figure 6, the display device can display multiple preset motion trajectories in the first interface 501, for example: preset motion trajectory a (used to open control A, corresponding to the preset trajectory key value KEY_5), preset motion trajectory b (used to open control B, corresponding to the preset trajectory key value KEY_2), preset motion trajectory c (used to open control C, corresponding to the preset trajectory key value KEY_6), preset motion trajectory d (used to open control D, corresponding to the preset trajectory key value KEY_1), preset motion trajectory e (used to open control E, corresponding to the preset trajectory key value KEY_7), preset motion trajectory f (used to open control F, corresponding to the preset trajectory key value KEY_8), preset motion trajectory g (used to open control G, corresponding to the preset trajectory key value KEY_9) and preset motion trajectory h (used to open control H, corresponding to the preset trajectory key value KEY_10).

[0208] It should be noted that the preset motion trajectories and the corresponding preset trajectory key values ​​shown in Figure 6 are only examples. The starting point of each preset motion trajectory can also be located at other positions in the first interface. The embodiment of the present disclosure does not limit the shape and direction of the preset motion trajectory and the corresponding preset trajectory key values.

[0209] In some implementations, after receiving the above-mentioned trajectory signal, the display device can parse the trajectory signal to obtain the corresponding target trajectory key value, and generate a corresponding second control instruction according to the target trajectory key value to open the corresponding target control according to the second control instruction.

[0210] In some embodiments, as shown in Figure 9, if the display device receives the trajectory signal and parses it, the obtained target trajectory key value is KEY_1 (corresponding to the horizontal left motion trajectory b), then the display device can display the motion trajectory of the remote control device in the first interface 501 (the motion trajectory b shown in Figure 9), and generate a corresponding second control instruction based on the target trajectory key value KEY_1 to open the target control D according to the second control instruction.

[0211] S1480: The display device controls the display to display the second interface corresponding to the target control in the first interface according to the second control instruction.

[0212] In some embodiments, after the display device generates the second control instruction, it can control the display to display the second interface corresponding to the above-mentioned target control according to the second control instruction, that is, open the second interface corresponding to the target control according to the second control instruction.

[0213] In the infrared communication mode of the disclosed embodiment, the remote control device can determine the motion trajectory of the remote control device and the corresponding trajectory signal based on the acquired motion information of the remote control device, and transmit the trajectory signal to the display device, so that the display device generates a corresponding second control instruction based on the trajectory signal to open the second interface corresponding to the target control in the first interface. As a result, the disclosed embodiment does not require the user to press the relevant button of the remote control device to move the focus position in the display device interface. Instead, the remote control device can be directly controlled to move according to a certain motion trajectory to open the corresponding control in the display device, thereby improving the convenience of the user controlling the display device through the remote control device.

[0214] An embodiment of the present disclosure provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a display device / remote control device / display device control apparatus based on motion trajectory, the display device / remote control device / display device control apparatus based on motion trajectory executes the display device control method based on motion trajectory in any of the above-mentioned method embodiments.

[0215] The executable instructions may be specifically used to enable the display device / remote control device / display device control apparatus based on motion trajectory to execute the above-mentioned display device control method based on motion trajectory.

[0216] In this embodiment, computer-readable storage media may be any tangible media that can contain or store a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0217] More specific examples (a non-exhaustive list) of computer-readable storage media may include: an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0218] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present disclosure are not directed to any particular programming language.

[0219] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure can be practiced without these specific details. Similarly, in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present disclosure, various features of the embodiments of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. The claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present disclosure.

[0220] Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively changed and installed in one or more devices different from the embodiments. The modules, units, or components in the embodiments can be combined into one module, unit, or component, and furthermore, they can be divided into multiple submodules, subunits, or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0221] It should be noted that the above embodiments illustrate rather than limit the present disclosure, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A display device, comprising: a display configured to display a user interface; A first communication device is configured to receive motion information of the remote control device sent by the remote control device at multiple times; wherein the motion information of the remote control device includes acceleration of the remote control device; At least one first processor, connected to the display and the first communication device, configured to execute computer instructions to cause the display device to perform: Controlling the display to display a first interface; wherein the user interface includes the first interface, and the first interface includes a plurality of controls; Determining a motion trajectory of the remote control device from a previous moment to the current moment based on the motion information of the remote control device at the current moment; wherein the multiple moments include the current moment and the previous moment; matching the motion trajectory of the remote control device from the previous moment to the current moment with a plurality of preset motion trajectories; If the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among the multiple preset motion trajectories, and the movement distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, a first control instruction corresponding to the first interface is generated according to the target motion trajectory; wherein the first control instruction is used to open a second interface corresponding to a target control in the first interface, the multiple controls include the target control, and the user interface includes the second interface.

2. The display device according to claim 1, wherein the first communication device is further configured to: receive a trajectory detection signal sent by a user through a control button of the remote control device; The at least one first processor is further configured to execute computer instructions to cause the display device to perform: In response to the trajectory detection signal, based on the motion information of the remote control device at the current moment, a motion trajectory of the remote control device from the previous moment to the current moment is determined.

3. The display device according to claim 1 or 2, wherein the at least one first processor is specifically configured to execute computer instructions to cause the display device to determine the motion trajectory of the remote control device from a previous moment to the current moment based on the motion information of the remote control device at the current moment in the following manner: Based on the acceleration of the remote control device at the current moment, determining the movement distance of the remote control device from the initial moment to the current moment; wherein, The initial moment is the moment when the motion information sent by the remote control device is received for the first time after the trajectory detection signal is received; determining a movement direction of the remote control device at the current moment based on a movement distance of the remote control device from the initial moment to the current moment; Based on the moving distance and moving direction of the remote control device, a moving trajectory of the remote control device from the previous moment to the current moment is determined.

4. The display device according to claim 1, wherein the first communication device is further configured to: receive a termination detection signal sent by a user through a control button of the remote control device; The at least one processor is further configured to execute computer instructions to cause the display device to perform: Before generating a first control instruction corresponding to the first interface according to the target motion trajectory, it is determined that the communication device receives the termination detection signal, or the movement distance of the remote control device from the initial moment to the current moment is greater than a first preset distance.

5. The display device according to claim 1 or 2, wherein the first processor is further configured to execute computer instructions to cause the display device to execute: If the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among the multiple preset motion trajectories, the display is controlled to display the motion trajectory of the remote control device from the initial moment to the current moment in the first interface.

6. A remote control device comprising: a second communication device configured to send a trajectory signal to the display device; At least one second processor, connected to the communication device, configured to execute computer instructions to cause the remote control device to: Acquiring motion information of the remote control device; wherein the motion information of the remote control device includes acceleration of the remote control device; determining a motion trajectory of the remote control device based on the motion information of the remote control device; Determining the trajectory signal corresponding to the motion trajectory of the remote control device; Control the second communication device to send the trajectory signal to the display device so that the display device receives the trajectory signal and generates a second control instruction corresponding to the first interface according to the trajectory signal; wherein the second control instruction is used to open the second interface corresponding to the target control in the first interface, and the display device displays a user interface, the user interface includes the first interface and the second interface, the first interface includes multiple controls, and the multiple controls include the target control.

7. The remote control device according to claim 6, wherein the at least one second processor is specifically configured to execute computer instructions to cause the remote control device to determine the motion trajectory of the remote control device based on the motion information of the remote control device in the following manner: In response to a user operating a control button of the remote control device, determining a motion trajectory of the remote control device based on motion information of the remote control device; or When the moving distance of the remote control device is greater than a second preset distance, a motion trajectory of the remote control device is determined based on the motion information of the remote control device.

8. The remote control device according to claim 6 or 7, wherein the at least one second processor is specifically configured to execute computer instructions to cause the remote control device to perform the step of determining the trajectory signal corresponding to the motion trajectory of the remote control device in the following manner: Determining a target trajectory key value that matches the motion trajectory of the remote control device from a plurality of preset trajectory key values; The trajectory signal corresponding to the target trajectory key value is determined.

9. A method for controlling a display device based on motion trajectory, applied to a display device, comprising: Display a first interface; wherein the first interface includes multiple controls; Receiving motion information of the remote control device sent by the remote control device at multiple times; wherein the motion information of the remote control device includes acceleration of the remote control device; Determining a motion trajectory of the remote control device from a previous moment to the current moment based on the motion information of the remote control device at the current moment; wherein the multiple moments include the current moment and the previous moment; matching the motion trajectory of the remote control device from the previous moment to the current moment with a plurality of preset motion trajectories; If the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among the multiple preset motion trajectories, and the movement distance of the remote control device from the initial moment to the current moment is greater than or equal to the length of the target motion trajectory, then a first control instruction corresponding to the first interface is generated according to the target motion trajectory; wherein the first control instruction is used to open a second interface corresponding to a target control in the first interface, and the multiple controls include the target control.

10. The method according to claim 9, further comprising: receiving a trajectory detection signal sent by a user through a control button of the remote control device; The determining, based on the motion information of the remote control device at the current moment, a motion trajectory of the remote control device from the previous moment to the current moment, includes: In response to the trajectory detection signal, based on the motion information of the remote control device at the current moment, a motion trajectory of the remote control device from the previous moment to the current moment is determined.

11. The method according to claim 9 or 10, wherein determining the motion trajectory of the remote control device from a previous moment to the current moment based on the motion information of the remote control device at the current moment comprises: determining a velocity of the remote control device at the current moment based on the acceleration of the remote control device at the current moment; Determining a movement distance of the remote control device from the initial moment to the current moment based on the speed of the remote control device at the current moment; wherein the initial moment is the moment when the motion information sent by the remote control device is first received after receiving the trajectory detection signal; determining a movement direction of the remote control device at the current moment based on a movement distance of the remote control device from the initial moment to the current moment; Based on the moving distance and moving direction of the remote control device, a moving trajectory of the remote control device from the previous moment to the current moment is determined.

12. The method according to claim 9, further comprising: receiving a termination detection signal sent by a user through a control button of the remote control device; The method further includes: before generating a first control instruction corresponding to the first interface according to the target motion trajectory, determining that the termination detection signal is received, or that the movement distance of the remote control device from the initial moment to the current moment is greater than a first preset distance.

13. The method according to claim 9 or 10, further comprising: If the motion trajectory of the remote control device from the initial moment to the current moment matches the target motion trajectory among the plurality of preset motion trajectories, the motion trajectory of the remote control device from the initial moment to the current moment is displayed in the first interface.

14. A method for controlling a display device based on motion trajectory, applied to a remote control device, comprising: Acquiring motion information of the remote control device; wherein the motion information of the remote control device includes acceleration of the remote control device; determining a motion trajectory of the remote control device based on the motion information of the remote control device; Determining the trajectory signal corresponding to the motion trajectory of the remote control device; The trajectory signal is sent to the display device so that the display device receives the trajectory signal and generates a second control instruction corresponding to the first interface based on the trajectory signal; wherein the second control instruction is used to open the second interface corresponding to the target control in the first interface, and the display device is used to display the first interface and the second interface, and the first interface includes multiple controls, and the multiple controls include the target control.

15. The method according to claim 14, wherein determining the motion trajectory of the remote control device based on the motion information of the remote control device comprises: In response to a user operating a control button of the remote control device, determining a motion trajectory of the remote control device based on motion information of the remote control device; or, When the moving distance of the remote control device is greater than a second preset distance, a motion trajectory of the remote control device is determined based on the motion information of the remote control device.

16. The method according to claim 14 or 15, wherein determining the trajectory signal corresponding to the motion trajectory of the remote control device comprises: Determining a target trajectory key value that matches the motion trajectory of the remote control device from a plurality of preset trajectory key values; The trajectory signal corresponding to the target trajectory key value is determined.

Images