Display device and remote controller

Abstract

The invention discloses a display device and a remote controller, and the method comprises the steps: receiving data representing the display position of a cursor of the remote controller, and drawing and displaying the cursor of the remote controller on an operation interface according to the data representing the display position of the cursor of the remote controller; when it is detected that the display position of the remote controller cursor falls into a display area of a target control and does not move out of the display area of the target control within preset time, determining whether preset audio matched with a mapping relation exists or not according to information used for representing media asset type attributes matched with the target control, and if yes, generating an instruction for playing the audio so as to indicate a player to play a preset audio which has a mapping relationship with the information representing the media asset type attribute.

Description

Technical Field

[0001] This application relates to the field of display technology, and particularly to a display device and a remote control. Background Art

[0002] With the rapid development of display devices, the functions that display devices can provide for users are becoming increasingly rich. Remote controls are also gradually becoming intelligent. Currently, when a user points a remote control at a certain control, only a focusing effect will appear on the display, and this interaction method is relatively single. Summary of the Invention

[0003] This application provides a display device and an interaction method to solve the problem of the single interaction method of existing remote controls.

[0004] In a first aspect, some embodiments provide a display device, including:

[0005] A display configured to: display an operation interface, where there is a cursor in the operation interface, and the display position of the cursor on the operation interface is determined by the position pointed by the remote control in a three-dimensional space;

[0006] A processor configured to: receive data representing the display position of the remote control cursor, and draw and display the remote control cursor on the operation interface according to the data representing the display position of the remote control cursor; when it is detected that the display position of the remote control cursor falls within the display area of a target control and does not move out of the display area of the target control within a preset time, determine whether there is a preset audio with a matching mapping relationship according to the information representing the media type attribute matched by the target control, and if so, generate an instruction to play the audio to instruct the player to play the preset audio having a mapping relationship with the information representing the media type attribute.

[0007] This mechanism for enhanced feedback using the information representing the media type attribute enables users to have a more intuitive perception of the control content, significantly improves the user's interaction experience, operation efficiency, and accuracy, and is particularly friendly to user groups with hearing and visual impairments.

[0008] In some embodiments, the processor is further configured to: determine whether there is a preset display state with a matching mapping relationship according to the information representing the media type attribute matched by the target control, and if so, generate a display state switching instruction to control the display area of the target control to switch to the preset display state having a mapping relationship with the information representing the media type attribute. This enhances the user's perception in the visual dimension.

[0009] In some embodiments, the processor is further configured to: send the information for characterizing the media type attribute matching the target control to the remote controller, so as to instruct the remote controller to control the sound - emitting module to emit a sound matching the information and / or control the vibration module to generate a vibration matching the information according to the information for characterizing the media type attribute. Through the visual, auditory, and tactile feedback mechanisms on the remote - controller side, users can more intuitively perceive the attributes of the control, significantly enhancing the user's interaction experience.

[0010] In some embodiments, before the processor determines whether there is a preset audio with a matching mapping relationship according to the information for characterizing the media type attribute matching the target control, the processor is further configured to: determine whether the interaction switch on the display device side is in an on state; if it is in an on state, then determine whether there is a preset audio with a matching mapping relationship according to the information for characterizing the media type attribute matching the target control. Before the processor sends the information for characterizing the media type attribute matching the target control to the remote controller, the processor is further configured to: determine whether the interaction switch on the remote - controller side is in an on state; if it is in an on state, then send the information for characterizing the media type attribute matching the target control to the remote controller. This realizes the personalized control of the interaction switches on the display - device side and the remote - controller side.

[0011] In some embodiments, the processor is further configured to: in response to the selection of the first control among multiple controls by the remote controller, display a jump interface associated with the first control on the display and send a jump - success instruction to the remote controller, so as to instruct the remote controller to control the visual module to switch to the corresponding visual state, control the sound - emitting module to emit the corresponding sound, and / or control the vibration module to generate the corresponding vibration based on the jump - success instruction; in response to a page - sliding operation triggered by the remote controller, display the content of the page after sliding on the display and send a slide - success instruction to the remote controller, so as to instruct the remote controller to control the visual module to switch to the corresponding visual state, control the sound - emitting module to emit the corresponding sound, and / or control the vibration module to generate the corresponding vibration based on the slide - success instruction; in response to a control - dragging operation on the second control triggered by the remote controller, display the process of dragging the second control on the display and send a drag - success instruction to the remote controller, so as to instruct the remote controller to control the visual module to switch to the corresponding visual state, control the sound - emitting module to emit the corresponding sound, and / or control the vibration module to generate the corresponding vibration based on the drag - success instruction. The user's selection, sliding, dragging and other behaviors will trigger corresponding audio - visual - tactile effects on the remote - controller side, enhancing the user's intuitive perception of these behaviors.

[0012] Second aspect, some embodiments further provide a remote control, including: an interaction module, the interaction module including: a sound - emitting module and a vibration module; and at least one processor, connected to the interaction module and configured to execute instructions to cause the remote control to: receive information sent by a display device for characterizing that a target control matches a media type attribute; and according to the information for characterizing that the target control matches the media type attribute, control the sound - emitting module to emit a sound matching the information, and / or control the vibration module to generate a vibration matching the information.

[0013] Adding visual, auditory, and tactile feedback mechanisms on the remote - control side enables users to more intuitively perceive the attributes of controls, significantly enhancing the user's interaction experience, operation efficiency, and accuracy, and is especially user - friendly to user groups with auditory and visual impairments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Shows an operation scenario between a display device and a control device according to some embodiments;

[0015] Figure 2 Shows a hardware configuration block diagram of a control device 100 according to some embodiments;

[0016] Figure 3 Shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0017] Figure 4 Shows a software configuration diagram in a display device 200 according to some embodiments;

[0018] Figure 5 Exemplarily shows a flowchart of a display - device execution scheme according to some embodiments;

[0019] Figure 6 Exemplarily shows a schematic diagram of a target page according to some embodiments;

[0020] Figure 7 Exemplarily shows a schematic diagram of upper - focus processing according to some embodiments;

[0021] Figure 8 Exemplarily shows a schematic diagram of a click behavior according to some embodiments;

[0022] Figure 9 Exemplarily shows a schematic diagram of a swipe behavior according to some embodiments;

[0023] Figure 10 Exemplarily shows a schematic diagram of a drag - and - drop behavior according to some embodiments;

[0024] Figure 11 Exemplarily shows a schematic of an interaction method according to some embodimentsFigure 1 ;

[0025] Figure 12 Exemplarily shown is a schematic diagram of an interaction method according to some embodiments Figure 2 . Detailed implementation manners

[0026] Embodiments will be described in detail below, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are merely examples of systems and methods consistent with some aspects of the present application detailed in the claims.

[0027] It should be noted that the brief description of terms in the present application is only for facilitating the understanding of the following described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0028] The terms "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0029] The terms "comprising" and "having" and any variations thereof are intended to cover but not be exclusive of inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0030] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to the element.

[0031] In the embodiments of the present application, the display device 200 generally refers to a device having a screen display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0032] Figure 1 It is a schematic diagram of an operation scenario between a display device and a control device provided for some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through touch operations, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote control, a stylus, a handle, etc.

[0033] The mobile terminal 300 can be used as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 to perform data interaction. In some embodiments, software applications can be installed on the mobile terminal 300 and the display device 200, and connection communication can be achieved through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve the synchronous display function.

[0034] As Figure 1 is also shown, the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 is allowed to establish a communication connection through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0035] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0036] Figure 2 For some embodiments of this application Figure 1 The hardware configuration block diagram of the display device 200 in

[0037] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0038] In some embodiments, the detector 230 is used to collect signals from the external environment or for external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0039] In some embodiments, the display 260 includes a display function component for presenting a picture and a driving component for driving image display. The display 260 is used to receive the image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu control interface, and a user control UI interface, etc.

[0040] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0041] The communication device 220 can enable the display device 200 to communicate with an external device or server 400 through a wireless or wired connection. Among them, the wired connection can connect the display device 200 with an external device through components such as a data cable and an interface. The wireless connection can connect the display device 200 with an external device through a wireless signal or a wireless network. The display device 200 can directly establish a connection relationship with an external device, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0042] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0043] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0044] In some embodiments, the user can input a user command on the graphical operation interface (Graphical User Interface, GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical operation interface (GUI).

[0045] In some embodiments, the audio output device 270 may be a built-in speaker of the display device 200, or an external audio output device connected to the display device 200. Among them, for the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0046] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0047] Figure 3 The Figure 1 hardware configuration block diagram of the control device provided in some embodiments of the present application. As Figure 3 shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0048] The control device 100 is configured to control the display device 200, and can receive input operation instructions from the user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, playing the role of an interaction intermediary between the user and the display device 200.

[0049] In some embodiments, the control device 100 may be an intelligent device. For example: the control device 100 can install various applications for controlling the display device 200 according to user needs.

[0050] In some embodiments, as Figure 1 shown, after installing an application for controlling the display device 200, the mobile terminal 300 or other intelligent electronic devices can perform functions similar to those of the control device 100.

[0051] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions between the external and internal.

[0052] Under the control of the controller 110, the communication interface 130 realizes the communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, and other near-field communication modules.

[0053] The user input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a button 144, and other input interfaces.

[0054] In some embodiments, the control device 100 includes at least one of the communication interface 130 and the input / output interface 140. The communication interface 130 is configured in the control device 100. For example: modules such as WiFi, Bluetooth, and NFC can encode user input instructions through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol, and send them to the display device 200.

[0055] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0056] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0057] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may (control the display device) provide an operation interface, allow the user to interact with the display device 200, and support the running of various application programs.

[0058] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0059] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0060] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0061] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0062] like Figure 4As shown, in the embodiments of the present application, the application framework layer includes a view system, managers, content providers, etc. Among them, the view system can design and implement the interfaces and interactions of application programs. The view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application program packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the Window Manager is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the operation interface.

[0063] In some embodiments, the Activity Manager is used to manage the life cycles of various application programs and the usual navigation back functions, such as controlling the exit, opening, and backward movement of application programs. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling the changes of the display window. For example, shrinking the display window, jittering the display, and distorting the display, etc.

[0064] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction libraries included in the system runtime layer, such as C / C++ instruction libraries, to implement the functions that the framework layer needs to achieve.

[0065] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking processing, and memory management. For example, as Figure 4 shown, hardware drivers can be configured in the kernel layer. The drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers, etc.

[0066] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. Depending on factors such as the functions of the display device and the type of operating system, the number of levels and the specific level types included in the operating system may be in other forms.

[0067] In some embodiments, the display device includes: a display for displaying an operation interface, where the operation interface has a cursor, and the display position of the cursor on the operation interface is determined by the position pointed by the remote control in the three-dimensional space; a processor configured to execute the solution as Figure 5 shown below:

[0068] Step 501: Receive data representing the display position of the remote control cursor, and draw and display the remote control cursor on the operation interface according to the data representing the display position of the remote control cursor.

[0069] Optionally, the remote control in the embodiments of the present application can be a pointing remote control or an ordinary remote control. The embodiments of the present application do not limit this. The following uses a pointing remote control as an example to illustrate the implementation process of the solution.

[0070] Optionally, when the user wants to start the display device, the user can trigger the switch on the pointing remote control. In response to this trigger operation, the pointing remote control sends a start instruction to the display device on the one hand, and wakes up the internal system of the pointing remote control on the other hand. After the internal system of the remote control is awakened, the pointing remote control can continuously perform Bluetooth broadcasting. After the display device receives the start instruction, it starts Bluetooth device scanning. After receiving the remote control information broadcast by the pointing remote control, it processes the remote control information and sends a Bluetooth connection request to the pointing remote control. After the pointing remote control receives the Bluetooth connection request, it processes the pairing information of the display device. After the processing is completed, it returns an agreed connection message to the display device. After the display device receives the agreed connection message, it completes the Bluetooth pairing connection with the pointing remote control. On this basis, a data channel can be established between the display device and the pointing remote control through the Logical Link Control and Adaptation Protocol (L2CAP) layer in the Bluetooth protocol stack for data packet transmission.

[0071] Among them, Bluetooth is a short-range wireless communication technology that supports two-way data transmission between devices. It is widely used in the connection between various electronic devices, such as mobile phones, earphones, remote controls, and smart TVs. To ensure the reliability of data packet transmission, the Logical Link Control and Adaptation Protocol (L2CAP) supports an acknowledgement (ACK) mechanism, and the receiving device sends an acknowledgement signal back to the sending device after successfully receiving the data packet.

[0072] Optionally, to improve the data packet transmission efficiency, the data packets can be compressed to reduce the amount of data transmitted. The Logical Link Control and Adaptation Protocol (L2CAP) also has built-in error detection and correction mechanisms to ensure the accuracy and integrity of data packet transmission. The display device and the pointing remote control also support the Bluetooth Low Energy mode, which can reduce the energy consumption of the device and extend the battery life without affecting the transmission performance.

[0073] After the display device and the pointing remote control complete the Bluetooth pairing connection, the user can use the pointing remote control to control the display device. For example, use the pointing remote control to trigger an open instruction for a target page, etc.

[0074] In response to the open instruction for the target page, the display device can obtain the page data of the target page. The page data includes the control content of at least one control pre-configured for the target page and the information representing the media type attribute corresponding to each control; based on the control content, at least one control pre-configured for the target page is rendered and displayed on the display, thereby realizing the display of the operation interface corresponding to the target page.

[0075] The display device can provide many pages, such as the home page, sub-pages corresponding to a certain label on the home page, pages provided by a certain application (abbreviated as app), etc. All or some of the pages can be set to support the interaction provided by the embodiments of the present application. In this case, these pages can all be used as target pages.

[0076] Optionally, the home page usually contains multiple labels, such as: movies, TV dramas, variety shows, education, etc. The sub-page corresponding to a certain label is the page that jumps to after the label is triggered.

[0077] Optionally, the display device can install applications (abbreviated as apps). The functions of the apps can be, for example, voice interaction, video, games, etc. The pages provided by the apps also include the home page, sub-pages, etc. The embodiments of the present application do not limit this.

[0078] In the case where all the pages provided by the display device are set to support the interaction provided by the embodiments of the present application, after the user triggers an instruction to open any page, the display device uses this page as the target page, uses this instruction as the open instruction for the target page, and in response to this instruction, obtains the page data of the target page from the cloud server.

[0079] When a partial page provided by a display device is set to support the interaction provided by the embodiments of the present application, after a user triggers an instruction to open a certain page, the display device determines whether the page supports the interaction provided by the embodiments of the present application. If it does, the page is used as the target page, and the instruction is used as an opening instruction for the target page. In response to the instruction, page data of the target page is obtained from a cloud server; if it does not support, it is determined that the page is not the target page, and the instruction is not an opening instruction for the target page. In response to the instruction, the display device will also obtain the page data of the page from the cloud server, but the content included in the obtained page data is different from the content included in the page data of the above target page. The page data of the target page includes information representing the media asset type attribute corresponding to each control, while this page data does not include this information.

[0080] Considering that the page data of a page is updated once every certain period of time, when a user frequently opens a certain page within a short period of time, the page data obtained by the display device from the cloud server each time is actually the same. However, the display device frequently obtaining page data from the cloud server brings network pressure and untimely response. To solve this problem, in response to an opening instruction for the target page, the display device can determine the time interval since the last time it obtained the page data of the target page from the cloud server. If the time interval is long enough, then obtain the page data of the target page from the cloud server again; otherwise, directly use the page data of the target page obtained last time from the cloud server.

[0081] Among them, for each page that supports the interaction provided by the embodiments of the present application, an operator can configure at least one control for the page. For each control, the operator can configure the control content for the control, and can also configure information representing the media asset type attribute for the control according to the control content, greatly improving the flexibility of control display, reducing the hardware cost and maintenance difficulty, and improving the intelligence level and reliability of the display device.

[0082] Exemplarily, for a certain control, if the control content configured for the control is resource cover data, the operator can use the emotional type to which the resource corresponding to the resource cover data belongs as the media asset type attribute matched by the control. For example, if the emotional type to which the resource corresponding to the resource cover data belongs is happy, then use the happy attribute as the media asset type attribute matched by the control; if the emotional type to which the resource corresponding to the resource cover data belongs is sad, then use the sad attribute as the media asset type attribute matched by the control; if the emotional type to which the resource corresponding to the resource cover data belongs is horror, then use the horror attribute as the media asset type attribute matched by the control.

[0083] Exemplarily, for a certain control, the operator can use the recommendation degree of the control content configured for the control as the media type attribute matched by the control. For example, if the control content configured for the control is that the user is strongly expected to trigger the control, the operator can use the strongly recommended attribute as the media type attribute matched by the control.

[0084] Exemplarily, for a certain control, if the control content configured for the control does not support clicking, the operator can use the prohibited click as the media type attribute matched by the control; if the control content configured for the control supports clicking, the operator can use the supported click as the media type attribute matched by the control.

[0085] Among them, different information can be used to represent different media type attributes, and the information here can be any one or a combination of numbers, letters, and characters. Exemplarily, the number 1 can be used to represent the happy attribute, the number 2 can be used to represent the sad attribute, the number 3 can be used to represent the horror attribute, the number 4 can be used to represent the strongly recommended attribute, the number 5 can be used to represent the prohibited click; the number 6 can be used to represent the supported click. Under this representation rule, for each control, when the operator determines that the media type attribute matched by the control is the happy attribute, the information representing the media type attribute corresponding to the control can be configured as 1. When it is determined that the media type attribute matched by the control is the sad attribute, the information representing the media type attribute corresponding to the control can be configured as 2. When it is determined that the media type attribute matched by the control is the horror attribute, the information representing the media type attribute corresponding to the control can be configured as 3. When it is determined that the media type attribute matched by the control is the strongly recommended attribute, the information representing the media type attribute corresponding to the control can be configured as 4. When it is determined that the media type attribute matched by the control is the prohibited click, the information representing the media type attribute corresponding to the control can be configured as 5. When it is determined that the media type attribute matched by the control is the supported click, the information representing the media type attribute corresponding to the control can be configured as 6. However, this is only an example, and there can be other representation rules. The embodiments of the present application do not limit this.

[0086] It should be noted that: the attributes in the above examples are only examples, and there can be other attributes. The above examples do not constitute a limitation to the embodiments of the present application.

[0087] Optionally, the operator can manually configure the information representing the media type attribute through code or XML (such as the text of TextView, the background, alpha, translationX, etc. of View).

[0088] Exemplarily, an operator can modify the background color of a control through the view.setBackgroundColor(Color.RED) interface, modify the text content of the control through the view.setText("Hello") interface, and modify the position offset of the control through the view.setTranslationX(100f) interface. These interfaces will ultimately call the corresponding methods of the View class. For example, setBackgroundColor() will call setBackground(), thereby triggering the underlying logic.

[0089] The underlying process of control property update includes: Step 1: Property modification and marking of the dirty region. Trigger invalidate() or requestLayout(); invalidate(): Mark that the visual content of the current View needs to be redrawn (such as modifying the background color or text). requestLayout(): Mark that the size or position of the current View may change and needs to be re-measured (Measure) and laid out (Layout) (such as modifying width or height). Example: Modify text: Call invalidate() (the text content changes and needs to be redrawn) and requestLayout() (the text length may affect the control size). Modify translationX: Only call invalidate() (the position changes but the layout remains unchanged). Step 2: Coordination between ViewRootImpl and Choreographer. ViewRootImpl is the core class connecting the application to the system rendering pipeline, responsible for managing the measurement, layout, and drawing of the View tree. When invalidate() or requestLayout() is called: 1. ViewRootImpl registers a vertical sync signal (VSYNC) callback through Choreographer. 2. When the VSYNC signal of the next frame arrives, trigger ViewRootImpl to execute performTraversals(), that is, the complete view tree update process (Measure → Layout → Draw). A control is usually the basic component of a view. A view can be composed of multiple controls or can also contain other views.

[0090] Optionally, fields for triggering different interface special effects can be added to the control properties. The happy property, sad property, horror property, highly recommended property, prohibited click, and supported click mentioned above each correspond to a field value. When configuring the control properties, the operator can determine which of the happy property, sad property, horror property, highly recommended property, prohibited click, and supported click the control content of the corresponding control corresponds to, and configure the field for triggering different interface special effects as the field value corresponding to the determination result. In this embodiment, the field value is the information representing the media asset type property.

[0091] Optionally, during the process of the operation and maintenance personnel configuring controls for a page, a unique identifier can also be set for each control, and the interface display position can also be set for each control. The interface display position, control content, and information representing the media asset type property of each control constitute a set of control data, and this control data is identified by the above-mentioned unique identifier. The sets of control data with unique identifiers constitute the page data of the corresponding page. Or, for each page that supports the interaction provided in the embodiments of the present application, which controls it contains, the unique identifier of each control, and the interface display position of each control are all pre-fixed and do not require configuration by the operation and maintenance personnel. The operation and maintenance personnel only need to configure the control content and the information representing the media asset type property. The above implementation manners are all within the protection scope of the present application.

[0092] Or, for each control, its interface display position can be associated and bound with the view-related content of the control. The unique identifier of the control (such as CardCode), title, control content (or media asset content), information representing the media asset type property, etc. can be associated and bound with the control as an associated data group of the control.

[0093] Among them, the page data of the target page obtained by the display device from the cloud server includes the control content of at least one control pre-configured for the target page and the information representing the media asset type property corresponding to each control. The display device can render and display at least one control pre-configured for the target page on the display based on the control content of each control, so as to implement the display of the operation interface corresponding to the target page for the user to view.

[0094] Optionally, for each control in the page data, the display device can first determine the interface display position corresponding to the control according to the unique identifier of the control, and then perform rendering at this interface display position based on the control content of the control, so as to display the control. Exemplarily, the target page is the home page, and the rendering result is as Figure 6 shown.

[0095] As described above, the interface display position of each control can be configured by the operation staff or fixed in advance. In the case where the operation and maintenance staff configures the interface display position, the display device can find the corresponding control data from the page data according to the unique identifier of the control, and extract the corresponding interface display position from the found control data. When the interface display position is fixed in advance, the display device can find the corresponding interface display position from the database according to the unique identifier of the control.

[0096] Among them, there is a cursor on the operation interface, and the display position of the cursor on the operation interface is determined by the position pointed by the remote control in the three-dimensional space. In the embodiments of the present application, the position pointed by the remote control in the three-dimensional space is also called the position data of the remote control. For the specific implementation process of determining the display position of the cursor on the operation interface based on the position data of the remote control, refer to the following text.

[0097] In some embodiments, the position data of the remote control may include position coordinates and a pointing direction. Among them, the position coordinates are the physical position of the remote control relative to the display device (for example, the horizontal distance and vertical height from the display device). The pointing direction is the pointing angle of the remote control (for example, the horizontal angle and vertical angle relative to the display device).

[0098] In some embodiments, the pointing remote control is a remote control device that can detect its own position and direction in the three-dimensional space. The pointing remote control is built-in with a gyroscope for detecting the rotation angle of the remote control and an accelerometer for detecting the moving acceleration of the remote control. Through these sensors, the pointing remote control can determine its own position data in the three-dimensional space in real time.

[0099] In some embodiments, after the pointing remote control obtains its own position data in the three-dimensional space, it can send the position data to the display device. The display device can perform screen coordinate conversion through the position data, so as to convert the position data of the pointing remote control into the screen coordinate system and complete the mapping of the position data of the pointing remote control. At this time, the display device draws the remote control cursor on the operation interface according to the calculated screen coordinates (this screen coordinate is the display position of the cursor on the operation interface), so as to display the remote control cursor. When the user moves the remote control, the sensors of the remote control will continuously detect the changes in the position and direction and send the data to the display device. The display device updates the display position of the remote control cursor in real time according to the received data, so as to realize the movement of the display position of the remote control cursor with the change of the position data of the pointing remote control.

[0100] In some embodiments, a coordinate conversion module is included in the remote controller. After obtaining its own position data, the remote controller can input the position data into the coordinate conversion module. The coordinate conversion module can perform screen coordinate conversion through the position data, thereby converting the position data into the screen coordinate system to obtain the screen coordinate (this screen coordinate is the display position of the cursor on the operation interface), and use this screen coordinate as the data representing the display position of the remote controller cursor and send it to the display device. After receiving the data representing the display position of the remote controller cursor, the display device draws the remote controller cursor at the position corresponding to the data on the operation interface, thereby displaying the remote controller cursor.

[0101] Exemplarily, the data representing the display position of the remote controller cursor is the coordinate (a, b). After receiving this data, the display device draws the remote controller cursor at the position with the coordinate (a, b) on the operation interface, thereby displaying the remote controller cursor.

[0102] The interaction process between a traditional wireless mouse and a computer is that when the sensor in the wireless mouse detects movement, it calculates the displacement increments in the horizontal and vertical directions and sends the displacement increments to the computer. The computer displays the mouse cursor based on the displacement increments. In the scenario of this application, the remote controller obtains its own position data, which includes the position coordinates and the pointing direction. Among them, the position coordinates are the physical position of the remote controller relative to the display device (for example, the horizontal distance and vertical height from the display device). The pointing direction is the pointing angle of the remote controller (for example, the horizontal angle and vertical angle relative to the display device). As described above, the remote controller converts the position data into the screen coordinate system to obtain the screen coordinate, and uses this screen coordinate as the data representing the display position of the remote controller cursor and sends it to the display device. The display device draws and displays the remote controller cursor based on the data representing the display position of the remote controller cursor, which is different from the interaction process between a traditional wireless mouse and a computer.

[0103] The working process of a traditional remote controller is: button pressed → encoding chip generates encoding → modulated to 38kHz carrier wave → drives the infrared tube to emit. The traditional remote controller sends key indication information to the display device, while in this application, the remote controller sends the data representing the display position of the remote controller cursor to the display device. After receiving the key indication information, the traditional remote controller updates the focus state of the control corresponding to the key indication information. The traditional remote controller performs corresponding focus effect processing based on the focus state of the control. In the scenario of this application, the display device triggers various effects as described below based on the floating state of the control, and there are differences in the principles between the two.

[0104] Step 502: When it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within the preset time, determine whether there is a preset audio with a matching mapping relationship according to the information characterizing the media type attribute of the target control. If so, generate an instruction to play the audio to instruct the player to play the preset audio that has a mapping relationship with the information characterizing the media type attribute.

[0105] Optionally, when the display position of the remote control cursor falls within the display area of a certain control, the control can be used as the target control. Exemplarily, the coordinate ranges corresponding to the display position of the remote control cursor and the display area of the control can be compared to determine whether the display position of the remote control cursor falls within the display area of the control.

[0106] Among them, for all controls displayed on the operation interface, hovering (Hover) listening can be configured for all controls or some of them. When configuring hovering (Hover) listening, multiple listening events can be configured, such as enter event, move event, exit event, etc. In this way, for the controls configured with hovering (Hover) listening, when the display position of the remote control cursor enters the display area of the control, the system will trigger the enter event for the control, that is, case MotionEvent.ACTION_HOVER_ENTER; when the remote control cursor moves within the display area of the control, the system will trigger the move event for the control, that is, case MotionEvent.ACTION_HOVER_MOVE; when the remote control cursor moves out of the display area of the control, the system will trigger the exit event for the control, that is, case MotionEvent.ACTION_OUTSIDE. The above events can be captured through the hovering listening (OnHoverListener) interface. When the enter event for the target control is captured, it can be considered that the display position of the remote control cursor falls within the display area of the target control.

[0107] Among them, for the controls configured with hovering (Hover) listening, after the enter event for the control is captured, a timer can be started. If an exit event for the control is received during the timer timing, it is considered that the display position of the remote control cursor has moved out of the display area of the control within the preset time. If no exit event for the control is received during the timer timing, it is considered that the display position of the remote control cursor has not moved out of the display area of the control within the preset time. When the exit event for the control is captured, the timer can be reset.

[0108] Among them, the duration of the timer can be configured according to the above preset time. For example, if the preset time is 2s, the duration of the timer is set to 2s.

[0109] Among them, when the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, information for characterizing the media type attribute matching the target control can be obtained from the page data introduced above, that is, the information for characterizing the media type attribute corresponding to the target control. According to this information, it is determined whether there is a preset audio with a matching mapping relationship. If so, an instruction to play the audio is generated to instruct the player to play the preset audio having a mapping relationship with the above information. Since the movement of the remote control cursor on the operation interface is a continuous process, when moving from one control to the next control that the user is interested in, it may pass through multiple other controls that are not the controls the user is really interested in. The above limitation of the preset time can avoid generating multimodal feedback on these controls, saving computer resources and improving the user's interaction experience at the same time.

[0110] Among them, the display device stores audio identifiers corresponding to different information for characterizing different media type attributes. After the display device obtains the information for characterizing the media type attribute matching the target control from the page data, it can search for the audio identifier corresponding to this information. If it can be found, it is determined that there is a preset audio with a matching mapping relationship. If it cannot be found, it is determined that there is no preset audio with a matching mapping relationship.

[0111] Among them, when it is determined that there is a preset audio with a matching mapping relationship, the display device can generate an audio play instruction according to the audio identifier corresponding to the information for characterizing the media type attribute matching the target control, and send the audio play instruction to the player. After receiving the audio play instruction, the player parses the audio play instruction to obtain the audio identifier corresponding to the information for characterizing the media type attribute matching the target control, and uses the audio corresponding to this audio identifier as the preset audio having a mapping relationship with the above information for characterizing the media type attribute matching the target control. The player further plays this audio.

[0112] Among them, when it is determined that there is no preset audio with a matching mapping relationship, the display device will no longer perform the operation of instructing the player to play the audio.

[0113] Optionally, while the display device displays the remote control cursor on the operation interface, it can also perform a focusing-up process on the display area of the target control to prominently remind the user that the display position of the remote control cursor falls within the display area of the target control, as Figure 7 shown.

[0114] In the above embodiments, the display device includes a display and a processor. The display is configured to display an operation interface, and the operation interface includes a plurality of controls. The processor is configured to: receive data representing the display position of the remote control cursor, draw and display the remote control cursor on the operation interface according to the data representing the display position of the remote control cursor; when it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, determine whether there is a preset audio with a matching mapping relationship according to the information representing the media type attribute of the target control. If so, generate an instruction to play the audio to instruct the player to play the preset audio having a mapping relationship with the information representing the media type attribute. This mechanism of using the information representing the media type attribute for enhanced feedback enables users to have a more intuitive perception of the control content, significantly improving the user's interaction experience, operation efficiency, and accuracy, and is particularly friendly to user groups with hearing and visual impairments.

[0115] Optionally, for each page that supports the interaction provided in the embodiments of the present application, an operator can configure at least one control for the page. For each control, the operator can configure the control content for the control. Further, the operator can also determine whether to configure the information representing the media type attribute for the corresponding control according to the control content. Exemplarily, when the control content is the content to be promoted, the information representing the media type attribute is configured for the corresponding control; otherwise, the information representing the media type attribute may not be configured.

[0116] Optionally, after the user triggers an instruction to open a certain page, the display device determines whether the page supports the interaction provided by the embodiments of the present application. If it does, the page is used as the target page, and the instruction is used as the opening instruction for the target page. In response to this instruction, page data of the target page is obtained from the cloud server. The page data includes the control content of at least one control pre-configured for the target page and information characterizing the media asset type attribute configured by the operator. The display device can render and display at least one control pre-configured for the target page on the display based on the control content of each control, so as to implement the display of the operation interface corresponding to the target page for the user to view. When it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, the display device can determine whether the page data of the target page contains information for characterizing the matching media asset type attribute of the target control, that is, the information characterizing the media asset type attribute corresponding to the target control. If it does not contain such information, it means that the operator has not configured information characterizing the media asset type attribute for the target control, and the display device will no longer perform the operation of controlling the player to play audio; if it contains such information, it means that the operator has configured information characterizing the media asset type attribute for the target control. In this case, the display device can further determine whether there is a preset audio with a matching mapping relationship according to the information characterizing the media asset type attribute corresponding to the target control. If there is, an instruction to play the audio is generated to instruct the player to play the preset audio that has a mapping relationship with the information characterizing the media asset type attribute corresponding to the target control. The detailed process is as described above and will not be elaborated here.

[0117] In some embodiments, the processor is further configured to: determine whether there is a preset display state with a matching mapping relationship according to the information for characterizing the matching media asset type attribute of the target control. If there is, a display state switching instruction is generated to control the display area of the target control to switch to the preset display state that has a mapping relationship with the information characterizing the media asset type attribute.

[0118] Among them, in order to enhance the user's perception, the embodiments of the present application propose that when the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, not only audio playback control is performed, but also display state switching control can be performed according to the information for characterizing the matching media asset type attribute of the target control.

[0119] Among them, when it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, the display device can obtain the information for characterizing the matching media asset type attribute of the target control from the page data, and determine whether there is a preset display state with a matching mapping relationship according to this information.

[0120] Optionally, the display device stores display status identifiers corresponding to different information representing the attributes of different media asset types. After the display device obtains the information used to represent the media asset type attributes matched by the target control from the page data, it can search for the display status identifier corresponding to this information. If it can be found, it determines the preset display status with a matching mapping relationship; if not, it determines that there is no preset display status with a matching mapping relationship.

[0121] Among them, when determining the preset display status with a matching mapping relationship, the display device can generate a display status switching instruction according to the display status identifier corresponding to the information used to represent the media asset type attributes matched by the target control, and send the display status switching instruction to the corresponding component. After receiving the display status switching instruction, the component parses the display status switching instruction to obtain the display status identifier corresponding to the information used to represent the media asset type attributes matched by the target control, takes the display status corresponding to this display status identifier as the preset display status having a mapping relationship with the above-mentioned information used to represent the media asset type attributes matched by the target control, and switches the display area of the target control to this preset display status.

[0122] Among them, when determining that there is no preset display status with a matching mapping relationship, the display device will no longer perform the relevant operations of switching the display status.

[0123] As described above, the operator can determine whether information representing the media asset type attribute is configured for the corresponding control based on the control content. After the user triggers an open instruction for the target page, the display device obtains the page data of the target page from the cloud server. The page data includes the control content of at least one control pre-configured for the target page and the information representing the media asset type attribute configured by the operator. The display device can render and display at least one control pre-configured for the target page on the display based on the control content of each control, so as to implement the display of the operation interface corresponding to the target page. When it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within the preset time, the display device can determine whether the page data of the target page contains information used to represent the matching media asset type attribute of the target control, that is, the information representing the media asset type attribute corresponding to the target control. If it does not contain, it means that the operator has not configured the information representing the media asset type attribute for the target control, and the display device will no longer perform the relevant operations for switching the display state; if it contains, it means that the operator has configured the information representing the media asset type attribute for the target control. In this case, the display device can further determine whether there is a preset display state with a matching mapping relationship according to the information representing the media asset type attribute corresponding to the target control. If it exists, a display state switching instruction is generated to control the display area of the target control to switch to the preset display state that has a mapping relationship with the information representing the media asset type attribute corresponding to the target control. The detailed process refers to the above description and will not be elaborated here.

[0124] Optionally, the preset display state can be a zoom animation, a jitter animation, a flicker animation, a rotation animation, etc. This is only an example and does not constitute a limitation to the embodiments of the present application.

[0125] Among them, the switching of the display state has a complex implementation process at the bottom layer. Taking the switch from static display to animation display as an example, it mainly includes six stages: Stage 1: Event delivery; Stage 2: Animation start; Stage 3: Frame-by-frame animation update; Stage 4: Rendering pipeline processing; Stage 5: System composition and display; Stage 6: Animation end and restoration.

[0126] The following takes the example of switching the display state of the target control from the original static display to a zoom animation for illustration:

[0127] Stage 1 (Event delivery): The user places the remote control cursor within the preset area of the target control and hovers for a preset duration, and the display generates a hover event. The hover event is read by the InputReader and sent by the InputDispatcher to the UI thread of the application process. The ViewRootImpl of the UI thread distributes the event to the onTouchEvent method of the control.

[0128] Phase 2 (Animation Start (Main Thread)): The onTouchEvent method triggers the code for implementing the animation. After the code is triggered, it starts to initialize. During the initialization process: The ViewPropertyAnimator creates an ObjectAnimator object and binds the scaleX and scaleY properties. Set the animation duration (300 ms) and interpolator (default Accelerate Decelerate Interpolator). After initialization is complete, register the VSYNC signal: The ObjectAnimator listens for the next VSYNC signal (Vertical Sync signal, approximately 16.6 ms / frame for a 60 Hz screen) through the Choreographer.

[0129] Phase 3 (Frame-by-Frame Animation Update (Main Thread + RenderThread)): 1. VSYNC arrives: After the Choreographer receives the VSYNC signal, it calls back the AnimationHandler. 2. Calculate property values: The ValueAnimator calculates the interpolation values of scaleX and scaleY based on the current time progress (0 - 300 ms). For example: At 150 ms, the interpolator outputs a progress of 0.5, and at this time the scaling value is 1 - 0.2 * 0.5 = 0.9. 3. Update the View property: Through reflection or the optimized path of ViewPropertyAnimator, set button.scaleX = 0.9 and scaleY = 0.9. 4. Trigger a redraw: After the property is modified, call View.invalidate() to mark the control as a "dirty area" and request a redraw. Note: Only mark the area that needs to be redrawn here, and do not immediately execute the drawing.

[0130] Stage 4 (Render Pipeline Processing (Main Thread → RenderThread → GPU)): 1. The main thread generates a DisplayList: In the drawing stage of the next frame (View.draw()), the main thread records the drawing operations of the control (background, text, etc.) into the DisplayList (a list of drawing instructions). The DisplayList contains transformation matrices (including the current scale values scaleX = 0.9, scaleY = 0.9). 2. Submission to RenderThread: The main thread synchronizes the DisplayList to an independent RenderThread. Key point: The main thread continues to process other tasks (such as click events), and the rendering is executed asynchronously by the RenderThread. 3. GPU rendering: The RenderThread converts the DisplayList into OpenGL ES / Vulkan instructions. The GPU executes the instructions and renders the scaling effect of the control into the off-screen buffer (GraphicBuffer of the Surface). The scaling operation may be directly processed by the vertex shader of the GPU without the CPU recalculating the pixels.

[0131] Stage 5 (System Composition and Display (SurfaceFlinger + HWC)): 1. Submission of the frame buffer: The RenderThread notifies SurfaceFlinger that a new frame of the current Surface is ready. 2. Layer composition: SurfaceFlinger composes all the Surfaces of the applications (including the window where the current control is located) into the final frame. If the animation of the control only involves scaling and position changes, the Hardware Composer (HWC) may directly overlay the layers without the GPU redrawing (saving power consumption). 3. Pushing to the screen: The composed frame is submitted to the screen through the FrameBuffer, and the user sees the control shrunk to 90%.

[0132] Stage 6 (End and Restoration of Animation): 1. Animation completion: After 300 ms, the ValueAnimator triggers the onAnimationEnd callback. Reverse the animation or directly reset the properties (scaleX = 1.0, scaleY = 1.0). 2. Reverse process: Repeat Stages 3 - 5 to gradually restore the control size frame by frame until it returns to the initial state.

[0133] In the above embodiments, when it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within the preset time, according to the information characterizing the media type attribute matching of the target control, not only audio playback control but also display state switching control can be performed. This enhances the user perception in the visual dimension.

[0134] In some embodiments, the processor is further configured to: send information for characterizing the media type attribute matched by the target control to the remote controller, so as to instruct the remote controller to control the sound module to emit a sound matching the information and / or control the vibration module to generate a vibration matching the information according to the information characterizing the media type attribute.

[0135] Wherein, after the display device obtains the information for characterizing the media type attribute matched by the target control, it may package the information into a data packet and send the data packet to the pointing remote controller through the data channel between it and the pointing remote controller.

[0136] Wherein, the pointing remote controller stores sound identifiers and vibration mode identifiers corresponding to different information characterizing different media type attributes respectively. After receiving the data packet from the display device, the pointing remote controller parses the data packet to obtain the information for characterizing the media type attribute matched by the target control, looks up the sound identifier and vibration mode identifier corresponding to the information, and controls the sound module to emit the sound corresponding to the sound identifier and / or controls the vibration module to generate the vibration corresponding to the vibration mode identifier.

[0137] Optionally, the pointing remote controller further stores visual state identifiers corresponding to different information characterizing different media type attributes respectively. After receiving the data packet from the display device, the pointing remote controller parses the data packet to obtain the information for characterizing the media type attribute matched by the target control, and may also look up the visual state identifier corresponding to the information and control the visual module to switch to the visual state corresponding to the visual state identifier.

[0138] In the above embodiments, the information for characterizing the media type attribute matched by the target control is sent to the remote controller, so as to instruct the remote controller to control the sound module to emit a sound matching the information and / or control the vibration module to generate a vibration matching the information according to the information characterizing the media type attribute. Through the auditory and tactile feedback mechanisms on the side of the pointing remote controller, the user can more intuitively perceive the attributes of the control, significantly improving the user's interaction experience.

[0139] In some embodiments, when the processor executes to obtain the page data of the target page from the cloud server in response to an open instruction for the target page, it is further configured to: in response to the open instruction for the target page, obtain the time when the page data of the target page was last obtained from the cloud server, calculate the duration from the time to the current time, and when the duration is greater than a preset duration threshold, obtain the page data of the target page from the cloud server.

[0140] As described above, the page data of a page is updated at intervals. When a user frequently opens a certain page within a short period of time, the page data obtained by the display device from the cloud server each time is actually the same. However, the frequent acquisition of page data by the display device from the cloud server brings network pressure and untimely response. Therefore, in the embodiments of the present application, in response to an open instruction for a target page, the display device does not directly obtain the page data of the target page from the cloud server. Instead, it first obtains the time when the page data of the target page was last obtained from the cloud server, calculates the duration from that time to the current time, and if the duration is greater than a preset duration threshold, then obtains the page data of the target page from the cloud server. This can avoid the network pressure caused by frequently obtaining page data from the cloud server.

[0141] Among them, the preset duration threshold can be determined based on the update frequency of the page data. For example, when the update frequency of the page data is once every 5 minutes, the preset duration threshold can also be set to 5 minutes, or greater than 5 minutes.

[0142] In the above embodiments, in response to an open instruction for a target page, the display device obtains the time when the page data of the target page was last obtained from the cloud server, calculates the duration from that time to the current time, and when the duration is greater than the preset duration threshold, then obtains the page data of the target page from the cloud server. This can avoid the network pressure caused by frequently obtaining page data from the cloud server.

[0143] In some embodiments, the processor is further configured to: when it is detected that the display position of the remote control cursor falls within the display area of the target control, call the cursor hover method to obtain the identifier of the target control from the cursor entry event, and based on the identifier of the target control, call the tag acquisition method to obtain the information characterizing the media type attribute matching the target control.

[0144] Combined with the previous description, when an entry event for the target control is captured, it can be considered that the display position of the remote control cursor falls within the display area of the target control. At this time, the cursor hover (onHover) method can be called to obtain the identifier of the target control from the above entry event, and based on the identifier of the target control, the tag acquisition (getTag) method or other custom methods can be called to extract the information characterizing the media type attribute matching the target control from the page data.

[0145] In the above embodiments, a method is provided for extracting the information characterizing the media type attribute matching the target control by calling the cursor hover (onHover) method and the tag acquisition (getTag) method. Subsequently, visual, auditory, and tactile feedback can be triggered according to this information, enabling the user to more intuitively perceive the attributes of the control, significantly improving the user's interaction experience, operation efficiency, and accuracy.

[0146] In some embodiments, before the processor determines whether there is a preset audio with a matching mapping relationship according to the information for characterizing the media asset type attribute matched by the target control, the processor is further configured to: determine whether the interaction switch on the display device side is in an on state; if it is in the on state, then determine whether there is a preset audio with a matching mapping relationship according to the information for characterizing the media asset type attribute matched by the target control.

[0147] After the display device obtains the information for characterizing the media asset type attribute matched by the target control, it may first determine whether the interaction switch on the display device side is in an on state, that is, determine whether the user has enabled the audio-visual effect on the display device side. If it is in the on state, then determine whether there is a preset audio with a matching mapping relationship according to the information for characterizing the media asset type attribute matched by the target control. If there is, generate an instruction to play the audio to instruct the player to play the preset audio having a mapping relationship with the information for characterizing the media asset type attribute; and control the display area of the target control to switch to a preset display state having a mapping relationship with the information for characterizing the media asset type attribute. If it is in the off state, the corresponding audio-visual effect will not be triggered on the display.

[0148] The interaction switch on the display device side may be integrated in the settings or in the menu. The user can control whether the switch is in the on state or the off state by himself / herself. When the switch is in the on state, it means that the user has enabled the audio-visual effect on the display device side; when the switch is in the off state, it means that the user has disabled the audio-visual effect on the display device side.

[0149] In some embodiments, before the processor sends the information for characterizing the media asset type attribute matched by the target control to the remote control, the processor is further configured to: determine whether the interaction switch on the remote control side is in an on state; if it is in the on state, then send the information for characterizing the media asset type attribute matched by the target control to the remote control.

[0150] After the display device obtains the information for characterizing the media asset type attribute matched by the target control, it may first determine whether the interaction switch on the remote control side is in an on state, that is, determine whether the user has enabled the audio-visual touch effect pointing to the remote control side; if it is in the on state, then send the information for characterizing the media asset type attribute matched by the target control to the remote control. If it is in the off state, the information for characterizing the media asset type attribute matched by the target control will not be sent to the remote control.

[0151] Among them, the interaction switch pointing to the remote control side can be integrated in the setting or in the menu. The user can control whether the switch is in the on state or the off state. When the switch is in the on state, it indicates that the user has enabled the audio-visual touch effect pointing to the remote control side; when the switch is in the off state, it indicates that the user has disabled the audio-visual touch effect pointing to the remote control side.

[0152] In the above embodiments, the user can independently control whether the interaction switch on the display device side is on or off. After the display device obtains the information used to characterize the media asset type attribute matching the target control, when it determines that the user has enabled the interaction switch on the display device side, it determines whether there is a preset audio with a matching mapping relationship according to the information used to characterize the media asset type attribute matching the target control. If so, it generates an instruction to play the audio to instruct the player to play the preset audio that has a mapping relationship with the information characterizing the media asset type attribute. The personalized control of the interaction switch on the display device side is realized. In addition, the user can independently control whether the interaction switch pointing to the remote control side is on or off. After the display device obtains the information used to characterize the media asset type attribute matching the target control, when it determines that the user has enabled the interaction switch pointing to the remote control side, it then sends the information used to characterize the media asset type attribute matching the target control to the remote control. The personalized control of the interaction switch pointing to the remote control side is realized.

[0153] In some embodiments, the processor is further configured to: when the information used to characterize the media asset type attribute matching the target control is the first information, update the background color of the display area of the target control to a color matching the first information, switch the display area of the target control to a display state matching the first information, and control the player to play the audio matching the first information.

[0154] Optionally, the media asset type attribute may be an emotion attribute, a highly recommended attribute, an interaction behavior attribute, etc., and the embodiments of the present application do not limit this.

[0155] When the media asset type attribute matched by the target control is an emotion attribute, that is, when the information used to characterize the media asset type attribute matching the target control is the information corresponding to the emotion attribute, update the background color of the display area of the target control to a color matching the information corresponding to the emotion attribute, switch the display area of the target control to a display state matching the information corresponding to the emotion attribute, and control the player to play the audio matching the information corresponding to the emotion attribute.

[0156] Optionally, the emotion attribute may be, for example, a happy attribute, a sad attribute, a horror attribute, etc., and the embodiments of the present application do not limit this.

[0157] As described above, the display device stores audio identifiers and display status identifiers corresponding to different information representing the attributes of different media types. After the display device obtains the information used to represent the media type attributes matched by the target control from the page data, it can find the audio identifier and display status identifier corresponding to this information. On the one hand, it controls the player to play the audio corresponding to this audio identifier, and on the other hand, it switches the display area of the target control to the display status corresponding to this display status identifier.

[0158] As described above, the pointing remote control stores visual status identifiers, sound identifiers, and vibration mode identifiers corresponding to different information representing the attributes of different media types. After the pointing remote control receives the data packet from the display device, it parses the data packet to obtain the information used to represent the media type attributes matched by the target control, and finds the visual status identifier, sound identifier, and vibration mode identifier corresponding to this information, controls the visual module to switch to the visual status corresponding to this visual status identifier, controls the sound generating module to emit the sound corresponding to this sound identifier, and / or controls the vibration module to generate the vibration corresponding to this vibration mode identifier.

[0159] In some embodiments, when the media type attribute matched by the target control is a happy attribute, that is, when the information used to represent the media type attribute matched by the target control is the information corresponding to the happy attribute, on the display device side, the background color of the display area of the target control is updated to a color that matches the information corresponding to the happy attribute, a scaling animation of the target control is displayed, and the player is controlled to play the audio that matches the information corresponding to the happy attribute. On the pointing remote control side, the visual module is controlled to switch to the visual status that matches the information corresponding to the happy attribute, the sound generating module is controlled to emit the sound that matches the information corresponding to the happy attribute, and / or the vibration module is controlled to generate the vibration that matches the information corresponding to the happy attribute. Specifically, the vibration module can be controlled to generate vibrations at the first frequency, and the sound generating module can be controlled to emit the beeping sound that matches the information corresponding to the happy attribute.

[0160] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to green, accompanied by a slight scaling animation and a pleasant sound effect of the target control. On the pointing remote control side, the vibration module can be controlled to generate high-frequency vibrations, and the sound generating module can be controlled to emit a pleasant beeping sound.

[0161] In some embodiments, when the media type attribute matched by the target control is the sad attribute, that is, when the information characterizing the media type attribute matched by the target control is the information corresponding to the sad attribute, on the display device side, update the background color of the display area of the target control to a color matching the information corresponding to the sad attribute, display the blinking animation of the target control, and control the player to play the audio matching the information corresponding to the sad attribute; on the pointing remote control side, control the visual module to switch to the visual state matching the information corresponding to the sad attribute, control the sound module to emit the sound matching the information corresponding to the sad attribute, and / or control the vibration module to generate the vibration matching the information corresponding to the sad attribute. Specifically, the vibration module can be controlled to generate vibrations at a second frequency, and the sound module can be controlled to emit a beeping sound matching the information corresponding to the sad attribute; the second frequency is less than the first frequency.

[0162] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to blue, accompanied by a slow blinking animation and a low-pitched sound effect of the target control. On the pointing remote control side, the vibration module can be controlled to generate low-frequency vibrations, and the sound module can be controlled to emit a low-pitched beeping sound.

[0163] In some embodiments, when the media type attribute matched by the target control is the horror attribute, that is, when the information characterizing the media type attribute matched by the target control is the information corresponding to the horror attribute, on the display device side, update the background color of the display area of the target control to a color matching the information corresponding to the horror attribute, display the jitter animation of the target control, and control the player to play the audio matching the information corresponding to the horror attribute; on the pointing remote control side, control the visual module to switch to the visual state matching the information corresponding to the horror attribute, control the sound module to emit the sound matching the information corresponding to the horror attribute, and / or control the vibration module to generate the vibration matching the information corresponding to the horror attribute. Specifically, the vibration module can be controlled to generate vibrations at a preset speed, and the sound module can be controlled to emit a beeping sound matching the information corresponding to the horror attribute.

[0164] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to red, accompanied by a rapid jitter animation and a warning sound effect of the target control. On the pointing remote control side, the vibration module can be controlled to emit rapid vibrations, and the sound module can be controlled to emit a warning beeping sound.

[0165] In the above embodiments, when the media type attribute matched by the target control is an emotional attribute, the display device will trigger the corresponding audio-visual effects for this emotional attribute, and the pointing remote control will trigger the corresponding audio-visual and tactile effects for this emotional attribute, enhancing the user's intuitive perception of the resource content corresponding to the target control.

[0166] When the media type attribute matched by the target control is the highly recommended attribute, that is, when the information used to represent the media type attribute matched by the target control is the information corresponding to the highly recommended attribute, on the display device side, the background color of the display area of the target control can be updated to a color that matches the information corresponding to the highly recommended attribute, the rotation animation of the target control can be displayed, and the player can be controlled to play the audio that matches the information corresponding to the highly recommended attribute; on the pointing remote control side, the visual module can be controlled to flash a color that matches the information corresponding to the highly recommended attribute.

[0167] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to yellow, accompanied by the rotation animation and prompt sound effect of the target control. On the pointing remote control side, the pointing remote control controls the visual module to flash yellow.

[0168] In the above embodiments, when the media type attribute matched by the target control is the highly recommended attribute, the display device will trigger the audio-visual effects corresponding to the highly recommended attribute, and the pointing remote control will trigger the audio-visual touch effects corresponding to the highly recommended attribute, which can guide the user to click on the target control and improve the reach rate of the resource content corresponding to the target control.

[0169] In some embodiments, when the media type attribute matched by the target control is the prohibited click, that is, when the information used to represent the media type attribute matched by the target control is the information corresponding to the prohibited click, on the display device side, the background color of the display area of the target control is updated to a color that matches the information corresponding to the prohibited click, and the player is controlled to play the audio that matches the information corresponding to the prohibited click; on the pointing remote control side, the visual module is controlled to switch to the visual state that matches the information corresponding to the prohibited click, the sound module is controlled to emit the sound that matches the information corresponding to the prohibited click, and / or the vibration module is controlled to generate the vibration that matches the information corresponding to the prohibited click, including: controlling the vibration module to generate the third frequency vibration and controlling the sound module to emit a warning sound. The third frequency can be equal to or greater than the first frequency.

[0170] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to dark red, accompanied by a warning sound. On the pointing remote control side, the vibration module can be controlled to generate high-frequency vibration and the sound module can be controlled to emit a warning sound.

[0171] In some embodiments, when the media type attribute matched by the target control supports clicking, that is, when the information indicating the media type attribute matched by the target control is the information corresponding to support for clicking, on the display device side, update the background color of the display area of the target control to a color matching the information corresponding to support for clicking, display an enlarged animation of the target control, and control the player to play audio matching the information corresponding to support for clicking; on the pointing remote control side, control the visual module to switch to a visual state matching the information corresponding to support for clicking, control the sound generating module to emit a sound matching the information corresponding to support for clicking, and / or control the vibration module to generate a vibration matching the information corresponding to support for clicking, including: controlling the vibration module to generate a single discontinuous vibration.

[0172] Exemplarily, on the display device side, the background color of the display area of the target control can be updated to light red, accompanied by an enlarged effect of the target control and a system button sound. On the pointing remote control side, control the vibration module to generate a single discontinuous vibration with an interval of 2 s.

[0173] In the above embodiments, when the media type attribute matched by the target control supports clicking and prohibits clicking, different effects will be triggered on the display device and the pointing remote control, enhancing the user's intuitive perception of whether the target control itself supports clicking, and being more user-friendly to the user groups with hearing and visual impairments.

[0174] In some embodiments, the processor is further configured to: in response to selecting a first control among multiple controls through the remote control, display a jump interface associated with the first control on the display, and send a jump success instruction to the remote control to instruct the remote control to control the sound generating module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the jump success instruction; in response to a page sliding operation triggered through the remote control, display the page content after sliding on the display, and send a sliding success instruction to the remote control to instruct the remote control to control the sound generating module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the sliding success instruction; in response to a control dragging operation on a second control triggered through the remote control, display the process of dragging the second control on the display, and send a dragging success instruction to the remote control to instruct the remote control to control the sound generating module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the dragging success instruction.

[0175] Among them, for at least one control included in the target page, when the remote control cursor of the pointing remote control hovers over a certain control, the user can select the control through the pointing remote control. For example, press the confirmation key on the pointing remote control, see Figure 8As shown, for the convenience of description, this control is referred to as the first control. In response to the first control being selected, the display device displays the jump interface associated with the first control on the display and sends a jump success instruction to the pointing remote control. The jump success instruction carries the unique identifier of the first control. After receiving the jump success instruction, the pointing remote control controls the visual module to switch to the corresponding visual state, controls the sound module to emit the corresponding sound, and / or controls the vibration module to generate the corresponding vibration. Exemplarily, after receiving the jump success instruction, the pointing remote control controls the sound module to emit a jump success prompt sound.

[0176] Among them, the pointing remote control has a touch area, and the user can trigger a page sliding operation on the currently displayed page of the display device through this touch area. Refer to Figure 9 As shown, in response to this page sliding operation, the display device displays the content of the page after sliding on the display and sends a sliding success instruction to the pointing remote control. After receiving the jump success instruction, the pointing remote control controls the visual module to switch to the corresponding visual state, controls the sound module to emit the corresponding sound, and / or controls the vibration module to generate the corresponding vibration. Exemplarily, after receiving the sliding success instruction, the pointing remote control can control the sound module to emit a sliding success prompt sound.

[0177] Among them, for at least one control included in the target page, when the remote control cursor of the pointing remote control hovers over a certain control, the user can trigger a control dragging operation on this control through the pointing remote control. For example, hold down the confirmation key on the pointing remote control to control the remote control cursor to move to the target position. Refer to Figure 10 As shown, for the convenience of description, this control is referred to as the second control. In response to the above control dragging operation, the display device displays the process of dragging the second control on the display and sends a dragging success instruction to the pointing remote control. After receiving the dragging success instruction, the pointing remote control controls the visual module to switch to the corresponding visual state, controls the sound module to emit the corresponding sound, and / or controls the vibration module to generate the corresponding vibration. Exemplarily, after receiving the dragging success instruction, the pointing remote control can control the sound module to emit a dragging success prompt sound.

[0178] In the above embodiments, the user's selection, sliding, dragging and other behaviors will trigger corresponding audio-visual and tactile effects on the side of the pointing remote control, enhancing the user's intuitive perception of these behaviors.

[0179] In some embodiments, an interaction method is provided, such as Figure 11As shown, the user turns on the display device, points the pointing remote control and connects it to the display device via Bluetooth. The user opens the target page, and the display device determines whether the cached content needs to be updated. Specifically, the display device can obtain the time when the page data of the target page was last obtained from the cloud server, calculate the duration from that time to the current time, and determine that the cached content needs to be updated when the duration is greater than the preset duration threshold; otherwise, it does not need to be updated. When the cached content needs to be updated, the display device obtains the page data of the target page from the cloud server. The page data includes the control content of at least one control pre-configured for the target page and the information representing the media type attribute corresponding to each control. When the cached content does not need to be updated, the display device directly uses the page data of the target page obtained from the cloud server last time. After the user points the pointing remote control at the target control, the display device obtains the information used to represent the media type attribute matching the target control, and determines whether the interaction switch on the display device side is in the on state. If it is in the on state, it controls the player to play the corresponding audio according to the information used to represent the media type attribute matching the target control, and switches the target control to the corresponding display state to trigger the corresponding audio-visual effect. The display device can also determine whether the interaction switch on the remote control side is in the on state. If it is in the on state, it sends the information used to represent the media type attribute matching the target control to the pointing remote control. The pointing remote control controls the visual module to switch to the corresponding visual state, controls the sound module to emit the corresponding sound, and / or controls the vibration module to generate the corresponding vibration according to the information used to represent the media type attribute matching the target control, so as to trigger the corresponding audio-visual and tactile effects. The user makes a decision on the next interaction behavior based on the audio-visual effects on the display device side and the audio-visual and tactile effects on the pointing remote control side.

[0180] In some embodiments, an interaction method is provided, such as Figure 12As shown, it includes the following steps: 1. When the user wants to start the display device, the user can trigger the switch on the pointing remote control. In response to this trigger operation, the pointing remote control sends a start instruction to the display device. 2. The pointing remote control wakes up the internal system and continuously performs Bluetooth broadcasting. 3. The display device starts up and initializes. 4. The display device starts Bluetooth device scanning. 5. The display device receives the pointing remote control information from the pointing remote control. 6. The display device processes the pointing remote control information. 7. The display device requests a Bluetooth connection from the pointing remote control. 8. The pointing remote control processes the TV pairing information. 9. The pointing remote control sends an approval connection message to the display device. 10. The display device completes the Bluetooth pairing connection with the pointing remote control. 11. The user can normally use the pointing remote control to click. 12. The user opens the target page. 13. The display device performs page initialization and checks whether the cached content needs to be updated. 14. If an update is required, the display device sends a page data acquisition request for the target page to the cloud server, and the request includes the device information of the display device. 15. The cloud server sorts out the page data according to the device information. 16. The cloud server sends the page data to the display device. 17. The display device updates the target page, displays the UI control, and sets the information characterizing the media asset type attribute. 18. The user uses the pointing remote control to point at the target control. 19. The pointing remote control sends the data characterizing the display position of the remote control cursor to the display device. 20. The display device obtains the information characterizing the media asset type attribute matching the target control. 21. Determine whether the user enables the audio-visual effect on the display device side and indicates the audio-visual and touch effect on the pointing remote control side. 22. If the audio-visual and touch effect on the pointing remote control side is enabled, the information characterizing the media asset type attribute matching the target control is sent to the pointing remote control. 23. The display device triggers the corresponding audio-visual effect on the display based on the information characterizing the media asset type attribute matching the target control. 24. The pointing remote control triggers the corresponding audio-visual and touch effect based on the information characterizing the media asset type attribute matching the target control. For the detailed implementation, reference can be made to the foregoing embodiments, which will not be elaborated herein.

[0181] In some embodiments, an interaction method is provided, including the following steps: receiving the data characterizing the display position of the remote control cursor, and drawing and displaying the remote control cursor on the operation interface according to the data characterizing the display position of the remote control cursor; wherein, the display position of the cursor on the operation interface is determined by the position pointed by the remote control in the three-dimensional space; when it is detected that the display position of the remote control cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, according to the information characterizing the media asset type attribute matching the target control, determine whether there is a preset audio with a matching mapping relationship, and if so, generate an instruction to play the audio to instruct the player to play the preset audio having a mapping relationship with the information characterizing the media asset type attribute. For the detailed implementation of this method, reference can be made to the above, and the embodiments of the present application will not be elaborated herein.

[0182] In some embodiments, a remote controller is provided, including: an interaction module; the interaction module includes a sound-emitting module and a vibration module; and at least one processor, which is connected to the interaction module and is configured to execute instructions to cause the remote controller to: receive information sent by a display device for characterizing that a target control matches a media type attribute; according to the information for characterizing that the target control matches the media type attribute, control the sound-emitting module to emit a sound matching the information, and / or control the vibration module to generate a vibration matching the information. For the detailed implementation process on the remote controller side, refer to the above, and the embodiments of the present application will not be elaborated herein again.

[0183] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0184] For the sake of convenience of explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, Including: A display configured to: display an operation interface, where a cursor is present in the operation interface, and the display position of the cursor on the operation interface is determined by the position pointed to by a remote controller in a three-dimensional space; A processor configured to: receive data representing the display position of the remote controller cursor, and based on the data representing the display position of the remote controller cursor, draw and display the remote controller cursor on the operation interface; When it is detected that the display position of the remote controller cursor falls within the display area of a target control and does not move out of the display area of the target control within a preset time, determine whether there is a preset audio with a matching mapping relationship according to the information characterizing the media type attribute matched by the target control. If there is, generate an instruction to play the audio to instruct a player to play the preset audio having a mapping relationship with the information characterizing the media type attribute.

2. The display device according to claim 1, wherein The processor is further configured to: Determine whether there is a preset display state with a matching mapping relationship according to the information characterizing the media type attribute matched by the target control. If there is, generate a display state switching instruction to control the display area of the target control to switch to the preset display state having a mapping relationship with the information characterizing the media type attribute.

3. The display device according to claim 1, wherein The processor is further configured to: Send the information characterizing the media type attribute matched by the target control to the remote controller to instruct the remote controller to control a sound generating module to emit a sound matching the information, and / or control a vibration module to generate a vibration matching the information according to the information characterizing the media type attribute.

4. The display device according to claim 1, wherein The processor is further configured to: When it is detected that the display position of the remote controller cursor falls within the display area of a target control, call a cursor hovering method to obtain the identifier of the target control, and based on the identifier of the target control, call a label obtaining method to obtain the information characterizing the media type attribute matched by the target control.

5. The display device according to claim 1, wherein Before the processor executes to determine whether there is a preset audio with a matching mapping relationship according to the information characterizing the media type attribute matched by the target control, the processor is further configured to: Judge whether an interaction switch on the display device side is in an on state; If it is in an on state, determine whether there is a preset audio with a matching mapping relationship according to the information characterizing the media type attribute matched by the target control.

6. The display device according to claim 3, wherein Before the processor executes to send the information characterizing the media type attribute matched by the target control to the remote controller, the processor is further configured to: Judge whether an interaction switch on the remote controller side is in an on state; If it is in an on state, send the information characterizing the media type attribute matched by the target control to the remote controller.

7. The display device according to claim 1, characterized in that The processor is further configured to: In the case where the information characterizing the media type attribute matched by the target control is the first information, update the background color of the display area of the target control to a color matching the first information, switch the display area of the target control to the display state matching the first information, and control the player to play the audio matching the first information.

8. The display device according to claim 1, wherein The processor is further configured to: In response to the selection of the first control among the multiple controls via the remote controller, a jump interface associated with the first control is displayed on the display, and a jump success instruction is sent to the remote controller to instruct the remote controller to control the sound module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the jump success instruction; In response to a page sliding operation triggered via the remote controller, the content of the slid page is displayed on the display, and a slide success instruction is sent to the remote controller to instruct the remote controller to control the sound module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the slide success instruction; In response to a control dragging operation on the second control triggered via the remote controller, the process of dragging the second control is displayed on the display, and a drag success instruction is sent to the remote controller to instruct the remote controller to control the sound module to emit a corresponding sound and / or control the vibration module to generate a corresponding vibration based on the drag success instruction.

9. A display method, characterized in that, Comprising: Receiving data representing the display position of the remote controller cursor, and drawing and displaying the remote controller cursor on the operation interface according to the data representing the display position of the remote controller cursor; wherein, the display position of the cursor on the operation interface is determined by the position pointed to by the remote controller in the three-dimensional space; When it is detected that the display position of the remote controller cursor falls within the display area of the target control and does not move out of the display area of the target control within a preset time, determine whether there is a preset audio with a matching mapping relationship according to the information used to characterize the media type attribute matching of the target control. If so, generate an instruction to play the audio to instruct the player to play the preset audio having a mapping relationship with the information used to characterize the media type attribute.

10. A remote control, characterized in that, Comprising: An interaction module, the interaction module includes: a sound module and a vibration module; And at least one processor, which is connected to the interaction module and is configured to execute instructions to enable the remote controller to: Receive the information used to characterize the media type attribute matching of the target control sent by the display device; control the sound module to emit a sound matching the information and / or control the vibration module to generate a vibration matching the information according to the information used to characterize the media type attribute matching of the target control.

Images