A display device

By introducing global menu controls and global return controls into the display device, the characteristics of the touch screen are used to solve the problem of cumbersome operation of home touch display devices, improving user experience and simplicity of operation, and becoming a unified entrance to smart TV functions.

CN114296623BActive Publication Date: 2024-11-26HISENSE VISUAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011551199.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-24
Publication Date
2024-11-26
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

There are fewer products for home touch display devices, the user's operation steps are complicated, and the touch advantages cannot be fully reflected, and the user experience is poor.

Method used

It provides a display device that uses a touch screen and a controller, and uses global menu controls and global return controls to quickly find functional portals and simplify the operation process.

Benefits of technology

Through the design of global menu controls and global return controls, we make full use of touch characteristics to simplify user operations, improve user experience, make up for the shortcomings of remote control operations, and achieve centralized control of smart TV functions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114296623B_ABST
    Figure CN114296623B_ABST
Patent Text Reader

Abstract

The present application discloses a display device, which is used to centrally control the various function entrances of a smart touch-screen TV, so that users can quickly find the functions they want to use, thereby improving the user experience. The display device includes: a display, the display includes a touch screen; a user interface; a controller connected to the display and the user interface respectively, and used to execute: in response to a first input instruction, control the display to display a global menu control, the first input instruction is input by the user touching the touch screen with a finger.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to a display device. Background Art

[0002] At present, there are few types of household touch display devices on the market, and there is almost no interactive design based on touch in related technologies. After using touch screens on display devices, traditional remote controls cannot reflect the advantages of touch display devices, and when users want to find the applications or tools they want, the operation steps are cumbersome and the user experience is not good. Summary of the invention

[0003] The embodiment of the present application provides a display device for centrally controlling various function entrances of a smart touch-screen TV, so that a user can quickly find the function he or she wants to use, thereby improving the user experience.

[0004] In a first aspect, a display device is provided, comprising:

[0005] A display, wherein the display comprises a touch screen;

[0006] User interface;

[0007] A controller connected to the display and the user interface, respectively, for executing:

[0008] In response to a first input instruction, the display is controlled to display a global menu control, wherein the first input instruction is input by a user touching the touch screen with a finger.

[0009] In some embodiments, the controller is further configured to execute:

[0010] In response to a second input instruction, the display is controlled to display a global menu control at a position where the user's finger stops moving, wherein the second input instruction is input by the user pressing the global menu control with a finger and then moving the finger on the touch screen.

[0011] In some embodiments, the controller is configured to execute:

[0012] In response to a third input instruction, the position of the global menu control is synchronously updated according to the center point coordinates of all finger touch coordinates, and the third input instruction is input by the user touching the touch screen with at least four fingers.

[0013] In some embodiments, the controller is further configured to execute:

[0014] In response to an instruction of a user selecting the global menu control, controlling the display to display a menu display area, wherein the menu display area includes a menu editing control;

[0015] The menu display area also includes at least one application or tool control;

[0016] The menu display area also includes a global return control.

[0017] In some embodiments, the controller is further configured to execute:

[0018] In response to an instruction by a user to select the menu editing control, controlling the display to display a menu editing area;

[0019] In response to a fourth input instruction from the user, an application or tool control is added to the menu display area, or an application or tool control in the menu display area is deleted, wherein the fourth input instruction is input by the user in the menu editing area by clicking or dragging the application or tool control with his finger.

[0020] In a second aspect, a display device is provided, including:

[0021] A display, wherein the display comprises a touch screen;

[0022] User interface;

[0023] A controller connected to the display and the user interface, respectively, for executing:

[0024] In response to a first input instruction, the display is controlled to display a global menu control and a global return control, wherein the first input instruction is input by a user touching the touch screen with a finger.

[0025] In some embodiments, the controller is further configured to execute:

[0026] In response to a second input instruction, the display is controlled to display a global menu control and a global return control at the position where the user's finger stops moving. The second input instruction is input by the user pressing the global menu control and the global return control with his finger and then moving the input on the touch screen.

[0027] In some embodiments, the controller is further configured to execute: in response to a third input instruction, synchronously updating the positions of the global menu control and the global return control according to the center point coordinates of all finger touch coordinates, wherein the third input instruction is input by the user touching the touch screen with at least four fingers;

[0028] The synchronously updating the positions of the global menu control and the global return control according to the center point coordinates of all finger touch coordinates specifically includes:

[0029] Get the touch coordinates of each touch point;

[0030] Determine the maximum and minimum coordinates of all touch coordinates;

[0031] Calculate the Cartesian coordinate distance between the maximum value of the coordinate and the minimum value of the coordinate;

[0032] If the Cartesian coordinate distance is greater than the preset pixel, calculate the center point coordinates of all touch coordinates;

[0033] The global menu control and the global return control are displayed at positions corresponding to the center point coordinates.

[0034] In some embodiments, the controller is further configured to execute:

[0035] In response to a screen rotation instruction input by a user, controlling the rotating component to drive the display to rotate;

[0036] Recording the drag coordinates of the global menu control and the global return control in real time;

[0037] Calculate a scale factor according to the state of the screen before rotation and the dragging coordinates;

[0038] Calculate display coordinates according to the scale factor and the state of the screen after rotation;

[0039] After the rotation is completed, the global menu control and the global return control are displayed at the positions corresponding to the display coordinates.

[0040] In some embodiments, the controller is further configured to execute:

[0041] In response to an instruction of a user selecting the global menu control, controlling the display to display a menu display area, wherein the menu display area includes a menu editing control;

[0042] The menu display area also includes at least one application or tool control;

[0043] In response to an instruction by a user to select the menu editing control, controlling the display to display a menu editing area;

[0044] In response to a fourth input instruction of the user, adding an application or a tool control to the menu display area, or deleting an application or a tool control in the menu display area, wherein the fourth input instruction is input by the user in the menu editing area by clicking or dragging an application or a tool control with a finger;

[0045] The adding of an application or a tool control to the menu display area, or the deleting of an application or a tool control in the menu display area, specifically includes:

[0046] If it is detected that the user's finger touches the touch screen, the item coordinates of the point where the user's finger falls are obtained;

[0047] If the item coordinates are valid coordinates and the user's finger touch duration exceeds a first preset duration, obtain a background image of the application or tool control corresponding to the current item coordinates, and draw a floating mirror image of the background image;

[0048] hiding the application or tool controls;

[0049] Update the position of the suspended image according to the position of the user's finger movement;

[0050] If it is detected that the user's finger leaves the touch screen, the display position of the application or tool control is updated, and the floating image is hidden.

[0051] In the above embodiment, by setting and displaying the global menu control and the global return control, the touch characteristics of the display device are fully utilized to make up for the shortcomings of the remote control operation. The global menu control and the global return control also become the unified entrance to various functions of the smart TV, and each function is centrally controlled, so that users can quickly find the functions they want to use, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1A shows a usage scenario of a display device according to some embodiments;

[0053] Figure 1B exemplarily shows a rear view of a display device according to some embodiments;

[0054] Figure 2 shows a hardware configuration block diagram of a control device 100 according to some embodiments;

[0055] Figure 3 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0056] Figure 4 shows a software configuration diagram in a display device 200 according to some embodiments;

[0057] Figure 5 A user interface schematic diagram according to some embodiments is exemplarily shown in FIG.

[0058] Figure 6-Figure 8 A user interface schematic diagram according to some embodiments is exemplarily shown in FIG.

[0059] Fig. 9 A schematic diagram of a menu display area according to some embodiments is exemplarily shown in FIG.

[0060] Figure 10-12 A schematic diagram exemplarily illustrates a global menu control and a global return control according to some embodiments;

[0061] Fig.13 A schematic diagram exemplarily illustrates a global menu control according to some embodiments;

[0062] Figure 14-16 Another user interface schematic diagram according to some embodiments is exemplarily shown in FIG.

[0063] Figure 17-Figure 20 exemplarily shows a schematic diagram of another menu display area according to some embodiments;

[0064] Figure 21-22 Another user interface schematic diagram according to some embodiments is exemplarily shown in FIG.

[0065] Figure 23-Figure 27 A schematic diagram of a menu editing area according to some embodiments is exemplarily shown in FIG. DETAILED DESCRIPTION

[0066] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0067] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0068] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0069] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0070] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0071] Figure 1A Schematic diagram of a usage scenario of a display device according to an embodiment. Figure 1AAs shown, the display device 200 also performs data communication with the server 400 , and the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0072] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, to control the display device 200 wirelessly or wired. The user may control the display device 200 by inputting user commands through at least one of buttons on the remote controller, voice input, and control panel input.

[0073] In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a laptop computer, an AR / VR device, etc.

[0074] In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 may be controlled using an application running on the smart device.

[0075] In some embodiments, the smart device 300 and the display device may also be used to communicate data.

[0076] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0077] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0078] In some embodiments, the software steps executed by a step execution subject can be migrated to another step execution subject in data communication with it as needed for execution. Exemplarily, the software steps executed by a server can be migrated to a display device in data communication with it as needed for execution, and vice versa.

[0079] In some embodiments, Figure 1BAs shown, the display device 200 includes a rotating assembly 276, a controller 250, a display 275, a terminal interface 278 extending from a gap on the back panel, and a rotating assembly 276 connected to the back panel, and the rotating assembly 276 can rotate the display 275. From the perspective of viewing the display device from the front, the rotating assembly 276 can rotate the display screen to a vertical screen state, that is, a state in which the vertical side length of the screen is greater than the horizontal side length, and can also rotate the screen to a horizontal screen state, that is, a state in which the horizontal side length of the screen is greater than the vertical side length.

[0080] In some embodiments, the display 275 includes a touch screen, which includes a capacitive touch screen. The capacitive touch screen is a four-layer composite glass screen. The inner surface and the interlayer of the glass screen are each coated with a layer of ITO (coated conductive glass). The outermost layer is a thin layer of silica glass protective layer. The ITO coating serves as a working surface, and four electrodes are led out from the four corners. The inner layer of ITO is a shielding layer.

[0081] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0082] In some embodiments, the communication interface 130 is used for communicating with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module.

[0083] In some embodiments, the user input / output interface 140 includes at least one of a microphone, a touch pad, a sensor, a button, or alternative modules.

[0084] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0085] In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0086] In some embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0087] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0088] In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0089] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0090] In some embodiments, the communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0091] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0092] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0093] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0094] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0095] In some embodiments, the object may be any one of the selectable objects, such as a hyperlink, an icon or other operable controls. Operations related to the selected object include: displaying a page, document, image, etc. connected to the hyperlink, or executing a program corresponding to the icon.

[0096] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0097] CPU processor. Used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video content. CPU processor may include multiple processors. For example, it includes a main processor and one or more sub-processors.

[0098] In some embodiments, the graphics processor is used to generate various graphic objects, such as at least one of icons, operation menus, and user input command display graphics. The graphics processor includes an operator, which performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes; and a renderer, which renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0099] In some embodiments, the video processor is used to receive an external video signal and perform at least one of the following video processing operations, such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis, according to a standard codec protocol of the input signal, to obtain a signal that can be directly displayed or played on the display device 200.

[0100] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, and a display formatting module. The demultiplexing module is used to demultiplex the input audio and video data stream. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling. The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the scaled video image to generate an image signal for display. The frame rate conversion module is used to convert the input video frame rate. The display formatting module is used to receive the video output signal after frame rate conversion and change the signal to conform to the display format signal, such as outputting an RGB data signal.

[0101] In some embodiments, the audio processor is used to receive an external audio signal, perform decompression and decoding according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

[0102] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0103] In some embodiments, "user interface" refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on a display screen of an electronic device, wherein a control may include at least one of icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0104] In some embodiments, the user interface 280 is an interface that can be used to receive control input (such as a physical button on the display device body, or other interfaces).

[0105] In some embodiments, the system of the display device may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together constitute the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0106] See also Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0107] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0108] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0109] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0110] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0111] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0112] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0113] In some embodiments, after the user turns on the display device and enters the boot interface or an application, the user touches the screen of the display with a finger, and the controller controls the display to display the global menu control and the global return control.

[0114] In some embodiments, after the user touches the screen of the display with a finger, the controller controls the display to display only the global menu control, and then displays the global return control after the user clicks the global menu control.

[0115] In some embodiments, Figure 5 As shown, the controller controls the display to display the global menu control and the global return control, including:

[0116] Step S51: monitor the display device startup broadcast and start SoftControlService (soft control service);

[0117] Step S52: Register InputFilter (input filter) monitoring;

[0118] In some implementations, the display device will pull the soft control service after it is turned on, and the gesture action instructions or remote control instructions sent by the user can be obtained by registering input filter monitoring.

[0119] Step S53: receiving a command input by a user through a user interface;

[0120] Step S54: determining whether the instruction input by the user is a MotionEvent (touch event);

[0121] After receiving the user input command, it is necessary to distinguish whether the input command is a MotionEvent (touch event) or a KeyEvent (keyboard event).

[0122] If it is a touch event, execute step S55.

[0123] Step S55: performing algorithm matching processing on the touch event, and determining whether to display the global menu control and the global return control according to the processing result.

[0124] In some embodiments, the algorithm matching process includes pointCount (number of touch points), flingerdistance (sliding distance), flinger Vector (sliding vector), etc.

[0125] Exemplarily, taking the number of touch points as an example, the method for performing algorithm matching processing on touch events is as follows:

[0126] Touch events include the following five scenarios:

[0127] Scenario 1: Only one finger touches the screen, and its coordinate point can be recorded as: pos=0, X=156.90448, Y=533.3757;

[0128] Scenario 2: Two fingers touch the screen, and their coordinates can be recorded as: pos=0, X=156.90448, Y=533.3757, pos=1, X=119.90448, Y=747.3757;

[0129] Scenario 3: Three fingers touch the screen, and their coordinate points can be recorded as: pos=0, X=156.90448, Y=533.3757, pos=1, X=119.90448, Y=747.3757, pos=2, X=229.90448, Y=678.3757;

[0130] Scenario 4: Four fingers touch the screen, and their coordinate points can be recorded as: pos=0, X=156.90448, Y=533.3757, pos=1, X=119.90448, Y=747.3757, pos=2, X=229.90448, Y=678.3757, pos=3, X=236.90448, Y=603.3757;

[0131] Scene 5: Five fingers touch the screen, and their coordinate points can be recorded as: pos=0, X=156.90448, Y=533.3757, pos=1, X=119.90448, Y=747.3757, pos=2, X=229.90448, Y=678.3757, pos=3, X=236.90448, Y=603.3757, pos=4, X=210.90448, Y=574.3757.

[0132] The number of touch points in the above scenes one to five corresponds to 1-5 respectively. In some embodiments, only when the number of touch points is 1-3, the display is controlled to display the global menu control and the global return control. Exemplarily, when the number of touch points is 1, the display is controlled to display the global menu control and the global return control; when the number of touch points is 2, it is necessary to calculate the Cartesian distance between the two touch points, and when the Cartesian distance is greater than 100px, the display is controlled to display the global menu control and the global return control; when the number of touch points is 3, it is necessary to calculate the Cartesian distance of the maximum and minimum coordinates of the three touch points, and when the Cartesian distance is greater than 200px, the display is controlled to display the global menu control and the global return control.

[0133] In some embodiments, ideally, when two or three fingers touch the screen of the display at the same time, the global menu control and the global return control will be displayed. However, after the user touches the screen of the display, the fingers often touch the screen one after another. When one finger touches the screen, the global menu control and the global return control are often displayed directly.

[0134] In some embodiments, the global menu control and the global return control are displayed by default, and the custom menu display area is displayed after clicking the global menu control. The global menu control and the global return control are suspended above all applications and are visible in all scenarios.

[0135] In some embodiments, after the display device is turned on, the global menu control and the global return control can be displayed as long as the user's finger touches the screen of the display except in some specific scenarios. Among them, in order not to interrupt the ATV channel search process, the global menu control and the global return control are not displayed after the user's finger touches the screen of the display during the ATV channel search process; in some modes that require remote control operation to complete, or in the process of basic parameter setting, the global menu control and the global return control are not displayed after the user's finger touches the screen of the display, such as in factory mode and during the boot navigation process; when the finger touches the screen to complete drawing or text annotation, the global menu control and the global return control are also not displayed, such as in whiteboard applications and annotation applications; during the rotation of the display, the display interface shows the rotation animation and does not need to display the global menu control and the global return control. Therefore, during the rotation of the display, the global menu control and the global return control are not displayed after the user's finger touches the screen of the display.

[0136] In some embodiments, after turning on the display device, the user will enter the startup homepage or enter an application through the remote control. Figure 7 As shown in FIG. 1 , when the user's finger touches any position of the display screen, Figure 6 As shown in FIG. 8 , the display device provides a user interface schematic diagram to the display. Figure 6 The user interface is in Figure 7 A global menu control 61 and a global return control 62 are added on the basis of the user interface. Figure 8 The user interface is in Figure 7 Add a global menu control 61 based on the user interface of Figure 8 In FIG. 6 , when the user clicks the global menu control 61, the menu display area 63 displays the global return control 62, such as Fig. 9 The global menu control 61 and the global return control 62 can be suspended or embedded. Figure 7 User interface.

[0137] It should be noted that a control refers to a visual object displayed in a user interface of a display device to represent corresponding content such as an icon, a thumbnail, a video clip, a link, etc.

[0138] The display forms of controls are usually diverse. For example, a control may include text content and / or an image for displaying a thumbnail related to the text content. For another example, a control may be text and / or an icon of an application.

[0139] In some embodiments, Fig.10 As shown, the global menu control 61 and the global return control 62 may be simplified graphics; Fig.11 As shown, the global menu control 61 and the global return control 62 can be text; Fig.12As shown, the global menu control 61 and the global return control 62 may be simplified graphics and text; Fig.13 As shown, the global menu control 61 may be a character image.

[0140] In some embodiments, the current user interface displays a global menu control and a global return control, but the user does not input any instructions within a second preset time period, and the controller controls the display to display the global menu control and the global return control in a semi-transparent state; after a third preset time period, the user still does not input any instructions, and the display does not display the global menu control and the global return control.

[0141] In some embodiments, Figure 6 As shown, when no user input instruction is received within 5 seconds after the global menu control 61 and the global return control 62 are called out, the global menu control 61 and the global return control 62 become semi-transparent, as shown in FIG. Fig.14 If no user input is received after another 5 seconds, the global menu control 61 and the global return control 62 automatically disappear. Figure 7 The user interface in .

[0142] In some embodiments, the user controls the rotating component to rotate the display by sliding at least two fingers on the screen or inputting a screen rotation instruction through a remote controller.

[0143] In some embodiments, during rotation of the display, a rotation animation is displayed and the global menu control and the global back control are not displayed.

[0144] After receiving the user's rotation instruction, the drag coordinates of the global menu control and the global return control are recorded in real time, which helps to determine the actual coordinates of the global menu control and the global return control before rotation.

[0145] In some embodiments, the dragging coordinates of the global menu control and the global return control can be the coordinates of the center point of the global menu control and the global return control, or the coordinates of a point on the edge of the global menu control and the global return control, or the coordinates of any pre-set point in the global menu control and the global return control.

[0146] Calculate the scale factor based on the state of the screen before rotation and the drag coordinates;

[0147] The calculation method of the scale coefficient: scaleX = moveX / wide1; scaleY = moveY / high1;

[0148] Among them, scaleX and scaleY are scale factors, moveX and moveY are the drag coordinates of the global menu control and the global return control, wide1 is the screen pixel width before rotation, and high1 is the screen pixel height before rotation.

[0149] Calculate the display coordinates according to the scale factor and the state of the screen after rotation;

[0150] In some embodiments, the display coordinates are calculated as follows:

[0151] showX=scaleX / wide2; showY=scaleY / high2;

[0152] Among them, scaleX and scaleY are scale coefficients, showX and showY are display coordinates of the global menu control and the global return control, wide2 is the pixel width of the screen after rotation, and high2 is the pixel height of the screen after rotation. ;

[0153] For example, taking a display device with a resolution of 1920*1080 as an example, if the screen is in a vertical state before rotation, the calculation method of the scale factor is: scaleX=moveX / 1080; scaleY=moveY / 1920; the screen is in a horizontal state after rotation, and the calculation method of the display coordinates is: showX=scaleX / 1920; showY=scaleY / 1080;

[0154] If the screen is in landscape mode before rotation, the proportional coefficient is calculated as follows: scaleX=moveX / 1920; scaleY=moveY / 1080; if the screen is in portrait mode after rotation, the display coordinates are calculated as follows: showX=scaleX / 1080; showY=scaleY / 1920.

[0155] After the rotation is completed, the global menu control and the global return control are displayed at the positions corresponding to the display coordinates;

[0156] In some embodiments, after the rotation is completed, the updateViewLayout (update interface layout) method is called to update the positions of the global menu control and the global return control.

[0157] In some embodiments, Figure 6 In the example, when the user inputs a rotation command, the rotating component drives the display to rotate, and displays a rotation animation during the rotation process. After the rotation is completed, the following is displayed: Fig.15 The user interface shown.

[0158] In some embodiments, the user presses the global menu control or the global return control with a finger and moves the finger on the touch screen to drive the global menu control and the global return control to move on the screen. The user can select the global menu control and the global return control with a finger and drag the global menu control and the global return control to any position on the screen.

[0159] In some embodiments, it can be determined by comparing the distance the user's finger moves on the screen with the preset pixels to determine whether the user is performing a click operation or a drag operation on the global menu control and the global return control. For example, when the distance the user's finger moves on the screen is greater than 4px, it can be determined that the user is performing a drag operation, and the global menu control and the global return control are moved to the position where the finger stops. At this time, the user can keep the finger on the screen to continue moving the global menu control and the global return control to the next required position. The user can also leave the screen with the finger, and the global menu control and the global return control will no longer move. When the distance the user's finger moves on the screen is not greater than 4px, it can be determined that the user is performing a click operation. If the user clicks on the global menu control, the operation of displaying the menu display area is executed; if the user clicks on the global return control, the instruction to return to the previous operation is executed.

[0160] The embodiment of the present application makes full use of the touch characteristics of the display device and supports a global drag function. The global menu control and the global return control can be dragged to any position on the display screen, which is easy to operate and improves the user experience.

[0161] In some embodiments, the user touches the touch screen with at least four fingers, so that the global menu control and the global return control are displayed on the center point coordinates of the user's fingers, thereby achieving multi-finger positioning.

[0162] In some embodiments, the step of multi-finger positioning includes:

[0163] Intercept MotionEvent (touch event) based on InputFilter (input filtering) mechanism;

[0164] By intercepting touch events, you can get action_down (down operation) and action_up (up operation) multi-finger touch events.

[0165] Customize GestureDetectManager (gesture management class) based on touch events.

[0166] Among them, the custom gesture management class includes:

[0167] Record a complete action_down (downward operation) and action_up (upward operation) multi-finger touch event, and record each click coordinate in the form of an array, that is, the touch coordinates of each touch point;

[0168] Determine the maximum and minimum coordinates of all touch coordinates;

[0169] Calculate the Cartesian coordinate distance between the maximum and minimum coordinate values;

[0170] If the Cartesian coordinate distance is greater than the preset pixel, calculate the center point coordinates of all touch coordinates;

[0171] Display the global menu control and the global return control at the position corresponding to the center point coordinates.

[0172] For example, taking five-finger positioning as an example, the coordinates of each click are recorded in the form of an array, as shown below:

[0173] pos=0, X0=623.98535, Y0=388.89227;

[0174] pos=1, X1=778.3119, Y1=307.42264;

[0175] pos=2, X2=741.2829, Y2=381.04852;

[0176] pos=3, X3=774.5621, Y3=270.57675;

[0177] pos=4, X4=737.88464, Y4=214.87946;

[0178] In some embodiments, the method for determining the maximum and minimum coordinates of all touch points is as follows:

[0179] Define initial values ​​minX = maxX = X0 and minY = maxY = Y0;

[0180] if(X1>X0), maxX=X1, minX=X0;

[0181] if(X1<X0), minX=X1, maxX=X0;

[0182] By analogy, we compare the X and Y coordinates of the five points and get (minX, minY) and (maxX, maxY), which are (623.98535, 214.87946) and (778.3119, 388.89227).

[0183] Calculate the Cartesian coordinate distance between the above two points. The Cartesian coordinate distance calculation result is d;

[0184] In some embodiments, the number of touch points is 5 and the preset pixel is 300px. If the Cartesian coordinate distance d between the above two points is not greater than 300, it is determined that it is not a valid five-finger operation; if the Cartesian coordinate distance d between the above two points is greater than 300, it can be determined that it is a valid five-finger operation.

[0185] When the gesture meets the established algorithm, that is, it is determined to be a valid five-finger operation, the coordinates of the center point are calculated;

[0186] In some embodiments, the center point coordinates are the average values ​​of the x and y coordinates of the five touch points. The center point coordinates of this time are: centerX=731.2054, centerY=312.56396;

[0187] Synchronously update the display positions of the global menu control and the global return control according to the centerX and centerY coordinates.

[0188] In some embodiments, Figure 7 As shown in the figure, the current global menu control and the global return control are displayed at the lower left of the display screen. When the user touches the lower right of the display screen with five fingers, as shown in the figure, Fig.16 As shown, the global menu control and the global return control are displayed at the lower right side of the display screen.

[0189] The embodiment of the present application makes full use of the touch characteristics of the display device and can directly change the display position of the global menu control and the global return control on the display screen by five-finger positioning. There is no need to drag the global menu control and the global return control, which is easy to operate and improves the user experience.

[0190] In some embodiments, after the user clicks the global menu control, the display displays a menu display area, the menu display area includes a menu editing control, and the menu display area may also include at least one application or tool control. If the global return control is not displayed together with the global menu control, the menu display area may also include a global return control.

[0191] In some embodiments, Fig.10 As shown, the global menu control 61 can be displayed with an expansion arrow, and the global return control 62 can be displayed with a return arrow; when the user clicks the global menu control 61, if no tool or application is currently added, such as Fig.17 As shown, the menu display area 63 includes an add tool / application control 64 and a menu collapse control 66. When the user clicks the add tool / application control 64, the menu editing interface can be entered. Fig. 22When the user clicks the menu collapse control 66, the menu display area 63 can be collapsed, as shown in FIG. Fig.10 If a tool or application has been added, such as Fig.18 As shown, the menu display area 63 includes at least one application or tool control, a menu collapse control 66, and a menu editing control 65. When the user clicks the menu editing control 65, the menu editing interface can be entered, such as Fig. 22 When too many applications or tool controls are added to the menu display area 63, the display controls of the menu display area 63 can be changed by sliding the finger left and right on the display screen. The position of the menu retracting control 66 is always displayed in the menu display area 63; when the user slides the finger to the right, the menu display area 63 is as shown in FIG. Fig.19 As shown. Fig.19 In the example, when the user clicks the volume control 67, Fig. 20 As shown, a volume adjustment bar 68 is displayed in the menu display area 63, and the user can change the volume by sliding his finger left and right on the display screen.

[0192] In some embodiments, Figure 17-Figure 19 If the user does not perform any operation within the defined time, the menu display area 63 will be automatically closed. Fig. 20 If the user does not perform any operation within the defined time, the volume adjustment bar 68 will automatically retract. Fig.19 shown.

[0193] In some embodiments, the user clicks on the menu editing control, and the display shows the menu editing area;

[0194] In the menu editing area, the user can add an application or tool control to the menu display area, or delete an application or tool control in the menu display area by clicking the application or tool control with a finger.

[0195] In some embodiments, Fig.21 In the user interface shown, the user can select the menu display area 63 with a finger and drag the menu display area 63 to any position on the screen. To ensure that the content of the menu display area is displayed completely, when the edge of the menu display area 63 coincides with the edge of the screen, the user cannot continue to drag in the original direction, and a prompt message can be displayed when it cannot be moved. The user can also display the menu display area at the center coordinate point of the finger touching the screen by multi-finger positioning. In some embodiments, when the global menu control 61 is dragged to the rightmost end of the screen, the menu display area 63 can be adaptively moved to the left when the global menu control 61 is clicked to fully display the content of the menu display area.

[0196] exist Fig.21 In the example, when the user clicks the menu edit control 65, Fig. 22As shown, the display device provides a user interface to the display, in which the menu editing area 71 is included. The menu editing area 71 includes a selected menu area 72, a menu area 73 to be selected, a reset control 74, and a completion control 75. The selected menu area 72 includes the application or tool control that has been selected to be displayed in the menu display area 63, and the menu area 73 to be selected includes the application or tool control that has not been selected to be displayed in the menu display area. When the user clicks the reset control 74, the application or tool control in the selected menu area 72 is cleared; when the user clicks the completion control 75, the user returns to the Fig.21 The user interface shown in FIG. 6 shows that the application or tool control displayed in the menu display area 63 is the application or tool control in the menu area 72 that has been selected before clicking the completion control 75 .

[0197] like Fig.23 In the menu editing area shown in FIG. 1 , when the user clicks on the friends circle control 76 in the menu area 73 to be selected, as shown in FIG. Fig.24 As shown, the circle of friends control 76 moves from the menu area 73 to be selected to the selected menu area 72, wherein, by clicking on an application or tool control in the menu area 73 to be selected, the application or tool control can be added to the end of the selected menu area 72. When the number of application or tool controls in the selected menu area 72 reaches the preset number, if the user continues to add application or tool controls, a prompt message "up to 13 applications or tools can be added" will pop up.

[0198] exist Fig.24 In the example, when the user clicks the delete control 77 on the friend circle control 76, Fig.23 As shown, the circle of friends control 76 is moved from the selected menu area 72 to the menu area to be selected 73. In some embodiments, when the user deletes the application or tool control in the selected menu area 72, the application or tool control may be displayed first in the tool or application list.

[0199] In some embodiments, in the menu editing area, the user can add an application or a tool control to the menu display area, or delete an application or a tool control in the menu display area by dragging the application or the tool control with a finger.

[0200] In some embodiments, the step of adding an application or tool control to the menu display area, or deleting an application or tool control in the menu display area specifically includes:

[0201] Detect whether an action_down event occurs;

[0202] If an action_down event is detected, that is, the user's finger is detected touching the touch screen, pointToPosition (drop point position function) obtains the item coordinates of the drop position (project coordinates);

[0203] Determine whether the item coordinates are valid;

[0204] In some embodiments, the method for determining whether the item coordinates are valid is to determine whether there is an application or tool control for the item coordinates. If the item coordinates do not have any application or tool control, the item coordinates are invalid; if the item coordinates are within the range of an application or tool control, the item coordinates are valid;

[0205] If the item coordinates are invalid, execute the step of detecting whether an action_down event occurs;

[0206] If the item coordinates are valid, send a long press judgment message;

[0207] Determine whether the action_down event exceeds a first preset duration, that is, whether the user's finger touch duration exceeds the first preset duration; in some embodiments, the first preset duration is 800 milliseconds;

[0208] If the action_down event exceeds the first preset duration, the background image of the application or tool control corresponding to the current project coordinates is obtained, and a floating mirror image of the background image is drawn through the window manager;

[0209] Hide app or tool controls;

[0210] Detect action_move (movement operation) events and update the mirror position, that is, update the position of the floating mirror according to the position of the user's finger movement;

[0211] Detect whether an action_up event occurs;

[0212] If an action_up event is detected, that is, it is detected that the user's finger leaves the touch screen, the display position of the application or tool control is updated, and the floating image is hidden.

[0213] In some embodiments, the display position of the application or tool control may be updated to the position where the user's finger leaves the touch screen, or may be updated to a pre-set position, such as the end of the selected application or tool control.

[0214] If no action_up event is detected, the steps of detecting an action_move event and updating the mirror position are performed.

[0215] In some embodiments, the user can drag the application or tool control in the menu area 73 to be selected to the range of the selected menu area 72. When the user drags the application or tool control in the menu area 73 to be selected to the range of the selected menu area 72, the application or tool control can be added to the end of the selected menu area 72. Fig.23 In the menu editing area shown in FIG. 1 , when the user selects the friends circle control 76 in the menu area 73 to be selected, and Fig.25 The gesture dragging method shown is to drag the friend circle control 76 from the menu area to be selected 73 to the selected menu area 72, such as Fig.24 When the number of applications or tool controls in the selected menu area 72 reaches the preset number, if the user continues to add applications or tool controls, a prompt message "up to 13 applications or tools can be added" will pop up.

[0216] In some embodiments, the user can drag an application or tool control in the menu area 73 to be selected to a specified position in the selected menu area 72. If the specified position is between two selected applications or tool controls, the dragged application or tool control will be displayed at the position of the next selected application or tool control, and the next selected application or tool control will be moved to the next position, and the other applications or tool controls will be moved to the next position in sequence. If there is no application or tool control occupying the specified position, the dragged application or tool control can be directly added to the specified position or added to the end of the selected menu area 72. If there is already an application or tool control occupying the position, the dragged application or tool control will be added to the specified position; and the original application or tool control at the position can be directly moved to the first position in the tool or application list in the menu area 73 to be selected, so as to replace the positions of the two application or tool controls.

[0217] In some embodiments, Fig.26 In the example, when the user drags the annotation control 78 in the menu area 73 to be selected to the position of the homepage control 79 in the menu area 72, the position is already occupied by the homepage control 79. Fig. 27 As shown, the annotation control 78 will be added to the position; and the original homepage control 79 at the position can be moved to the next position in sequence, and other application or tool controls are moved to the next position in sequence. If the number of application or tool controls in the selected menu area 72 reaches the preset number, the application or tool control at the end of the original menu can be moved to the menu area 73 to be selected.

[0218] exist Fig.24 In the example, when the user selects the friend circle control 76 in the selected menu area 72 and drags the friend circle control 76 from the selected menu area 72 to the menu area to be selected 73, Fig.23 In some embodiments, when the user deletes an application or tool control in the selected menu area 72 , the application or tool control may be placed first in the list of tools or applications in the menu area 73 to be selected.

[0219] The embodiments of the present application make full use of the touch characteristics of the display device, can customize menu options, centrally control various functions, and improve user experience.

[0220] In the above embodiment, by setting and displaying the global menu control and the global return control, the touch characteristics of the display device are fully utilized to make up for the shortcomings of the remote control operation. The global menu control and the global return control also become the unified entrance to various functions of the smart TV, and each function is centrally controlled, so that users can quickly find the functions they want to use, thereby improving the user experience.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0222] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display, for displaying a user interface; User interface; A controller connected to the display and the user interface, respectively, for executing: In a case where the display displays a first interface, in response to receiving a first touch operation input by a user on the display, controlling the display to display a global menu control on an upper layer of the first interface; wherein the first interface includes a startup interface of the display device or an application interface of any application in the display device, and the global menu control is displayed suspended on an upper layer of the first interface; In response to a second touch operation of the user on the global menu control, controlling the display to display a menu display area on an upper layer of the first interface, wherein the menu display area includes at least one application control and / or at least one tool control; In response to a third touch operation of the user on the application control, the display is controlled to display content corresponding to the application control; or, in response to a fourth touch operation of the user on the tool control, the display is controlled to display content corresponding to the tool control.

2. The display device according to claim 1, characterized in that The controller is further configured to execute: In response to receiving a fifth touch operation of the user moving the global menu control to a target position, controlling the display to display the global menu control at the target position.

3. The display device according to claim 1, characterized in that The controller is used to execute: In response to receiving a sixth touch operation input by the user on the display, the position of the global menu control is synchronously updated according to the center point coordinates of the touch points, and the sixth touch operation includes at least four of the touch points.

4. The display device according to claim 1, characterized in that The controller is further configured to execute: In response to a seventh touch operation of the user on the menu editing control in the menu display area, controlling the display to display the menu editing area; In response to the user's eighth touch operation on the application control or tool control in the menu editing area, the application or tool control is added to the menu display area, or the application or tool control in the menu display area is deleted, and the eighth touch operation includes a click operation or a drag operation.

5. A display device, characterized in that: include: A display, for displaying a user interface; User interface; A controller connected to the display and the user interface, respectively, for executing: In a case where the display displays a first interface, in response to receiving a first touch operation input by a user on the display, controlling the display to display a global menu control and a global return control on an upper layer of the first interface, wherein the first interface includes a startup interface of the display device or an application interface of any application in the display device, and the global menu control is displayed suspended on an upper layer of the first interface; In response to a second touch operation of the user on the global menu control, controlling the display to display a menu display area on an upper layer of the first interface, wherein the menu display area includes at least one application control and / or at least one tool control; In response to a third touch operation of the user on the application control, the display is controlled to display content corresponding to the application control; or, in response to a fourth touch operation of the user on the tool control, the display is controlled to display content corresponding to the tool control.

6. The display device according to claim 5, characterized in that The controller is further configured to execute: In response to receiving a fifth touch operation of the user moving the global menu control and the global return control to a target position, the display is controlled to display the global menu control and the global return control at the target position.

7. The display device according to claim 5, characterized in that The controller is further configured to execute: in response to receiving a sixth touch operation input by a user on the display, the sixth touch operation comprising at least four touch points, acquiring touch coordinates of the touch points; Determine the maximum value and the minimum value of the touch coordinates; Calculate the Cartesian coordinate distance between the maximum value of the coordinate and the minimum value of the coordinate; If the Cartesian coordinate distance is greater than the preset pixel, calculating the center point coordinates of the touch point; The global menu control and the global return control are displayed at positions corresponding to the center point coordinates.

8. The display device according to claim 5, characterized in that Also includes: A rotating assembly connected to the display; the controller is further used to execute: In response to a screen rotation instruction input by a user, controlling the rotating component to drive the display to rotate; Recording the drag coordinates of the global menu control and the global return control; Calculate a scale factor according to the state of the screen before rotation and the drag coordinates; Calculate display coordinates according to the scale factor and the state of the screen after rotation; After the rotation is completed, the global menu control and the global return control are displayed at the positions corresponding to the display coordinates.

9. The display device according to claim 5, characterized in that The controller is further configured to execute: In response to a seventh touch operation of the user on the menu editing control in the menu display area, controlling the display to display the menu editing area; If it is detected that the user's finger touches the display, the item coordinates of the point where the user's finger falls are obtained; If the item coordinates are the coordinates of the application control or tool control in the menu editing area, and the user's finger touch duration exceeds a first preset duration, obtaining a background image of the application control or tool control in the menu editing area, and drawing a floating mirror image of the background image; Hide the application controls or tool controls in the menu editing area; Update the position of the suspended image according to the position of the user's finger movement; If it is detected that the user's finger leaves the display, the display position of the application control or tool control in the menu editing area is updated according to the position where the user's finger leaves, and the floating mirror image is hidden.