Display device and floating window scaling method

Abstract

The embodiment of the invention provides a display device and a floating window scaling method. In response to a floating window display instruction, a display device displays a floating window and sets a window scaling area. Obtaining a starting touch point and an ending touch point of the touch action; and obtaining an instruction type corresponding to the touch action according to the initial touch point. And if the instruction type is a floating window scaling instruction, executing scaling operation on the floating window according to the starting touch point and the ending touch point. In this way, a user can adjust the size of the floating window through the window scaling area, the display device can accurately recognize the user instruction, and the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a floating window scaling method. Background Art

[0002] A display device refers to a terminal device that can output specific display images. With the rapid development of display devices, the functions of display devices will become more and more abundant, and the performance will become more and more powerful. It can realize two-way human-computer interaction functions, integrate multiple functions such as audio and video, entertainment, and data, and is used to meet the diverse and personalized needs of users.

[0003] The display device can have an application floating window function and a touch screen function. Among them, the touch screen function means that the user can directly touch the display to control the display device, such as clicking on controls, adjusting various display parameters, etc. The application floating window function means that the display interface of the application is displayed in the form of a floating window, so as to break through the limitation that the original technology can only display in full screen. Considering the user's usage needs, the size of the floating window can also be adjusted.

[0004] When adjusting the size of the floating window, if the user uses the touch screen function, they can press and hold the border line of the floating window and drag it to the desired size. However, the user may not be able to accurately touch the border line of the floating window, resulting in the display device not responding to the user's instructions, causing the floating window size adjustment to fail, giving the user a poor user experience. Summary of the invention

[0005] The present application provides a display device and a floating window scaling method to solve the problem in the related art that a user cannot accurately touch the boundary line of a floating window, resulting in failure to adjust the size of the floating window, giving the user a poor user experience.

[0006] In a first aspect, some embodiments of the present application provide a display device, including a display, a touch component, and a controller. The display is configured to display a user interface, wherein the user interface includes an application program interface; the touch component is configured to collect touch actions input by a user; and the controller is configured to:

[0007] In response to a floating window display instruction, controlling the display to display a floating window; the floating window is used to display the application interface;

[0008] Setting a window scaling area within a preset width of a border around the floating window;

[0009] Obtaining a starting touch point and an ending touch point of the touch action;

[0010] Acquiring the instruction type corresponding to the touch action according to the starting touch point; the instruction type includes a floating window zoom instruction and an application control instruction, the floating window zoom instruction is used to instruct to change the size of the floating window, and the application control instruction is used to instruct to control the display content in the floating window; wherein, when the starting touch point is located in the window zoom area, the instruction type is a floating window zoom instruction;

[0011] If the instruction type is a floating window zoom instruction, a zoom operation is performed on the floating window according to the starting touch point and the ending touch point.

[0012] In a second aspect, some embodiments of the present application provide a floating window scaling method, which is applied to a display device, including:

[0013] In response to a floating window display instruction, controlling the display to display a floating window; the floating window is used to display the application interface;

[0014] Setting a window scaling area within a preset width of a border around the floating window;

[0015] Get the starting touch point and the ending touch point of the touch action;

[0016] Acquiring the instruction type corresponding to the touch action according to the starting touch point; the instruction type includes a floating window zoom instruction and an application control instruction, the floating window zoom instruction is used to instruct to change the size of the floating window, and the application control instruction is used to instruct to control the display content in the floating window; wherein, when the starting touch point is located in the window zoom area, the instruction type is a floating window zoom instruction;

[0017] If the instruction type is a floating window zoom instruction, a zoom operation is performed on the floating window according to the starting touch point and the ending touch point.

[0018] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a floating window scaling method. In response to a floating window display instruction, the display device displays a floating window and sets a window scaling area. The starting touch point and the ending touch point of the touch action are obtained; the instruction type corresponding to the touch action is obtained according to the starting touch point. If the instruction type is a floating window scaling instruction, the scaling operation is performed on the floating window according to the starting touch point and the ending touch point. The user can adjust the size of the floating window through the window scaling area, and the display device can accurately recognize the user's instructions, thereby improving the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 shows a usage scenario of a display device according to some embodiments;

[0021] Figure 2 shows a hardware configuration block diagram of a control device 100 according to some embodiments;

[0022] Figure 3 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0023] Figure 4 shows a software configuration diagram in a display device 200 according to some embodiments;

[0024] Figure 5 A schematic diagram of a user interface in some embodiments is shown;

[0025] Figure 6 A schematic diagram showing application panels in some embodiments;

[0026] Figure 7 A schematic diagram showing a display device displaying a playback interface in some embodiments is shown;

[0027] Figure 8 A schematic diagram showing a floating window displayed by a display device in some embodiments is shown;

[0028] Fig. 9 A schematic diagram showing a display device displaying floating windows of multiple applications in some embodiments is shown;

[0029] Fig.10 A schematic diagram of a floating window size adjustment interface in some embodiments is shown;

[0030] Fig.11 A schematic diagram showing a user zooming in or out of a floating window in some embodiments;

[0031] Fig.12 A schematic diagram showing confirmation information of a floating window adaptive adjustment mode in some embodiments is shown;

[0032] Fig.13 A flowchart of the interaction between components of a display device in some embodiments is shown;

[0033] Fig.14 A schematic diagram showing a floating window and a window zooming area in some embodiments is shown;

[0034] Fig.15 A schematic diagram showing critical regions in some embodiments;

[0035] Fig.16 A schematic diagram showing a floating window moving area in some embodiments. DETAILED DESCRIPTION

[0036] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0037] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0038] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0039] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0040] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is a specific implementation of the display device of the present application.

[0041] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0042] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0043] In some embodiments, a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0044] In some embodiments, the display device may not use the above-mentioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0045] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0046] In some embodiments, the display device 200 also communicates data with the server 400. It can be set to allow the display device 200 to communicate through a local area network (LAN), a wireless local area network (WLAN) and other networks. The server 400 can provide various content and interactions to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0047] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0048] like Figure 3 The display device 200 includes at least one of a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0049] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0050] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0051] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0052] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0053] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0054] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0055] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0056] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0057] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0058] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0059] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory, RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0060] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0061] "User interface" is the medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is the Graphical User Interface (GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0062] like Figure 4 As shown, the system of the display device is divided into three layers, namely, the application layer, the middleware layer and the hardware layer from top to bottom.

[0063] The application layer mainly includes commonly used applications on TV and application frameworks. Common applications are mainly applications developed based on browsers, such as HTML5 APPs and native APPs.

[0064] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0065] Native apps can support online or offline, message push or local resource access.

[0066] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0067] The hardware layer mainly includes HAL interface, hardware and driver. Among them, HAL interface is the unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0068] Based on the above display device 200, the display device 200 can support touch interaction function by adding a touch component 276. Generally, the touch component 276 can form a touch screen together with the display 260. On the touch screen, the user can input different control instructions through touch operations. For example, the user can input touch instructions such as click, slide, long press, double click, etc. Different touch instructions can represent different control functions.

[0069] In order to realize the above-mentioned different touch actions, the touch component 276 can generate different electrical signals when the user inputs different touch actions, and send the generated electrical signals to the controller 250. The controller 250 can extract features from the received electrical signals, thereby determining the control function to be executed by the user according to the extracted features.

[0070] For example, when a user inputs a click touch action at any program icon position in the application interface, the touch component 276 will sense the touch action and generate an electrical signal. After receiving the electrical signal, the controller 250 can first determine the duration of the corresponding level of the touch action in the electrical signal, and when the duration is less than the preset time threshold, it recognizes that the user input is a click touch instruction. The controller 250 then extracts the position feature generated by the electrical signal to determine the touch position. When the touch position is within the display range of the application icon, it is determined that the user has input a click touch instruction at the application icon position. Accordingly, the click touch instruction is used to execute the function of running the corresponding application in the current scenario, so the controller 250 can start running the corresponding application.

[0071] For another example, when the user inputs a sliding action in the media display page, the touch component 276 also sends the sensed electrical signal to the controller 250. The controller 250 first determines the duration of the signal corresponding to the touch action in the electrical signal. When it is determined that the duration is greater than the preset time threshold, the position change of the signal generation is judged. Obviously, for interactive touch actions, the position where the signal is generated will change, thereby determining that the user has input a sliding touch command. The controller 250 then determines the sliding direction of the sliding touch command based on the change in the signal generation position, and controls the page turning of the display screen in the media display page to display more media options. Furthermore, the controller 250 can also extract the sliding speed, sliding distance and other features of the sliding touch command, and control the page turning screen according to the extracted features to achieve a hand-following effect, etc.

[0072] Similarly, for touch commands such as double-click and long press, the controller 250 can extract different features, determine the type of touch command through feature judgment, and then execute the corresponding control function according to the preset interaction rules. In some embodiments, the touch component 276 also supports multi-touch, so that the user can input touch actions on the touch screen through multiple fingers, such as multi-finger click, multi-finger long press, multi-finger slide, etc.

[0073] The display device 200 may have multiple functions, such as browsing web pages, playing media resources, entertainment games, screen projection, etc., so as to provide users with a variety of services. The user may control the display device 200 to start a related application program, thereby opening the corresponding function.

[0074] In some embodiments, when the user controls the display device to turn on, the controller 250 can control the display 260 to display a user interface. The user interface can be a specific target image, such as various media resources obtained from a network signal source, including videos, pictures and other content. The user interface can also be some UI interfaces of the display device. Figure 5Schematic diagrams of user interfaces in some embodiments are shown. Figure 5 As shown, the user interface includes a first navigation bar 500, a second navigation bar 510, a function bar 520 and a content display area 530, and the function bar 520 includes multiple function controls such as "watch history", "my favorites" and "my apps". Among them, the content displayed in the content display area 530 will change with the changes of the selected controls in the first navigation bar 500 and the second navigation bar 510. When on the application panel page, the user can trigger entry into the corresponding application panel by clicking on the "my apps" control to input a display instruction for the application panel page. It should be noted that the user can also input the selection operation of the function control in other ways to trigger entry into the application panel. For example, use voice control function or search function to control entry into the application panel page.

[0075] The user can view the applications installed on the display device through the application panel, that is, the functions supported by the display device. The user can select one of the applications to enable the display device to start the application, thereby realizing the function of the application. It should be noted that the application installed on the display device can be a system application or a third-party application. By starting an application, the user controls the display device to realize the corresponding function of the application. Figure 6 Schematic diagrams of application panels in some embodiments are shown. Figure 6 As shown, the application panel includes three controls: "browser", "network media" and "video chat". Among them, the user can control the display device to open the browser application by clicking the "browser" control. The user can perform corresponding operations in the browser, such as searching for media. The user can click the "network media" control to use the display device to watch some network media channels, including various media resources provided on the network. The user can click the "video chat" control to use the display device for video chat.

[0076] In some embodiments, when a display device starts an application, it can be set by default to display the display interface corresponding to the application in full screen. For example, when a user controls the display device to start a browser or a network media application, a certain media can be searched for viewing. The display device can display the playback interface corresponding to the media in full screen. The playback interface generally refers to a user interface that can present specific screen content on a display, which can be the specific screen content of the current target media being played.

[0077] Figure 7 Schematic diagram showing a display device displaying a playback interface in some embodiments. Figure 7As shown, the playback interface may include a media asset screen and UI controls related to the playback process, such as a pause / play button, a fast forward / rewind button, a volume adjustment button, a full screen / window button, a setting button, and a playback progress control, etc. The current playback progress of the media asset and the total playback time of the media asset may also be displayed.

[0078] In some embodiments, the user can use a control device, such as a remote controller, a mobile phone, and a mobile terminal, to control the display device. Figure 7 Taking the playback interface shown in the figure as an example, the user can use the control device to adjust the playback parameters of the media, such as the playback progress. The user can use the direction keys on the control device to control the focus cursor to select the fast forward / fast rewind button, and after pressing the "Confirm / OK" key, control the fast forward of the playback progress of the current media.

[0079] Considering the user experience, the display device also has a touch screen function, that is, a touch interaction function, which allows users to interact with the display device by touch. Users can operate the host by simply touching the display with their fingers, thus getting rid of the keyboard, mouse, and remote control operation, making human-computer interaction more straightforward.

[0080] In some embodiments, the display device may have a floating window function. The application program may be displayed on the display in the form of a floating window. The user may instruct the display device to switch the current full-screen displayed application program to the form of a floating window. Figure 8 Schematic diagram showing a floating window displayed by a display device in some embodiments. Figure 8 As shown, for Figure 7 The playback interface can be displayed through a floating window.

[0081] In some embodiments, the display device can display multiple applications at the same time, and multiple applications can be displayed on the screen in the form of floating windows, so that multiple applications can share the screen, allowing users to interact with multiple applications at the same time under the screen display of the display device 200, thereby broadening the user's usage scope and improving the user experience. Fig. 9 FIG. 1 is a schematic diagram showing a display device displaying floating windows of multiple applications in some embodiments. Fig. 9 As shown, the left side of the user interface displayed by the display device 200 displays floating windows corresponding to application A and application B, so that the user can interact with application A and application B.

[0082] In some embodiments, the user can adjust the size of the floating window taking into account the user's viewing needs. Fig.10 Schematic diagrams of floating window size adjustment interfaces in some embodiments are shown. Fig.10As shown, the user can call up the floating window size adjustment interface of the display device to adjust the size of the floating window. The size of the floating window can be 100%, 80%, 50%, 25% of the display size, or the user can set the size of the floating window by himself, for example, the user can enter the vertical height and horizontal width information of the floating window.

[0083] The user can also use the touch screen function of the display device to adjust the size of the floating window. For the floating window, the user can adjust the window size based on the needs, including zooming in and out. The user can control the display device 200 to perform the corresponding window adjustment operation through specific interactive actions. Fig.11 Schematic diagram of a user zooming in and out of a floating window in some embodiments is shown. Fig.11 As shown, the user can press and keep the finger in contact with the top corner of the floating window on the display, and then move the finger outside or inside the floating window while keeping the finger in contact with the display to enlarge or reduce the floating window.

[0084] It can also be set so that the user presses the border line of the floating window, keeps the finger in contact with the display, and moves to the outside or inside of the floating window to adjust the size of the floating window.

[0085] It should be noted that the border line of the floating window may be narrow, and its width may be only a few pixels in size. When the user adjusts the size of the floating window by touch, he may not be able to accurately touch the border line of the floating window, but directly touch the inside or outside of the floating window. If the inside of the floating window is touched, the display device will determine that the user is controlling the content displayed inside the floating window, that is, the display content of the application. If the outside of the floating window is touched, the display device will determine that the user is controlling content outside the application, such as controlling the system main interface or other applications. At this time, the display device will recognize the user's touch as an instruction other than the instruction to adjust the size of the floating window, causing the adjustment of the size of the floating window to fail, giving the user a poor user experience.

[0086] To solve the above problems, the display device provided in the embodiment of the present application may have a floating window adaptive adjustment function, and may recognize the user's instruction to adjust the size of the floating window. The display device may be provided with a floating window adaptive adjustment mode. When detecting the user's touch action, the display device may recognize whether the user needs to adjust the size of the floating window, and adaptively perform a zoom operation on the floating window. This avoids the failure of instruction recognition when the user needs to adjust the size of the floating window, thereby improving the user's experience.

[0087] In some embodiments, the user can send a floating window adaptive adjustment mode instruction to the display device by operating a designated button of the remote control. In the actual application process, the correspondence between the floating window adaptive adjustment mode instruction and the remote control button is pre-bound. For example, a floating window adaptive adjustment mode button is set on the remote control. When the user touches the button, the remote control sends the floating window adaptive adjustment mode instruction to the controller, and the controller controls the display device to enter the floating window adaptive adjustment mode.

[0088] You can also set a floating window adaptive adjustment mode option in the system UI interface of the display device. When the user clicks on the option, the display device can be controlled to enter or exit the floating window adaptive adjustment mode.

[0089] In some embodiments, to prevent the user from accidentally triggering the floating window adaptive adjustment mode, when the controller receives the floating window adaptive adjustment mode instruction, it can control the display to display the floating window adaptive adjustment mode confirmation information, so that the user can make a second confirmation whether to control the display device to enter the floating window adaptive adjustment mode. Fig.12 A schematic diagram showing confirmation information of a floating window adaptive adjustment mode in some embodiments is shown.

[0090] Fig.13 A flowchart of the interaction between components of a display device in some embodiments is shown, including the following steps:

[0091] S101. In response to a floating window display instruction, control a display to display a floating window, where the floating window is used to display the application program interface.

[0092] S102, setting a window scaling area within a preset width of a border around the floating window;

[0093] S103: Acquire a starting touch point and an ending touch point of the touch action.

[0094] S104: Acquire an instruction type corresponding to the touch action according to the initial touch point.

[0095] S105: If the instruction type is a floating window zoom instruction, perform a zoom operation on the floating window according to the starting touch point and the ending touch point.

[0096] In some embodiments, the user may instruct to display the application in the form of a floating window, and may input a floating window display instruction to the display device. In response to the instruction, the controller may control the display to display a floating window, and the floating window is used to display the application interface.

[0097] The controller can detect whether the FreeForm function of the display device is turned on. The FreeForm function refers to the split-screen display function in the display device system. If it is not turned on, turn on this function. When the FreeForm function is turned on and available, the controller can obtain the window size and window coordinates of the floating window, and build a floating window (ActivityOption) according to the window size and window coordinates. After the floating window is built, the controller can control the display to display the floating window. Among them, the window size and window coordinates can use the default parameters set in the display device, such as the parameters set by the user last time. It can also be the parameters entered by the user when entering the floating window display instruction. The controller can obtain the window size and window coordinates from the floating window display instruction.

[0098] When the floating window is displayed, the floating window can be in a focused state to prompt the user that the floating window has been constructed. In the embodiment of the present application, the focused state refers to the highlighted display of the area, such as a highlighted display. After a preset time, such as 15 seconds, if no user instruction has been received during this period of time, the focused state of the floating window can be canceled.

[0099] After constructing the floating window, in order to enable the user to more conveniently adjust the size of the floating window, the controller can set a window scaling area for the floating window. In the embodiment of the present application, the window scaling area refers to an area that can be used to adjust the size of the floating window. The user can touch the window scaling area and drag it to adjust the size of the floating window. The window scaling area can be set within a preset width of the border around the floating window.

[0100] The window scaling area may be an area surrounding the floating window. Fig.14 Schematic diagrams of floating windows and window scaling areas in some embodiments are shown. Fig.14 As shown, area A is the floating window, and area Z is the window scaling area. The window scaling area may be a closed area around the outline of the floating window.

[0101] When setting the window scaling area, the size of the window scaling area can be set first. The extended pixel value can be set first, for example, a preset pixel value is used. The extended pixel value is used to characterize the width of the window scaling area. The horizontal width and vertical width of the window scaling area can be the same or different, which is not limited in the embodiment of the present application. According to the extended pixel value, the controller can generate a window scaling area around the floating window.

[0102] In some embodiments, the user can control the display device by touch, including controlling the display content in the floating window, adjusting the size of the floating window, and controlling other display content outside the floating window. According to different touch actions of the user, the controller can generate and execute corresponding control instructions.

[0103] The controller can control the touch component to collect the touch actions input by the user. According to different touch actions, the controller will recognize different instructions to achieve different functions. The user can input various touch actions through the touch component, and the touch component forms touch instructions by detecting the input touch actions. Depending on the different touch actions input, the formed touch instructions are also different. Usually, touch instructions can include multiple types. For example, single-click touch instructions, multiple-click touch instructions, long-press touch instructions, sliding touch instructions, etc. Multi-finger touch instructions can also be further generated according to the number of corresponding touch points in the touch action, such as multi-finger click instructions, multi-finger sliding instructions, multi-finger long press instructions, etc.

[0104] The touch instructions generated by the touch component can be obtained by the controller to perform corresponding interactive control actions according to the rules set in the operating system. Different touch instructions can correspond to different control actions. For example, a single-click touch instruction on a media playback link control can be used to open the playback link to obtain media data; a long-press touch instruction on a function icon can be used to indicate entering the editing state, so that the position of the function icon can be adjusted by sliding the touch instruction in the editing state.

[0105] The controller can obtain the starting touch point and the ending touch point of the user's touch action, and thereby determine whether the size of the floating window needs to be adjusted according to the starting touch point and the ending touch point, that is, identify whether the user's control instruction is a floating window scaling instruction. Among them, the floating window scaling instruction is used to indicate the change of the size of the floating window. The interaction between the starting touch point and the ending touch point constitutes the user's touch action. The starting touch point can be the touch point where the user's finger first touches the display, and the ending touch point can be the touch point when the user's finger leaves the display.

[0106] For display devices that support touch functions, users can press the screen of the display with their fingers. The controller can determine the user's touch event based on the touch action collected by the touch component. After the user touches the display, the display device will continue to track the position of the user's finger. When the user's finger leaves the display screen, the touch action ends. During the entire interaction, the display device system distributes the MotionEvent object to the onTouchEvent() function, including relevant information for each interaction. For example, "MotionEvent.ACTION_DOWN" means that the first touch point is pressed, "MotionEvent.ACTION_UP" means that the last touch point is left, and "MotionEvent.ACTION_MOVE" means that the touch point moves. "MotionEvent.ACTION_POINTER_DOWN" means that other touch points are pressed, and "MotionEvent.ACTION_POINTER_UP" means that other touch points are left. The display device determines whether a specific interaction action occurs by using the relevant information provided by the MotionEvent object.

[0107] In some embodiments, in response to the touch action captured by the touch component, the controller can obtain the position information of the user's starting touch point. When the controller detects a touch press event (MotionEvent.ACTION_DOWN), it determines that the user starts touching the display, and can record the starting coordinates of the starting touch point of the current touch Point (x, y) = event.getX(), event.getY().

[0108] If the starting touch point is in the floating window, it means that the user wants to control the floating window. At this time, the floating window can be set to a focused state, that is, the floating window is highlighted to prompt the user that the floating window can be controlled. At the same time, the controller can continue to detect the user's touch behavior and obtain the position information of the end touch point of the touch action.

[0109] If the starting touch point is in the window zoom area, it means that the user wants to adjust the size of the floating window. At this time, the controller can obtain the number of starting touch points. When the number of starting touch points is a preset value, the window zoom area can be set to a focused state, that is, the window zoom area is highlighted to prompt the user that the current touch action can adjust the size of the floating window. The preset value can be 1. The controller can continue to detect the user's touch behavior and obtain the position information of the end touch point of the touch action. When the number of touch points is greater than the preset value, it means that the user currently has multiple fingers touching the display. At this time, the controller will not execute the acquisition of the position information of the end touch point of the touch action, that is, the controller will not adjust the size of the floating window according to the user's touch action.

[0110] If the initial touch point is neither in the floating window nor in the window zoom area, it means that the user wants to control the content outside the floating window. The controller can generate corresponding instructions for controlling other content according to the user's touch action, which is not specifically limited here.

[0111] In some embodiments, if the initial touch point is located in the floating window or the window zoom area, it means that the user wants to control the floating window. At this time, the controller can obtain the action type of the touch action. The various types may include sliding action, long press action, click action, continuous click action, etc. Among them, the sliding action refers to the action of the user continuously touching the display with a finger and sliding, which will leave a touch track on the display.

[0112] If the action type is a sliding action, it means that the user may have the need to scale the floating window. At this time, the user may instruct to scale the floating window, or may instruct to control other content (for example, sliding to adjust parameters such as volume and brightness). Therefore, the controller can obtain the instruction type corresponding to the touch action according to the starting touch point to determine whether the size of the floating window needs to be adjusted.

[0113] If the action type is not a sliding action, it is considered that the user has no need to scale the floating window. However, at this time, the user may instruct to control the display content in the floating window, or may instruct to control the display content outside the floating window. Therefore, the controller can detect the position information of the termination touch point. If the termination touch point is located in the floating window, it means that the user wants to control the display content in the floating window, and the controller can generate an application control instruction according to the touch action and execute the application control instruction. Among them, the application control instruction is used to instruct to control the display content in the floating window. If the termination touch point is not located in the floating window, the controller can also generate corresponding control instructions to control the display content outside the floating window that the user needs to control.

[0114] In some embodiments, the controller may obtain the instruction type corresponding to the touch action according to the starting touch point. The instruction types include floating window scaling instructions and application control instructions. The floating window scaling instructions are used to instruct to change the size of the floating window, and the application control instructions are used to instruct to control the display content in the floating window. It should be noted that when the starting touch point is located in the window scaling area, the instruction type is the floating window scaling instruction.

[0115] The controller can detect the position information of the initial touch point.

[0116] If the starting touch point is located in the window scaling area, it means that the user wants to adjust the size of the floating window, and the command type can be directly marked as a floating window scaling command.

[0117] If the initial touch point is located in the floating window, the controller may detect area change information of the touch action, and obtain the instruction type corresponding to the touch action according to the area change information.

[0118] In some embodiments, considering that there may be certain errors in touch point recognition, when the user wants to touch the window zoom area, the finger is closer to the boundary of the window zoom area, resulting in the starting touch point being located just at the boundary of the floating window, that is, the junction of the floating window and the window zoom area. Even more, the user's finger may touch the floating window and the window zoom area at the same time, but the starting touch point is recognized in the floating window. In order to avoid failure in user command recognition, a critical area can be set in the floating window, and the critical area can be a preset boundary area in the floating window. The critical area is an area of ​​preset width within the boundary of the floating window. The critical area can play the same function as the window zoom area, that is, the user can touch the critical area and slide to adjust the size of the floating window. Fig.15 Schematic diagrams of critical regions in some embodiments are shown. Fig.15 As shown, region L is the critical region. The critical region may be the outermost region in the floating window.

[0119] If the initial touch point is located in the floating window, the controller may detect whether the initial touch point is located in the critical area.

[0120] If the initial touch point is located in the critical area, it means that the user may have a need to adjust the size of the floating window. At this time, the controller can detect the area change information of the touch action.

[0121] If the initial touch point is not located in the critical area, the user does not need to adjust the size of the floating window, but wants to control the display content in the floating window, such as adjusting the volume, brightness, playback progress and other parameters. The controller can mark the instruction type as an application control instruction. At the same time, the controller can generate an application control instruction according to the touch action and execute the application control instruction.

[0122] In some embodiments, the starting touch point is located in the critical area, and when detecting area change information of the touch action, the controller can detect position information of the ending touch point.

[0123] If the end touch point is in the floating window, the end touch point may still be in the critical area, or may be in other display areas outside the critical area of ​​the floating window. This means that the user's touch action is always in the floating window and has not entered the area outside the floating window. In this case, the area change information of the touch action can be marked as a touch action in the same area.

[0124] If the end touch point is not located in the floating window, the end touch point may be located in the window scaling area or in other display areas outside the window scaling area, indicating that the user's touch action has slid from the floating window to the outside, and the area change information of the touch action can be marked as a cross-area touch action.

[0125] In some embodiments, if the area change information of the touch action is a same-area touch action, the user has been performing touch interaction in the floating window, indicating that the user does not have the need to adjust the size of the floating window, and the controller can mark the instruction type as an application control instruction.

[0126] If the area change information of the touch action is a cross-area touch action, the user's touch interaction slides from the floating window to the outside, indicating that the user has the need to adjust the size of the floating window, and the controller can mark the command type as a floating window zoom command.

[0127] In some embodiments, if the instruction type is a floating window zoom instruction, the controller may perform a zoom operation on the floating window according to the starting touch point and the ending touch point.

[0128] According to the relative position relationship between the start touch point and the end touch point, the controller can calculate the scaling type and scaling ratio. According to the scaling type and scaling ratio, the controller can perform a scaling operation on the floating window.

[0129] In some embodiments, when calculating the zoom type and the zoom ratio, the controller may obtain a first coordinate of a starting touch point and a second coordinate of an ending touch point.

[0130] The controller may obtain the center coordinates of the floating window, calculate a first distance between the center coordinates and the first coordinates, and calculate a second distance between the center coordinates and the second coordinates.

[0131] If the first distance is less than the second distance, the marking zoom type is a zoom-in operation. If the first distance is greater than the second distance, the marking zoom type is a zoom-out operation.

[0132] According to the user's sliding operation, the zoom ratio can be determined to obtain the size and coordinates of the adjusted floating window. When calculating the zoom ratio, the zoom ratios of the vertical height and horizontal width can be calculated separately. The horizontal extension distance and the vertical extension distance can be calculated by the first coordinate of the starting touch point and the second coordinate of the ending touch point. The zoom ratio can be calculated based on the extension distance and the original size of the floating window.

[0133] The controller can update and display the floating window according to the new size and coordinates.

[0134] In some embodiments, when the user touches the display, the touch component also detects the touch action at a fixed period, that is, the period for the display device to respond to the touch action, which can be 20 - 30 ms. Thus, the effect that the display device responds to the user's touch in real time is achieved.

[0135] Taking the above touch event as an example, based on the callback of "MotionEvent.ACTION_MOVE", the display device 200 can obtain the updated touch coordinates UpdatePoint(x1, y1), that is, the real-time coordinates, which can be used as the position information of the termination touch point under the current round of touch action. Based on the current window display area of the first floating window, the center coordinates CenterPoint(x0, y0) are obtained.

[0136] Based on the Pythagorean theorem, the distances between the center coordinates CenterPoint(x0, y0) and the first coordinate Point(x, y), and the second coordinate UpdatePoint(x1, y1) are calculated respectively according to the following formula:

[0137] d1 = Math.sqrt[(x - x0) * (x - x0) + (y - y0) * (y - y0)];

[0138] d2 = Math.sqrt[(x1 - x0) * (x1 - x0) + (y1 - y0) * (y1 - y0)];

[0139] Among them, Math.sqrt() represents a function that returns the square root of a number, d1 represents the first distance, and d2 represents the second distance. If d1 < d2, that is, d2 - d1 > 0, it is a zoom-in operation. If d1 > d2, that is, d2 - d1 < 0, it is a zoom-out operation. Thus, the display device 200 performs the window scaling operation until it receives the touch release event (MotionEvent.ACTION_UP) and the instruction ends.

[0140] In addition, for the window scaling operation, the display device 200 can calculate the updated window boundary (mWindowDragBounds) when the floating window is dragged, and update the updated window boundary to the floating window list (Task list) through the resizeTask() method. Then, based on the floating window list, it controls and adjusts the size and position of the image interface (surface) corresponding to the floating window, and notifies SurfaceFlinger for update, where SurfaceFlinger is a system service in the display device 200 responsible for managing and drawing the screen image.

[0141] In some embodiments, in order to improve user experience, a window limit threshold may also be set. When the user instructs to enlarge the floating window, the controller may detect the window size ratio of the floating window, which is the ratio of the area of ​​the floating window to the display area.

[0142] When the window size exceeds the window limit threshold, the window is directly made full-screen. For example, the default window size ratio when the floating window is started is 35%. When the user controls the display device to perform a window enlargement operation of the floating window through a specific interactive action, the display device detects the window size of the floating window. When the window size ratio is increased to 77%, a prompt message such as "will be displayed in full screen soon" can be displayed. If the user continues to increase the window size, the window is made full-screen, and the display is controlled to display the floating window in full screen.

[0143] Before executing the full-screen operation of the window, the window status may also be displayed at the bottom of the user interface displayed on the display to prompt the user that the floating window is about to enter the full-screen operation.

[0144] In some embodiments, in order to ensure the display clarity of the floating window, a window reduction threshold may be set, which is the minimum window size ratio of the floating window. If the window size ratio of the floating window is less than the window reduction threshold after the user instructs the floating window to reduce, the floating window will not be scaled. For example, the window reduction threshold may be set to 20%, that is, the window size ratio shall not be less than 20%, and the size of the floating window when minimized is 20% of the display size.

[0145] In some embodiments, the user can move the floating window. The floating window can be provided with a floating window moving area, and the floating window moving area can be an area on the upper part of the floating window. Fig.16 Schematic diagrams showing the floating window moving area in some embodiments are shown. Fig.16 As shown, area Y is the floating window moving area.

[0146] The display device can obtain the touch start coordinates input by the user through the finger, and determine the current window operation according to the interactive area of ​​the floating window. When the touch start coordinates are located in the floating window moving area, the window operation is determined to be a floating window moving operation.

[0147] The present application also provides a floating window scaling method, which is applied to a display device, and the method includes:

[0148] Step 101: In response to a floating window display instruction, control the display to display a floating window; the floating window is used to display an application program interface;

[0149] Step 102: Setting a window zoom area within a preset width of a border around the floating window;

[0150] Step 103: obtaining the starting touch point and the ending touch point of the touch action;

[0151] Step 104: acquiring the instruction type corresponding to the touch action according to the starting touch point and the ending touch point; the instruction type includes a floating window zoom instruction and an application control instruction, the floating window zoom instruction is used to instruct to change the size of the floating window, and the application control instruction is used to instruct to control the display content in the floating window; wherein, when the starting touch point is located in the window zoom area, the instruction type is a floating window zoom instruction;

[0152] Step 105: If the instruction type is a floating window zoom instruction, a zoom operation is performed on the floating window according to the starting touch point and the ending touch point.

[0153] In some embodiments, obtaining the starting touch point and the ending touch point of the touch action further includes:

[0154] In response to the touch action captured by the touch component, the position information of the user's initial touch point is obtained.

[0155] If the starting touch point is located in the floating window, the floating window is highlighted and the position information of the ending touch point of the touch action is obtained.

[0156] If the starting touch point is located in the window zoom area, obtain the number of starting touch points; when the number of starting touch points is 1, highlight the window zoom area and obtain the position information of the ending touch point of the touch action; when the number of touch points is greater than 1, do not execute the step of obtaining the position information of the ending touch point of the touch action.

[0157] In some embodiments, it also includes:

[0158] Get the action type of the touch action. If the action type is a sliding action, execute the step of obtaining the instruction type corresponding to the touch action according to the starting touch point and the ending touch point. If the action type is not a sliding action, detect the position information of the ending touch point; if the ending touch point is located in the floating window, generate an application control instruction according to the touch action and execute the application control instruction.

[0159] In some embodiments, acquiring the instruction type corresponding to the touch action according to the starting touch point and the ending touch point further includes:

[0160] Detect the position information of the starting touch point. If the starting touch point is in the window zoom area, mark the instruction type as a floating window zoom instruction. If the starting touch point is in the floating window, detect the area change information of the touch action, and obtain the instruction type corresponding to the touch action according to the area change information.

[0161] In some embodiments, if the initial touch point is located in the floating window, the controller further includes:

[0162] If the initial touch point is located in the critical area, the step of detecting area change information of the touch action is performed; the critical area is an area of ​​preset width within the boundary of the floating window.

[0163] If the initial touch point is not located in the critical area, the instruction type is marked as an application control instruction; and the application control instruction is generated according to the touch action and the application control instruction is executed.

[0164] In some embodiments, detecting the area change information of the touch action further includes:

[0165] Detect the position information of the termination touch point. If the termination touch point is located in the floating window, the area change information of the marking touch action is a same-area touch action. If the termination touch point is not located in the floating window, the area change information of the marking touch action is a cross-area touch action.

[0166] In some embodiments, obtaining the instruction type corresponding to the touch action according to the area change state further includes:

[0167] If the area change information of the touch action is a same-area touch action, the marked instruction type is an application control instruction. If the area change information of the touch action is a cross-area touch action, the marked instruction type is a floating window zoom instruction.

[0168] In some embodiments, performing a zoom operation on the floating window according to the starting touch point and the ending touch point further includes:

[0169] The zoom type and zoom ratio are calculated according to the relative position relationship between the start touch point and the end touch point. The zoom operation is performed on the floating window according to the zoom type and zoom ratio.

[0170] In some embodiments, calculating the scaling type and the scaling ratio further includes:

[0171] Get the first coordinate of the starting touch point and the second coordinate of the ending touch point. Get the center coordinate of the floating window. Calculate the first distance between the center coordinate and the first coordinate, and calculate the second distance between the center coordinate and the second coordinate.

[0172] If the first distance is less than the second distance, the marking zoom type is a zoom-in operation. If the first distance is greater than the second distance, the marking zoom type is a zoom-out operation.

[0173] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0174] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0176] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display configured to display a user interface, wherein the user interface includes an application program interface; A touch component configured to collect touch actions input by a user; The controller is configured as: In response to a floating window display instruction, controlling the display to display a floating window; the floating window is used to display the application interface; Setting a window scaling area within a preset width of a border around the floating window; Obtaining a starting touch point and an ending touch point of the touch action; Acquiring the instruction type corresponding to the touch action according to the starting touch point; the instruction type includes a floating window zoom instruction and an application control instruction, the floating window zoom instruction is used to instruct to change the size of the floating window, and the application control instruction is used to instruct to control the display content in the floating window; wherein, when the starting touch point is located in the window zoom area, the instruction type is a floating window zoom instruction; If the instruction type is a floating window zoom instruction, a zoom operation is performed on the floating window according to the starting touch point and the ending touch point.

2. The display device according to claim 1, characterized in that The controller executes acquiring the starting touch point and the ending touch point of the touch action, and is further configured to: In response to the touch action collected by the touch component, obtaining position information of the user's starting touch point; If the starting touch point is located in the floating window, highlighting the floating window, and acquiring the position information of the ending touch point of the touch action; If the starting touch point is located in the window zoom area, the number of the starting touch points is obtained; when the number of the starting touch points is a preset value, the window zoom area is highlighted and the position information of the ending touch point of the touch action is obtained; when the number of the touch points is greater than the preset value, the step of obtaining the position information of the ending touch point of the touch action is not performed.

3. The display device according to claim 2, characterized in that The controller is also configured to: Obtaining the action type of the touch action; If the action type is a sliding action, executing the step of acquiring the instruction type corresponding to the touch action according to the starting touch point; If the action type is not a sliding action, detecting the position information of the termination touch point; If the termination touch point is located in the floating window, an application control instruction is generated according to the touch action and the application control instruction is executed.

4. The display device according to claim 1, characterized in that The controller executes the instruction type corresponding to the touch action according to the starting touch point, and is further configured to: Detecting the position information of the starting touch point; If the starting touch point is located in the window zooming area, the marking instruction type is a floating window zooming instruction; If the initial touch point is located in the floating window, the area change information of the touch action is detected, and the instruction type corresponding to the touch action is acquired according to the area change information.

5. The display device according to claim 4, characterized in that If the initial touch point is located in the floating window, the controller is further configured to: If the initial touch point is located in a critical area, a step of detecting area change information of the touch action is performed; the critical area is an area of ​​a preset width within the boundary of the floating window; If the initial touch point is not located in the critical area, marking the instruction type as an application control instruction; and generating an application control instruction according to the touch action and executing the application control instruction.

6. The display device according to claim 5, characterized in that The controller executes detection of area change information of the touch action, and is further configured to: Detecting the position information of the termination touch point; If the termination touch point is located in the floating window, marking the area change information of the touch action as a same-area touch action; If the termination touch point is not located in the floating window, the area change information of the touch action is marked as a cross-area touch action.

7. The display device according to claim 6, characterized in that The controller executes the instruction type corresponding to the touch action according to the area change state, and is further configured to: If the area change information of the touch action is a same-area touch action, the marked instruction type is an application control instruction; if the area change information of the touch action is a cross-area touch action, the marked instruction type is a floating window zoom instruction.

8. The display device according to claim 1, characterized in that The controller performs a zoom operation on the floating window according to the starting touch point and the ending touch point, and is further configured to: Calculating a zoom type and a zoom ratio according to a relative position relationship between the starting touch point and the ending touch point; A scaling operation is performed on the floating window according to the scaling type and the scaling ratio.

9. The display device according to claim 8, characterized in that The controller performs calculation of the scaling type and scaling ratio and is further configured to: Acquire a first coordinate of the starting touch point and a second coordinate of the ending touch point; Obtaining the center coordinates of the floating window; Calculating a first distance between the center coordinate and the first coordinate, and calculating a second distance between the center coordinate and the second coordinate; If the first distance is less than the second distance, marking the zoom type as a zoom-in operation; If the first distance is greater than the second distance, the scaling type is marked as a zoom-out operation.

10. A floating window scaling method, applied to a display device, characterized in that: The method comprises: In response to a floating window display instruction, controlling the display to display a floating window; the floating window is used to display the application interface; Setting a window scaling area within a preset width of a border around the floating window; Get the starting touch point and the ending touch point of the touch action; Acquiring the instruction type corresponding to the touch action according to the starting touch point; the instruction type includes a floating window zoom instruction and an application control instruction, the floating window zoom instruction is used to instruct to change the size of the floating window, and the application control instruction is used to instruct to control the display content in the floating window; wherein, when the starting touch point is located in the window zoom area, the instruction type is a floating window zoom instruction; If the instruction type is a floating window zoom instruction, a zoom operation is performed on the floating window according to the starting touch point and the ending touch point.

Images