Display device and display method of acceleration layer

Abstract

The embodiment of the invention provides display equipment and a display method of an acceleration layer, and the method can respond to a target touch track input by a user based on a first application, refresh the target touch track to the acceleration layer and a first application layer used for displaying the first application, and detect a touch unit in the acceleration layer. Wherein the touch control unit is a display unit acted by the target touch control track. Reading an application display range identifier of the touch unit; and when the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit. Adjusting the transparency of the second application layer, and setting the display level of the second application layer to be higher than the display level of the acceleration layer. According to the method, by adjusting the transparency and the display level of the application layer, the touch track acting on the overlapped part can be displayed in the user interface through the upper application layer, so that the touch display effects of multiple applications are not mutually influenced, and the touch display effect of the multiple applications is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a display method of an acceleration layer. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] For display devices that support touch operations, users can use the built-in writing application to implement operations such as writing and erasing touch tracks in the display device. That is, the display device can convert the touch data into corresponding bitmap data through the application layer of the writing application and display it on the screen. However, since the application layer takes a certain amount of time to process the touch data, the user cannot feel the touch feedback in time when writing. Therefore, the display device is also provided with a physical acceleration layer that is higher than the application layer. The display device can display the touch process in real time through the physical acceleration layer, and then display the final touch effect through the application layer.

[0004] However, when multiple application windows are displayed at the same time, since multiple applications share the same acceleration layer, when there are overlapping parts in the application windows of multiple applications, the touch track input to the lower application of the overlapping part will be presented in the upper application window through the acceleration layer, which will interfere with the display content of the upper application, resulting in poor touch display effect between multiple applications. Summary of the invention

[0005] The present application provides a display device and a display method of an acceleration layer to solve the problem of mutual influence of touch display processes between multiple applications.

[0006] In a first aspect, some embodiments of the present application provide a display device, including a display and a controller. The display is configured to display a drawing interface, the drawing interface includes a first application window and a second application window, the first application window is a window displayed based on the first application, and the second application window is a window displayed based on the second application; the first application window and the second application window are displayed through different application layers; the display is connected to a touch interaction module, and the touch interaction module is used to detect the touch track of the user input; when displaying the drawing interface, the display also displays the touch track through an acceleration layer, the initial display level of the acceleration layer is higher than the display level of the application layer, and the display unit of the acceleration layer is provided with an application display range identifier; the controller is configured to execute the following program steps:

[0007] In response to a target touch track input by a user based on a first application, the target touch track is refreshed to the acceleration layer and the first application layer, and a touch unit in the acceleration layer is detected; the first application layer is an application layer for displaying the first application, and the touch unit is a display unit acted upon by the target touch track;

[0008] Reading an application display range identifier of the touch control unit;

[0009] When the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit, the first identifier being the application display range identifier of the first application;

[0010] The transparency of the second application layer is adjusted, and the display level of the second application layer is set to be higher than the display level of the acceleration layer, wherein the second application layer is an application layer for displaying a second application.

[0011] In a second aspect, some embodiments of the present application further provide a method for displaying an acceleration layer, the method being applied to the display device provided in the first aspect, the method comprising:

[0012] In response to a target touch track input by a user based on a first application, the target touch track is refreshed to an acceleration layer and a first application layer, and a touch unit in the acceleration layer is detected; the first application layer is an application layer for displaying the first application, and the touch unit is a display unit acted upon by the target touch track;

[0013] Reading an application display range identifier of the touch control unit;

[0014] When the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit, the first identifier being the application display range identifier of the first application;

[0015] The transparency of the second application layer is adjusted, and the display level of the second application layer is set to be higher than the display level of the acceleration layer, wherein the second application layer is an application layer for displaying a second application.

[0016] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a display method of an acceleration layer. The method can respond to a target touch track input by a user based on a first application, refresh the target touch track to the acceleration layer and the first application layer used to display the first application, and detect the touch unit in the acceleration layer. The touch unit is the display unit acted upon by the target touch track. Read the application display range identifier of the touch unit. When the application display range identifier of the touch unit is not the first identifier, detect the second application associated with the touch unit. Adjust the transparency of the second application layer, and set the display level of the second application layer to be higher than the display level of the acceleration layer. The second application layer is an application layer used to display the second application. The method can display the touch track acting on the overlapping part in the user interface through the upper application layer by adjusting the transparency and display level of the application layer, so that the touch display effects between multiple applications do not affect each other, thereby improving the touch display effects between multiple applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0019] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0020] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0021] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0022] Figure 5 A structural diagram of a display system provided for some embodiments of the present application;

[0023] Figure 6 An example diagram of a display hierarchy architecture provided for some embodiments of the present application;

[0024] Figure 7 An architecture diagram of a display device that performs an acceleration layer display method according to some embodiments of the present application;

[0025] Figure 8 A schematic diagram of a process for generating window dynamic events provided in some embodiments of the present application;

[0026] Fig. 9 A schematic diagram of the effect of updating an application display range identifier provided in some embodiments of the present application;

[0027] Fig.10 Another schematic diagram of the effect of updating the application display range identifier provided in some embodiments of the present application;

[0028] Fig.11 Another schematic diagram of the effect of updating the application display range identifier provided in some embodiments of the present application;

[0029] Fig.12 Another schematic diagram of the effect of updating the application display range identifier provided in some embodiments of the present application;

[0030] Fig.13 A schematic diagram of the display effect of the touch track provided in some embodiments of the present application;

[0031] Fig.14 A schematic diagram of a flow chart of a method for displaying an acceleration layer provided in some embodiments of the present application;

[0032] Fig.15 A schematic diagram of the effect of a touch track acting on a touch unit provided in some embodiments of the present application;

[0033] Fig.16 An example diagram of the touch track interference display effect provided in some embodiments of the present application;

[0034] Fig.17 A schematic diagram of the display effect of touch tracks when multiple applications are displayed overlappingly provided in some embodiments of the present application;

[0035] Fig.18 A schematic diagram of the effect of the touch track input process provided in some embodiments of the present application;

[0036] Fig.19 A schematic diagram of a process for adjusting the transparency of the second application layer provided in some embodiments of the present application;

[0037] Fig. 20 A schematic diagram of the effect of an erasure event provided in some embodiments of the present application;

[0038] Fig.21 A schematic diagram of the display effect of touch tracks when application windows are displayed overlappingly provided in some embodiments of the present application;

[0039] Fig. 22 A schematic diagram of the display effect of touch tracks when application windows provided in some embodiments of the present application are displayed side by side;

[0040] Fig.23 A schematic diagram of the display effect of an erasing event when application windows are displayed overlappingly provided in some embodiments of the present application. DETAILED DESCRIPTION

[0041] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0042] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0043] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0044] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0045] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0046] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0047] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through fingers, the mobile terminal 300 and the control device 100. The control device 100 may be a remote controller, a stylus, etc. In some embodiments, the mobile terminal 300 may install software applications with the display device 200, and realize connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 may also be transmitted to the display device 200 to realize a synchronous display function.

[0048] like Figure 1 It is also shown that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect via a local area network (LAN), a wireless local area network (WLAN) and other networks. In addition to providing a broadcast receiving television function, the display device 200 may also provide an intelligent network television function that provides a computer support function, including but not limited to network television, smart television, Internet protocol television (IPTV), etc.

[0049] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0050] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a communication device interface.

[0051] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0052] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0053] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0054] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0055] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0056] In some embodiments, the display device 200 is provided with a touch interaction module. The touch interaction module may be a built-in module or an external module. For example, the touch interaction module may be a touch display, which can receive touch tracks input by fingers, contacts, styluses, etc., to implement touch operation functions.

[0057] In some embodiments, the touch interaction module detects the touch track of the user input. According to the interaction principle of the touch interaction module, the touch interaction module can detect the relevant parameters of the touch track. For example, when the user's finger touches any position of the touch display, the capacitance of the touch display at the touch position will change. At this time, the touch interaction module can record the position of the capacitance change, and record the time when the capacitance change occurs in combination with the timer in the controller to generate the touch track.

[0058] Among them, the touch track can include different operation parameters according to the user's input method. For example, the touch track can include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. In a touch interaction process, the touch event types that can be detected by the touch interaction module include: the drop (down) event, move (move) event and lift (up) event of the touch action. Each touch event records the corresponding touch time and touch position.

[0059] In some embodiments, the device interface 240 may be an HDMI interface and a USB interface. The HDMI interface may be connected to a device having video and audio input / output functions, and the corresponding audio and video output / input functions may be realized through the HDMI driver. The USB interface may be connected to a device having signal and data transmission functions and interactive functions, and based on the USB driver, access signals, transmit data, and perform specific interactive actions.

[0060] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0061] In some embodiments, the communication device interface 280 is an interface that can be used to receive control input.

[0062] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0063] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0064] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0065] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0066] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0067] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0068] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0069] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0070] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0071] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0072] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0073] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0074] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0075] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0076] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0077] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0078] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0079] Based on the above-mentioned display device 200, in the process of executing the touch interaction response, the display device 200 can classify the touch tracks corresponding to different touch times, touch positions, number of touch points and touch event types, that is, by detecting the user's touch input gesture in the touch track, different interactive responses can be performed for different input gestures. For example, when the number of touch points in the touch track detected by the touch interaction module is 1, the touch event type only includes a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold, and the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 performs the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0080] Similarly, the display device 200 can also detect other touch interaction gestures input by the user, such as double-click gesture, slide gesture, long press gesture, multi-finger click gesture, multi-finger slide gesture, etc., based on the touch time, touch position, number of touch points, and touch track corresponding to the touch event type. Different interactive gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, and slide track shape. Different touch parameters can be used to perform different interactive controls.

[0081] The above-mentioned touch track can also be used in conjunction with a specific application to achieve a specific function. For example, when a user starts a touch display application for rendering a touch track, the display 260 can present an application window of the touch display application; in the application window of the touch display application, the user can draw a specific touch track in the drawing area of ​​the application window through a sliding gesture, and the controller 250 controls the display 260 to display the touch track received by the touch interaction module in real time to meet the demonstration effect.

[0082] The touch display application can be a system application or a third-party application developed based on a specific display device type and user needs. The touch display application can be installed in the display device 200 and started and run under user control. For example, the user can control the display device 200 to run the whiteboard drawing application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0083] It should be noted that the above-mentioned whiteboard drawing application can be an independent application, an associated application, or a functional module in a touch application. Different forms of whiteboard drawing applications can have different startup methods. For example, a user can start an independent whiteboard drawing application through the application interface of the display device 200; a user can also start a whiteboard drawing application by executing the "whiteboard" operation on the conference application; a user can also control the conference application to start the whiteboard drawing functional module by clicking the "whiteboard" icon on the conference application.

[0084] After the whiteboard drawing application is running, the display device 200 can be switched to display a whiteboard interface. The whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, graphics, line shape, color, erase, clear screen, selection and other operation controls. The user can click any control to draw graphics according to specific drawing needs.

[0085] like Figure 5 As shown, the display device 200 can render and display the drawing picture in real time through an image generation (Image Producer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module and a display.

[0086] During the display process, the media player (media player), camera peripheral (cameraper view) and graphics interface (openGLES) of the image generation module can generate image content and pass the image content to the image rendering module. The image rendering module includes a display unit (display), which is used to control the display content of the display, that is, the display unit renders the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the picture frames can be stored in the synthesis module through the frame buffer (frame buffer) and the video buffer (video buffer), and the picture content can be formed in the corresponding layers through the video process module. The synthesis module can form the picture content through software or hardware. For example, the synthesis module can be a hardware synthesis (Hardware Composer, HWC) module. That is, the image frames generated by the video playback process form the picture content through the video layer (video plane), and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the OSD layer (OSD plane). Finally, the display device 200 forms a final picture through a blending unit (blend) of the video process module and displays it on the display 260 .

[0087] Among them, the OSD layer of the display device 200 is also called a display layer, an intermediate layer, a menu layer or an application layer, and can be used to display application windows, application menus, toolbars and other contents.

[0088] In some embodiments, in order to obtain a better picture display effect, an additional extended display layer (external OSD plane) can be set in the video process module of the display device 200 on the basis of the OSD layer and the video layer. The extended display layer can be configured with different layer characteristics according to specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0089] Taking the display device 200 running a whiteboard drawing application as an example, in order to improve the hand tracking of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). The display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the lines drawn by the acceleration layer are updated to other layers, thereby reducing input delay and improving the real-time display effect of the whiteboard drawing application drawing process. For example, Figure 6 As shown, the display layers of the display device 200 may be acceleration layer-OSD layer-video layer from high to low.

[0090] Understandably, Figure 6 The hierarchical relationship shown is only an exemplary description, and the display hierarchy between the acceleration layer, the OSD layer and the video layer can be updated and adjusted in real time according to the configuration of the application. This application does not impose any limitation on this.

[0091] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch track input by the user at the same time as the OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, that is, when the up event is monitored, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawing graphics, so there is no need for the acceleration layer to refresh the drawing graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0092] However, when multiple application windows of touch display applications are displayed simultaneously in the user interface of the display device 200, the touch display effects between the multiple applications will interfere with each other. Since the applications in the display device 200 share an acceleration layer, when there are overlapping parts in the application windows of multiple applications, the touch track input by the user to the lower application in the overlapping part will be presented in the upper application window through the acceleration layer, causing problems such as occlusion and interference to the display content of the upper application window. For example, when both application A and application B turn on acceleration through the acceleration layer, the application windows of application A and application B have overlapping parts, and the display level of application A is lower than that of application B, when the user performs an erase operation on application A in the overlapping part, the touch track drawn in application B may be accidentally erased or scratched, resulting in poor touch display effects between multiple applications and reducing user experience.

[0093] Based on the above application scenarios, in order to improve the problem of mutual influence between touch display processes of multiple applications, some embodiments of the present application provide a display device 200, including a display 260, a user interface 280 and a controller 250. Figure 7 As shown, the display 260 is configured to display a drawing interface, which includes a first application window and a second application window. The first application window is a window displayed based on the first application, and the second application window is a window displayed based on the second application. The first application window and the second application window are displayed through different application layers.

[0094] The display 260 is connected to the touch interaction module. When displaying the drawing interface, the display 260 also displays the touch track through the acceleration layer. The initial display level of the acceleration layer is higher than the display level of the application layer. The display unit of the acceleration layer is provided with an application display range identifier.

[0095] In some embodiments, the acceleration layer is composed of multiple display units, and one display unit may include one pixel or multiple pixels. The acceleration layer can cover the entire user interface displayed by the display 260, and can be used to display the touch process of the touch track in real time or to display the content located on the upper layer of the application layer, etc. Among them, the display unit of the acceleration layer is provided with an application display range identifier for characterizing the associated application, and the display unit provided with an application display range identifier of a certain application is the application display area of ​​the application. For example, the application display range identifier of application A is "1", and the area constituted by the display units provided with "1" in the acceleration layer is the application display area of ​​application A.

[0096] In some embodiments, the application display range identifier associated with the target application is obtained based on the application package name registration of the target application. Since the application package names of each application are different and unique, the application display range identifier obtained by each application based on the application package name registration is also different, thereby ensuring the uniqueness of the application display range identifier.

[0097] In order to facilitate the setting of the application display range identifier of each display unit in the acceleration layer, in some embodiments, the display device 200 also detects the window change area of ​​the target application in response to the window dynamic event of the target application. Among them, the target application can be any application in the display device 200; the window dynamic event includes a window change event, a window start event and a window close event, and the window change event includes a window size change event and a window position change event, that is, the zoom-in, zoom-out, move, open and close operations performed on the application window of the target application in the display device 200 are all window dynamic events of the target application. The window change area includes the window display area of ​​the target application before the display device 200 executes the window dynamic event and the window display area of ​​the target application after the window dynamic event is executed. After detecting the window change area of ​​the target application, the display device 200 queries the target unit according to the window change area, and updates the application display range identifier of the target unit based on the window display level. Among them, the target unit includes the display unit located at the window change area position in the acceleration layer.

[0098] That is to say, once the display device 200 detects that any of the above-mentioned types of window dynamic events occur in the window of the target application, it detects the window change area of ​​the target application, and sets the application display range identifier of the target application in the display unit at the corresponding position of the acceleration layer according to the detected change area, thereby completing the application display area division of the application.

[0099] In some embodiments, the view system (View System) of the display device 200 monitors window dynamic events. When monitoring the window dynamic events of the target application, the view system sends the layout parameters of the target application before and after the window changes to the controller 250. For example, when the application window of the target application of the display device 200 changes, the view system sends a layoutchange message to the controller 250, and the controller 250 determines the window change area of ​​the target application based on the layout parameters contained in the layoutchange message. In some embodiments, the window dynamic event can be input by the user based on the control device 100; or, the window dynamic event can also be input by the user based on voice commands, touch operations, etc.; or, the window dynamic event can also be a dynamic event automatically generated by the display device 200 based on a specific program. For example, such as Figure 8 As shown, Figure 8 The application window shown is the application window of application A of the display device 200. The user drags the application window of application A through a touch operation, and the display device 200 generates a window dynamic event of application A according to the touch operation. In some embodiments, when the display device 200 detects the window change area of ​​the target application, it calls the data file of the target application and extracts the window position information and window size information in the data file. Among them, the window position information can be the boundary coordinates of the application window and / or the center coordinates of the window, and the window size information may include the shape attributes of the application window, the length of the application window, or the width of the application window. After extracting the window position information and the window size information, the display device 200 calculates the window display area of ​​the target application according to the window position information and the window size information, so as to obtain the window change area according to the window display area.

[0100] It is understandable that the above-mentioned window display area includes the window display area before the target application executes the window dynamic event and the window display area after the target application executes the window dynamic event, that is, the union of the window display areas before and after the target application window changes constitutes the above-mentioned window change area. In order to improve the accuracy of the application display range identification in each display unit of the acceleration layer, when updating the application display range identification of the target unit, the display device 200 can determine the application display range identification to be set for the current target unit through the display level of the application window.

[0101] Therefore, in some embodiments, when the display device 200 updates the ownership identification of the target unit, the window display level of the application in the changed area is obtained. The application in the changed area is an application whose application window is displayed in the window change area. Then the identification application of the target unit is queried according to the window display level. The identification application is the change area application with the highest window display level at each target unit position. If there is an identification application at the target unit position, the application display range identification of the target unit is updated to the application display range identification of the identification application; if there is no identification application at the target unit position, the application display range identification of the target unit is updated to the initial identification. The initial identification is the application display range identification preset in the display device 200, which is used to characterize the application display range identification of the current display unit for which no application is set.

[0102] For example, the drawing interface includes an application, and the display device 200 is provided with application A, the initial identification of the display device 200 is "NA", and the application display range identification of application A is "1"; Fig. 9 As shown, if there is no displayed application window in the drawing interface of the display device 200, all display units of the acceleration layer are set to the initial identification. At this time, the user starts application A in the display device 200 and displays the application window of application A in full screen. The display device 200 detects the window change area of ​​application A in response to the window start event of application A. After detecting that the display area after the start of application A is displayed in full screen, it is determined that the target unit is all the display units of the acceleration layer. The display device 200 updates the initial identification "NA" of all display units to the application display range identification "1" of application A.

[0103] For example, the drawing interface includes two applications. Application A and application B are set in the display device 200. The initial identification of the display device 200 is "NA", the application display range identification of application A is "1", and the application display range identification of application B is "2". The application window of application A is half-screen displayed in the drawing interface. At this time, the user starts application B in the display device 200, so that application B is suspended and displayed in the drawing interface. Fig.10 The display device 200 responds to the window start event of application B and detects the window display area of ​​application B before and after the change to determine the window change area. After detecting the window change area of ​​application B, the display unit at the corresponding position is determined according to the window change area of ​​application B as the target unit. Fig.10Before the application display range identifier of the target unit is updated, part of the application display range identifiers are the initial identifier "NA" and part of the application display range identifiers are the application display range identifier "1" of application A. Since the display level of application B is above the display level of application A, the display device 200 updates the application display range identifiers "NA" and "1" of the target unit to the application display range identifier "2" of application B.

[0104] In the above two examples, there is an identified application at the target unit position. Obviously, when there is no identified application at the target unit position, the application display range identifier needs to be updated to the initial identifier. For example, the drawing interface includes an application, and the display device 200 is provided with application A, the initial identifier of the display device 200 is "NA", and the application display range identifier of application A is "1"; Fig.11 As shown, the application window of application A is displayed in full screen on the display device 200, and all display units of the acceleration layer are set with the application display range identifier "1" of application A. At this time, the user adjusts the application window of application A to half screen display through the remote control of the display device 200. The display device 200 responds to the window change event of application A, detects the window display area before and after the change of application A, and determines that the target unit is a full screen display unit, and the changed area application is application A. Fig.11 , the highest-level change area application displayed on the left half screen of the display device 200 is application A, and the right half screen does not include the display content of any application window, then the display unit in the right half screen area is updated to the initial identifier "NA", and the target unit in the left half screen area still maintains the original application display range identifier "1".

[0105] For example, the drawing interface includes two applications, and the display device 200 is provided with application A and application B, the initial identification of the display device 200 is "NA", the application display range identification of application A is "1", and the application display range identification of application B is "2"; Fig.12 As shown in FIG. 1 , the application window of application B is displayed floating on the application window of application A, and the application display range of the display unit in the overlapping area of ​​the application B and application A windows is indicated as the application display range identifier "2" of application B. After the user drags the window of application B to the lower right, it is determined as follows Fig.12 The window change area includes application A and application B, and the application A and application B are determined to be the application A and application B. According to the display level of application A and application B, the target unit at the corresponding position is updated to the application display range identifier "1" of application A and the application display range identifier "2" of application B, and the display unit of the application that does not have the window change area identifier is updated to the initial identifier "NA". Fig.12 shown.

[0106] Understandably, Figure 9-12 The application display range identifier shown is not displayed in the drawing interface. The application display range identifier shown in the above-mentioned figure is only for the convenience of explaining the application display range identifier of the embodiment of the present application. The application display range identifier is data stored in the memory of the display device 200 and will not be displayed in the drawing interface of the display device 200.

[0107] In order to reduce the occupation of system resources, in some embodiments, the display device 200 also monitors the application type corresponding to the application window in the drawing interface. If the drawing interface does not include a touch display application, it means that the display device 200 does not need to perform physical acceleration through the acceleration layer, and the data resources of the acceleration layer are released. On the contrary, if the drawing interface includes a touch display application, the data resources of the acceleration layer are loaded to display the display content of the acceleration layer through the data resources. In other words, when the drawing interface displayed by the display device 200 does not include any touch display application such as a whiteboard drawing application, the display device 200 can automatically release the data resources of the acceleration layer to save the occupation of system resources.

[0108] It should be noted that the drawing interface described in the embodiment of the present application is the user interface displayed by the display device 200.

[0109] In some embodiments, the touch trajectory includes display units acted upon by the drop action, the move action, and the lift action. A touch trajectory must include display units acted upon by the drop action, the move action, and the lift action. Among them, the display unit acted upon by the drop action constitutes the touch start point, the display unit acted upon by the move action is the touch trajectory point, and the display unit acted upon by the lift action is the touch end point. The touch trajectory point and the touch end point are both display units acted upon after the touch start point time, and can be collectively referred to as touch drive points, that is, the touch drive points include the display units acted upon after the touch start point time in the touch trajectory.

[0110] For example, Fig.13 As shown, the user draws on the acceleration layer based on the whiteboard drawing application of the display device Fig.13 The touch track shown is composed of multiple touch points, touch point a is the touch starting point, touch point b is the touch track point, and touch point f is the touch ending point.

[0111] In order to facilitate the display device 200 to distinguish and identify the touch points, in some embodiments, the data information of each touch point in the touch track includes coordinate information and action attributes, and the data information can be in different data formats. For example, the data format of the touch start point can be [down; x, y], the data format of the touch track point can be [move; x, y], and the data format of the touch end point can be [up; x, y], where x and y represent the horizontal and vertical coordinates of the touch point. The display device 200 can identify the touch point by parsing the above data information.

[0112] like Fig.14 As shown, the controller 250 is configured to perform the following program steps:

[0113] S1: In response to a target touch track input by a user based on a first application, refreshing the target touch track to an acceleration layer and a first application layer, and detecting a touch unit in the acceleration layer.

[0114] When the display device 200 detects the target touch track input by the user based on the first application, in response to the target touch track, the target touch track is refreshed to the acceleration layer and the first application layer of the display device 200. The first application is a touch display application for rendering the touch track, such as a whiteboard drawing application, a writing application, etc., and the first application layer is an application layer for displaying the first application. Figure 6 As shown, in order to achieve the acceleration effect of the touch track through the acceleration layer, the display level of the acceleration layer in the display device 200 is set to be higher than the application layer by default. At this time, the touch display effect of the acceleration layer will cover all application layers of the display device 200.

[0115] In some embodiments, the display device 200 controls the display, hiding, and layer order of the application layer based on the window manager service (Window Manager Service, WMS). For example, the application layer of the display device 200 is provided with a Z-order attribute. The Z-order attribute is used to characterize the display level of the window. The smaller the value in the Z-order attribute, the lower the display level of the corresponding window; the larger the value, the higher the display level of the corresponding window. The display device 200 can modify the value of the Z-order attribute based on the window manager service to adjust the display level of each window.

[0116] In order to facilitate the display device 200 to identify the touch track, in some embodiments, the display device 200 also detects the target position of the touch start point in the target touch track, and obtains the application display range identifier of the target position. According to the application display range identifier of the target position, the target touch track is refreshed to the application layer of the first application. That is, the display device 200 determines the application layer on which the current touch track acts according to the touch start point of each touch track.

[0117] Taking the first application as a whiteboard drawing application as an example, the whiteboard drawing application is application A, and the application display range identifier of application A is "1"; Fig.15 As shown, after starting the whiteboard drawing application, the user draws in the application window of the whiteboard drawing application. Fig.15 The curve S shown. The curve S is the target touch trajectory, including the touch starting point S1. The display device 200 responds to the touch trajectory of the curve S, detects the position of the touch starting point S1, and obtains the application display range identifier "1" set by the display unit at the position S1. In this way, the display device 200 can determine that the curve S is the touch trajectory acting on the application A according to the application display range identifier "1", and refresh the target touch trajectory to the application layer of the application A according to the application display range identifier "1".

[0118] In order to ensure the display effect of the target touch track, the display device 200 refreshes the target touch track to the acceleration layer and the first application layer, and also detects the touch unit in the acceleration layer, where the touch unit is the display unit acted upon by the target touch track.

[0119] In some embodiments, when the display device 200 detects the touch unit in the acceleration layer, it obtains the touch point coordinates of the target touch track, and queries the display unit of the acceleration layer according to the touch point coordinates to determine the touch unit of the target touch track. In other words, when the user interface 280 of the display device 200 detects the touch track, it reports the touch point coordinates of each touch point in the touch track to the controller 250 in real time, and the controller 250 determines the display unit at the coordinate position in the acceleration layer according to the reported touch point coordinates as the touch unit of the touch track.

[0120] In order to display the touch track in the user interface of the display device 200, in some embodiments, when the display device 200 refreshes the target touch track to the acceleration layer, it also generates bitmap data of the target touch track. Then the bitmap data is copied to the touch unit, and the touch termination point of the target touch track is monitored. When the touch termination point of the target touch track is detected, the bitmap data is copied to the application layer of the first application.

[0121] In this way, the acceleration layer can display the touch effect of the target touch track in real time, so that the user can perceive the interactive feedback of the touch track in time; at the same time, when the touch end point is detected, the complete touch track is refreshed to the application layer, which can facilitate the display device 200 to process the complete bitmap data of the target touch track, and reduce the display content fragmentation problem in the application layer. For example, when the touch start point in the target touch track is [down; x, y], the touch track point is [move; x, y], and the touch end point is [up; x, y], the display device 200 refreshes the target touch track to the application layer of the first application when [up; x, y] is detected.

[0122] In some embodiments, when the display device 200 generates the bitmap data of the target touch trajectory, it performs preprocessing on the target touch trajectory. Among them, the preprocessing includes removing noise, smoothing the trajectory curve, etc. to ensure the accuracy and consistency of the data. The target touch trajectory is then converted into a pixel point sequence. After converting the pixel point sequence of the target touch trajectory, the touch point coordinates in the target touch trajectory are mapped to the touch point coordinates of the user interface according to the pixel point sequence, and a specific color value is set for the pixel point sequence according to the configuration parameters of the target touch trajectory to generate the bitmap data of the touch trajectory. The bitmap data is refreshed to the application layer and the acceleration layer, and the touch trajectory can be displayed. For example, when the user selects the color of the target touch trajectory as black in the first application, the display device 200 sets the color value of the pixel point sequence to black.

[0123] S2: Read the application display range identifier of the touch unit.

[0124] After the display device 200 detects the touch unit in the acceleration layer, it also reads the application display range identifier set in the touch unit to determine the action position of the target touch track through the application display range identifier of the touch unit. For example, the application display range identifier of application A is "1" and the application display range identifier of application B is "2"; when the target touch track acts on the application display area of ​​application A, the application display range identifier read by the display device 200 is "1"; when the target touch track acts on the application display area of ​​application B, the application display range identifier read by the display device 200 is "2".

[0125] S3: When the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit.

[0126] Among them, the first identifier is the application display range identifier of the first application. When the display device 200 reads the application display range identifier in the touch unit that is not the first identifier, it means that the display content of the current touch unit position in the user interface is not the application window of the first application. At this time, the target touch track refreshed to the acceleration layer will block the display content at the current position. Therefore, the display device 200 needs to detect the second application associated with it according to the application display range identifier of the touch unit, and the second application is the display content of the user interface at this position.

[0127] For example, Fig.15 As shown, the application display range of application A is marked as "1", and the application display range of application B is marked as "2". The user enters in the display device 200 Fig.15 The touch track curve S is shown. Fig.15 , the application display range identifier of the touch start point S1 of curve S is "1", indicating that curve S is a touch track acting on application A. When the user draws to the touch driving point S2, the display device 200 reads the application display range identifier of the position S2 as "2". Since the application display range identifier of the current touch unit is not the application display range identifier "1" of application A, the display device 200 detects the application B associated with the current touch unit according to the application display range identifier "2".

[0128] S4: adjusting the transparency of the second application layer, and setting the display level of the second application layer to be higher than the display level of the acceleration layer.

[0129] In order to achieve the acceleration effect of the touch track through the acceleration layer, the default display level of the acceleration layer is higher than the application layer. Since the display level of the second application is higher than the first application, when the application display range in the acceleration layer is not the touch unit with the first mark, the touch track of the first application may act on the application window of the second application.

[0130] For example, Fig.16 As shown, when the second application is displayed in a floating layer on the upper layer of the first application, when the target touch track based on the input of the first application passes through the overlapping area of ​​the first application and the second application, the target touch track will be displayed on the application window of the second application, interfering with the display effect of the second application. After the user inputs a complete touch track, the content displayed by the acceleration layer will be directly released. At this time, since the display level of the second application layer is higher than that of the first application layer, the second application layer will block the first application layer, so the touch track refreshed to the first application layer will not be displayed in the user interface, causing the touch track in the overlapping area to suddenly disappear in the user interface, thereby reducing the user experience.

[0131] Therefore, in order to improve the above problem, the display device 200 adjusts the transparency of the second application layer after detecting the second application. The second application layer is an application layer for displaying the second application. Fig.17 As shown, the display device 200 also sets the display level of the second application layer to be higher than the display level of the acceleration layer. In this way, when inputting a touch track, the user can observe the display content of the target touch track in the acceleration layer in real time through the second application layer located above the acceleration layer to perceive the touch process of the target touch track; after the display content of the acceleration layer is released, the user can still observe the display content of the target touch track in the first application layer through the second application layer above the acceleration layer to perceive the touch result of the target touch track. At this time, the display levels in the display device 200 are second application layer-acceleration layer-first application layer from high to low.

[0132] For example, Fig.18 As shown, the application display range of the first application is represented as "1", and the application display range of the second application is represented as "2". The second application is displayed in a floating manner on the upper layer of the first application, and the second application layer of the second application is completely opaque. The user draws the target curve G in the display device 200, and the touch starting point of the curve G is G1. The display device 200 reads the application display range identifier "1" at the position of G1, and determines that the curve G is the touch track acting on the first application. When the user draws to the position of G2, the application display range identifier read by the display device 200 is changed to "2", and the display device 200 determines the corresponding second application according to the application display range identifier "2", and then adjusts the transparency of the second application layer, and at the same time sets the display level of the second application layer to be higher than the display level of the acceleration layer, presenting the following. Fig.18 The display effect shown.

[0133] In order to achieve the transparent effect of the application layer, in some embodiments, a color value is set in the window parameters of the application layer. The color value consists of four parts: red (Red), green (Green), blue (Blue) and transparency (Alpha). For example, the color value in the window parameter can be #AARRGGBB. Among them, AA represents transparency, RR represents red value, GG represents green value, and BB represents blue value.

[0134] In some embodiments, taking hexadecimal as an example, the value range of Alpha is 0 to 255. Among them, 0 represents complete transparency and 255 represents complete opacity. In order to facilitate comparison of Alpha, the display device 200 can map the Alpha value to the range of [0,1], and identify the size of the Alpha value by a decimal value. For example, when the Alpha value is divided by 255 and the Alpha value is 128, the mapped decimal value is 128 / 255≈0.502.

[0135] It can be understood that the smaller the Alpha value, the higher the corresponding transparency value and the more transparent the color; the larger the Alpha value, the lower the corresponding transparency value and the more opaque the color.

[0136] Therefore, if Fig.19 As shown, in some embodiments, when the display device 200 adjusts the transparency of the second application layer, it reads the window parameters of the second application layer. The window parameters include color values. The transparency of the color value is parsed to obtain the initial transparency value of the second application layer. Since the display device 200 sets the display level of the second application layer to be higher than the display level of the acceleration layer, in order to facilitate the user to observe the display content of the acceleration layer and the first application layer through the second application layer, the display device 200 adjusts the transparency of the window parameters to a target value. The target value is higher than the initial transparency value.

[0137] For example, the initial transparency value of the second application layer is 0%, the Alpha value is 255, and the second application layer is an opaque application layer. When adjusting the transparency of the second application layer, the display device 200 can adjust the transparency value of the second application layer to 50%, that is, adjust the Alpha value in the second application layer window parameter to 127.

[0138] Furthermore, since the user inputs a touch track to the display device 200, the touch track may also have a transparent color. In order to ensure the display effect of the touch track, in some embodiments, the display device 200 also detects the track color value of the target touch track. The transparency of the track color value is analyzed to obtain the target transparency value of the target touch track. A target value having a value higher than the target transparency value is then generated. In this way, the transparency of the second application layer is adjusted to be lower than the transparency of the target touch track, so that the target touch track can be better displayed in the user interface through the second application layer.

[0139] For example, the second application is displayed in a floating manner on the upper layer of the first application, the display level of the acceleration layer is higher than the first application layer and the second application layer, and the display level of the second application layer is higher than the first application layer. The user inputs a target touch track with a transparency value of 50% to the display device 200 based on the first application. When the target touch track passes through the overlapping area of ​​the first application and the second application, the display device 200 generates a target value of 60% higher than the transparency value of 50%, and adjusts the transparency value of the second application layer to 60%, and sets the display level of the second application layer to be higher than the display level of the acceleration layer.

[0140] Since the touch track input by the touch display application also includes a touch event for erasing the touch track, when executing the erase event, the acceleration layer of the display device 200 needs to copy the bitmap background of the application window to the display area of ​​the touch track to achieve an erase effect.

[0141] Therefore, in some embodiments, the display device 200 also detects the event type of the target touch track. When the event type is an erase event, the bitmap background of the first application is obtained, and the bitmap background of the first application is copied to the touch unit. Since the touch unit is the display unit acted upon by the target touch track in the acceleration layer, the display level of the second application layer is set to be higher than the display level of the acceleration layer, and the display content in the acceleration layer will not block the second application layer, thereby ensuring the display effect of the second application. The user can also observe the occlusion of the first application layer by the bitmap background in the acceleration layer through the second application layer with transparency, thereby achieving the effect of erasing.

[0142] For example, Fig. 20 As shown, the second application is displayed in a floating manner on the upper layer of the first application. The display level of the acceleration layer is higher than the first application layer and the second application layer. The display level of the second application layer is higher than the first application layer. The user inputs the target touch track of the erase event to the display device based on the first application. The touch track of the erase event is shown in FIG. Fig. 20 As shown in the dashed part. In response to the erase event of the target touch track, the display device 200 copies the bitmap background of the first application to the action position of the target touch track in the acceleration layer. When the touch point of the target touch track is located in the application window area of ​​the second application, the display device 200 increases the transparency of the second application layer and sets the display level of the second application layer to be higher than the display level of the acceleration layer. In this way, since the display level of the acceleration layer is lower than the second application layer, the display content of the acceleration layer will not block the display content of the second application layer; at the same time, through the second application layer with transparency, the user can still observe the blocking effect of the acceleration layer on the first application, thereby achieving the erasing effect on the first application layer.

[0143] In some embodiments, the erase event is based on the erase control input, and the erase control is displayed in the acceleration layer based on specific bitmap data. When the display device 200 enters the erase mode, the display device 200 generates the bitmap data of the erase control, and refreshes the bitmap data of the erase control to the acceleration layer according to the target touch track, so that it is displayed in the drawing interface. When the user inputs the erase event, the erase control moves following the position of the touch point.

[0144] Since the display device 200 needs to copy the bitmap background to the corresponding position when the user inputs the erase event, in order to ensure the user's interactive experience, in some embodiments, the display device 200 also responds to the erase event and always refreshes the bitmap data of the erase control to the acceleration layer according to the touch track of the erase event, that is, the bitmap data of the erase control is refreshed in real time to the position where the touch track acts on the acceleration layer. In this way, after the display device 200 enters the erase mode, the erase control can be always displayed in the user interface of the display device 200, so that the user can perceive the process of the erase operation in real time, thereby improving the user's interactive experience.

[0145] Based on the above display device 200, some embodiments of the present application also provide a display method of an acceleration layer, such as Fig.14 As shown, the method comprises the following procedural steps:

[0146] S1: in response to a target touch track input by a user based on a first application, refreshing the target touch track to an acceleration layer and a first application layer, and detecting a touch unit in the acceleration layer; the first application layer is an application layer for displaying the first application, and the touch unit is a display unit acted upon by the target touch track;

[0147] S2: Reading the application display range identifier of the touch unit;

[0148] S3: when the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit, the first identifier being the application display range identifier of the first application;

[0149] S4: adjusting the transparency of a second application layer, and setting a display level of the second application layer to be higher than a display level of the acceleration layer, the second application layer being an application layer for displaying a second application.

[0150] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a display method of an acceleration layer. The method can respond to a target touch track input by a user based on a first application, refresh the target touch track to the acceleration layer and the first application layer used to display the first application, and detect the touch unit in the acceleration layer. The touch unit is the display unit acted upon by the target touch track. Read the application display range identifier of the touch unit. When the application display range identifier of the touch unit is not the first identifier, detect the second application associated with the touch unit. Adjust the transparency of the second application layer, and set the display level of the second application layer to be higher than the display level of the acceleration layer. The second application layer is an application layer used to display the second application. The method can display the touch track acting on the overlapping part in the user interface through the upper application layer by adjusting the transparency and display level of the application layer, so that the touch display effects between multiple applications do not affect each other, thereby improving the touch display effects between multiple applications.

[0151] In addition, in order to prevent the touch display effects between multiple applications from affecting each other, in some embodiments, the display device 200 can also detect the target position of the touch starting point in the target touch trajectory in response to the target touch trajectory, and obtain the target application display range identifier of the target position. Query the application display area of ​​the acceleration layer according to the target application display range identifier. Among them, the application display area is a display unit with an application display range identifier of the first application. For example, when the application display range associated with the first application is represented as "1", all display units in the acceleration layer with an application display range identifier of "1" constitute the application display area of ​​the first application.

[0152] After the application display area is queried, the target touch track is refreshed to the application display area, and the target touch track is refreshed to the first application layer according to the application display range identifier. That is, when the acceleration layer refreshes the target touch track, the display device 200 can only refresh the target touch track to the application display area corresponding to the first application, so that it does not interfere with the display effects of other applications.

[0153] In some embodiments, when the display device 200 refreshes the target touch track to the application display area, the action position of the touch driving point is detected. When the action position is within the application display area, the touch driving point is refreshed to the application display area of ​​the acceleration layer; when the action position is not within the application display area, the touch driving point is stopped from being refreshed to the acceleration layer.

[0154] Obviously, when there are overlapping parts of application windows, the display area of ​​the application windows actually exists at the overlapping part. Stopping the touch driving points located outside the application display area from being refreshed to the acceleration layer can ensure that the touch track is refreshed normally in the application layer and prevent it from interfering with the display effects of other applications. Fig.21As shown in the figure, the application window of application B is suspended and displayed on the upper layer of the application window of application A. The user draws Fig.21 The curve C shown in the figure has the touch starting point C1. Through the position of C1, it is determined that the application display area affected by curve C is the application display area of ​​application A, and the display device 200 stops refreshing the touch driving point located in the application display area of ​​application B to the acceleration layer, and refreshes the complete touch track of curve C to the application layer of application A normally. Since the display level of the acceleration layer is higher than the application layer, the overlapping touch track will not be presented in the user interface in the end, so that curve C will not interfere with the display effect of application B. And since the complete touch track has been refreshed in the application layer of application A, when the application window of application B is moved away from the upper layer of the application window of application A, as shown in FIG. Fig.21 As shown, the application window of application A can still normally display the complete touch track of curve C.

[0155] However, when there is no overlapping part between application windows, the touch driving points acting outside the application display area have exceeded the display area of ​​the application window, so they cannot be refreshed to the application layer normally, resulting in the display device 200 being unable to provide feedback on this part of the touch driving points, thereby reducing the user's interactive experience.

[0156] Therefore, in order to improve the user's interactive experience, in some embodiments, when the action position is not located in the application display area, the display device 200 also detects the window overlap state of the first application. When the window overlap state is a non-overlap state, the touch point coordinates of the touch drive point are obtained, and the touch point coordinates are converted into registration coordinates according to a preset coordinate mapping relationship. Among them, the registration coordinates are the boundary coordinates of the application display area. The touch drive point is refreshed to the application layer of the first application in the acceleration layer according to the registration coordinates. In this way, the display device 200 can convert the touch drive point located outside the application display area to the edge of the application display area through coordinate conversion, so that it is normally displayed in the acceleration layer and the application layer.

[0157] For example, Fig. 22 As shown, the application windows of application A and application B are displayed in a split screen on the display device 200, the left half of the screen is the application display area of ​​application A, and the right half of the screen is the application display area of ​​application B. The user draws on the display device 200 Fig. 22 The curve G shown in FIG. 1 has a starting touch point G1. The application display area affected by the curve G is determined to be the left half of the screen through the position of G1. The touch driving point coordinates beyond the left half of the screen are converted to coordinates at the edge of the left half of the screen, and are refreshed in the acceleration layer and the application layer of application A to present Fig. 22 The display effect shown.

[0158] For an erase event, in some embodiments, when the action position is not located in the application display area, the display device 200 further detects the event type of the target touch track. If the event type is an erase event, the touch drive point is converted into transparent bitmap data, and the transparent bitmap data is refreshed to the action position of the acceleration layer.

[0159] For example, Fig.23 As shown, the application window of application B is displayed floating on the upper layer of the application window of application A. The user inputs an erase event to the display device 200 with the touch start point in the application display area of ​​application A. The touch track corresponding to the erase event is shown in FIG. Fig.23 As shown by the dotted line in . For the touch driving points located in the application display area of ​​application A, the display device 200 copies the bitmap background of application A to the corresponding position of the touch driving points; for the touch driving points located in the application display area of ​​application B, the display device 200 performs transparent bitmap processing on these touch driving points. After the transparent bitmap is refreshed to the acceleration layer, it will not affect the display content of the application window, so that the application window of application B is not affected by the erase event and always maintains the original display effect.

[0160] Alternatively, in some embodiments, when the action position is not located in the application display area, the display device 200 detects the event type of the target touch track. If the event type is an erase event, the application display range identifier of the action position in the acceleration layer is detected. Then, the second application is queried according to the application display range identifier of the action position. The second application is an application associated with the application display range identifier of the action position. The bitmap background of the second application is obtained, and the bitmap background is copied to the action position of the acceleration layer.

[0161] For example, the application window of application B is displayed floating on the upper layer of the application window of application A. The user inputs an erase event to the display device 200 with the touch start point in the application display area of ​​application A. The touch track corresponding to the erase event is as follows: Fig.23 For a touch driving point located in the application display area of ​​application A, the display device 200 copies the bitmap background of application A to the corresponding position of the touch driving point; for a touch driving point located in the application display area of ​​application B, the display device 200 copies the bitmap background of application B to the corresponding position of the touch driving point, and the application window of application B will not be affected by the erase event and always maintain the original display effect.

[0162] It can be seen from the above technical solutions that the display device 200 provided in some embodiments of the present application can also detect the target position of the touch starting point in the target touch trajectory in response to the target touch trajectory input by the user based on the first application, and obtain the application display range identifier of the target position. Query the application display area of ​​the acceleration layer according to the application display range identifier of the target position. Then refresh the target touch trajectory to the application display area of ​​the acceleration layer, and refresh the target touch trajectory to the first application layer according to the application display range identifier. The method refreshes the acceleration layer based on the application display area of ​​each application, and refreshes the touch trajectory to the application layer corresponding to the application according to the application display range identifier. On the basis of speeding up the touch display speed of the application, the touch displays of each application will not affect each other, thereby improving the touch display effect between multiple applications.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0164] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A display, configured to display a drawing interface, the drawing interface comprising a first application window and a second application window, the first application window being a window displayed based on a first application, the second application window being a window displayed based on a second application, the first application window and the second application window being displayed through different application layers; the display being connected to a touch interaction module, the touch interaction module being used to detect a touch track input by a user; when displaying the drawing interface, the display also displays the touch track through an acceleration layer, the initial display level of the acceleration layer being higher than the display level of the application layer, and the display unit of the acceleration layer being provided with an application display range identifier; The controller is configured as: In response to a target touch track input by a user based on a first application, the target touch track is refreshed to the acceleration layer and the first application layer, and a touch unit in the acceleration layer is detected; the first application layer is an application layer for displaying the first application, and the touch unit is a display unit acted upon by the target touch track; Reading an application display range identifier of the touch control unit; When the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit, the first identifier being the application display range identifier of the first application; The transparency of the second application layer is adjusted, and the display level of the second application layer is set to be higher than the display level of the acceleration layer, wherein the second application layer is an application layer for displaying a second application.

2. The display device according to claim 1, characterized in that The touch track includes a touch termination point, and the touch termination point includes a display unit acted upon by a lifting action; the controller executes refreshing of the target touch track to the acceleration layer and the application layer of the first application, and is configured as follows: Generate bitmap data of the target touch trajectory; Copying the bitmap data to the touch unit, and monitoring the touch termination point of the target touch track; When the touch termination point of the target touch track is detected, the bitmap data is copied to the application layer of the first application.

3. The display device according to claim 1, characterized in that The controller performs adjustment of transparency of the second application layer and is configured to: Reading a window parameter of the second application layer, wherein the window parameter includes a color value; parsing the transparency of the color value to obtain an initial transparency value of the second application layer; The transparency of the window parameter is adjusted to a target value, where the target value is higher than the initial transparency value.

4. The display device according to claim 3, characterized in that The controller is also configured to: Detecting a track color value of the target touch track; Analyze the transparency of the track color value to obtain a target transparency value of the target touch track; The target value is generated to be higher than the target transparency value.

5. The display device according to claim 1, characterized in that The controller is also configured to: Detecting the event type of the target touch track; When the event type is an erase event, obtaining a bitmap background of the first application; The bitmap background of the first application is copied to the touch control unit.

6. The display device according to claim 5, characterized in that When the event type is an erase event, the controller is further configured to: Generate the bitmap data of the erase control; According to the target touch track, the bitmap data of the erase control is refreshed to the acceleration layer.

7. The display device according to claim 1, characterized in that The controller is also configured to: In response to a window dynamic event of a target application, detecting a window change area of ​​the target application, wherein the window dynamic event includes a window change event, a window start event, and a window close event; Querying a target unit according to the window change area, wherein the target unit includes a display unit located in the window display area in the acceleration layer; The application display range identifier of the target unit is updated based on the window display level.

8. The display device according to claim 7, characterized in that The controller executes updating of the application display range identifier of the target unit based on the window display level, and is configured to: Obtaining a window display level of a change area application, wherein the change area application is an application whose application window is displayed in the window change area; Querying the identification application of the target unit according to the window display level, the identification application is the change area application with the highest window display level at each target unit position; If there is an identification application at the target unit location, updating the application display range identifier of the target unit to the application display range identifier of the identification application; If there is no identified application at the target unit location, the application display range identifier of the target unit is updated to an initial identifier.

9. The display device according to claim 1, characterized in that The controller is configured to: Monitoring the application type corresponding to the application window in the drawing interface; If the drawing interface does not include a touch display application, releasing data resources of the acceleration layer; If the drawing interface includes a touch display application, the data resource of the acceleration layer is loaded to display the display content of the acceleration layer through the data resource.

10. A method for displaying an acceleration layer, characterized in that: Applied to the display device according to any one of claims 1 to 9, the method comprising: In response to a target touch track input by a user based on a first application, the target touch track is refreshed to an acceleration layer and a first application layer, and a touch unit in the acceleration layer is detected; the first application layer is an application layer for displaying the first application, and the touch unit is a display unit acted upon by the target touch track; Reading an application display range identifier of the touch control unit; When the application display range identifier of the touch unit is not the first identifier, detecting a second application associated with the touch unit, the first identifier being the application display range identifier of the first application; The transparency of the second application layer is adjusted, and the display level of the second application layer is set to be higher than the display level of the acceleration layer, wherein the second application layer is an application layer for displaying a second application.

Images