Display device and touch selection method

Abstract

Provided in the embodiments of the present disclosure are a display device and a touch selection method. The method comprises: in response to a target selection instruction input by a user, detecting the object types of target element objects indicated by the target selection instruction; if the target element objects are non-overlay-type objects, drawing an identification object on the basis of an adding sequence of the target element objects, and displaying the identification object; and if the target element objects are overlay-type objects, determining a drawing sequence on the basis of overlapping areas between the target element objects so as to draw an identification object, and displaying the identification object, the identification object being generated by superposing element objects formed by bolding the target element objects and the target element objects. In the present method, a drawing sequence is determined on the basis of an adding sequence of target element objects so as to perform outlining identification on non-overlay-type objects, and a drawing sequence is determined on the basis of overlapping conditions of overlay-type objects so as to perform outlining identification on the overlay-type objects, thus highlighting the effect of selecting each element object, and improving the accuracy of touch selection.

Description

Display device and touch selection method

[0001] Cross-reference to Related Applications

[0002] The present disclosure claims priority to Chinese application No. 202410474660.7, filed on April 19, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of display devices, and in particular, to a display device and a touch selection method. BACKGROUND

[0004] A display device refers to a terminal device capable of outputting a specific display screen, which can be a smart television, a whiteboard, a communication terminal, a smart advertising screen, a projector, or the like. Taking a smart television as an example, a smart television is a television product based on Internet application technology, with an open operating system and a chip, an open application platform, and bidirectional man-machine interaction function, integrating audio, video, entertainment, data, and other functions to meet the diversified and personalized needs of users.

[0005] For a display device supporting touch interaction operation, a whiteboard application can be run, and writing or drawing functions can be implemented based on the whiteboard application according to touch actions input by a user. When the user needs to perform editing operations such as moving, zooming, rotating, or the like on element objects in a whiteboard interface, the corresponding element objects need to be selected in a selection mode of the whiteboard application, and then the selected element objects are subjected to the above editing operations. The display device can identify the element objects selected by the whiteboard application in the selection mode through the way of outlining, so as to prompt the user which element objects are in the selected state, thereby facilitating the user to perform the editing operations.

[0006] However, if the selected element objects overlap, the lower-level element objects are covered by the higher-level element objects, and the corresponding outline effects are also covered by the higher-level element objects. In this way, when the user performs editing operations on the higher-level element objects, the user cannot intuitively perceive the selected state of the lower-level element objects, which may cause the lower-level element objects to be mistakenly selected or missed, thereby causing the accuracy of touch selection to decrease and affecting the user experience. SUMMARY

[0007] In a first aspect, the embodiments of the present disclosure provide a display device, which can include: a display configured to display a user interface; at least two element objects included in the user interface and added in response to touch instructions; a touch interaction module connected with the display, configured to receive touch instructions input by a user; a memory configured to store computer instructions and data associated with the display device; and at least one processor connected with the display and the memory, configured to execute the computer instructions to cause the display device to perform the following operations: in response to target selection instructions input by the user, detecting an object type of a target element object, the target element object being an element object indicated in the target selection instructions, the target selection instructions indicating at least two element objects; if the target element object is a non-overlapping class object, drawing a first identification object according to a first order and controlling the display to display the first identification object, the first order being determined based on an adding order of the non-overlapping class object, and the first identification object being generated by superimposing element objects obtained by using the non-overlapping class object and by performing bold processing on the non-overlapping class object; and if the target element object is an overlapping class object, drawing a second identification object according to a second order and controlling the display to display the second identification object, the second order being generated based on overlapping areas between the overlapping class objects, and the second identification object being generated by superimposing element objects obtained by using the overlapping class object and by performing bold processing on the overlapping class object.

[0008] In a second aspect, the embodiments of the present disclosure also provide a touch selection method, which can be applied to the display device provided in the first aspect, and the method can include: in response to target selection instructions input by a user, detecting an object type of a target element object, the target element object being an element object indicated in the target selection instructions, the target selection instructions indicating at least two element objects; if the target element object is a non-overlapping class object, drawing a first identification object according to a first order and displaying the first identification object, the first order being determined based on an adding order of the non-overlapping class object, and the first identification object being generated by superimposing element objects obtained by using the non-overlapping class object and by performing bold processing on the non-overlapping class object; and if the target element object is an overlapping class object, drawing a second identification object according to a second order and displaying the second identification object, the second order being generated based on overlapping areas between the overlapping class objects, and the second identification object being generated by superimposing element objects obtained by using the overlapping class object and by performing bold processing on the overlapping class object. BRIEF DESCRIPTION OF DRAWINGS

[0009] FIG. 1 is a schematic diagram of an operating scenario between a display device and a control device according to some embodiments;

[0010] FIG. 2 is a schematic diagram of a hardware configuration of a display device according to some embodiments;

[0011] FIG. 3 is a diagram of a software configuration of a display device according to some embodiments;

[0012] FIG. 4 is a diagram of operation parameters of touch interaction data according to some embodiments;

[0013] FIG. 5 is a diagram of a whiteboard interface according to some embodiments;

[0014] FIG. 6 is a diagram of a display system structure according to some embodiments;

[0015] FIG. 7 is a diagram of an acceleration layer and an OSD layer according to some embodiments;

[0016] FIG. 8 is a diagram of an element object according to some embodiments;

[0017] FIG. 9 is a diagram of an element object in a whiteboard interface according to some embodiments;

[0018] FIG. 10 is a diagram of a stroke effect according to some embodiments;

[0019] FIG. 11 is a diagram of a stroke effect of a non-overlay type object according to some embodiments;

[0020] FIG. 12 is a diagram of a stroke effect of a plurality of element objects according to some embodiments;

[0021] FIG. 13 is a diagram of another stroke effect of a plurality of element objects according to some embodiments;

[0022] FIG. 14 is a diagram of a flow of a touch selection method according to some embodiments;

[0023] FIG. 15 is a diagram of a flow of a stroke method and a stroke cancellation method in a selection process according to some embodiments;

[0024] FIG. 16 is a diagram of a stroke effect in a selection process according to some embodiments;

[0025] FIG. 17 is a diagram of another stroke effect in a selection process according to some embodiments;

[0026] FIG. 18 is a diagram of another stroke effect in a selection process according to some embodiments;

[0027] FIG. 19 is a diagram of a stroke cancellation effect in a selection process according to some embodiments;

[0028] FIG. 20 is a diagram of a drawing order of an identification object of a non-overlay type object and an overlay type object according to some embodiments;

[0029] FIG. 21 is a schematic diagram of a stroking effect when multiple-element objects overlap after optimization, according to some embodiments. DETAILED DESCRIPTION

[0030] The embodiments will be described in detail with examples shown in the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of implementations consistent with some aspects of the present disclosure as detailed in the claims.

[0031] It should be noted that the brief description of terms in the present disclosure is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present disclosure. Unless otherwise specified, these terms should be understood according to their ordinary and customary meanings.

[0032] The terms "first", "second", "third", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar or like objects or entities, and do not necessarily mean a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0033] The terms "include" and "have" and any variations thereof are intended to cover but not exclusively include, for example, a product or device that includes a list of components without being limited to all the components clearly listed, but can include other components not clearly listed or inherent to such products or devices.

[0034] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware and / or software code capable of performing a function associated with that element.

[0035] In the embodiments of the present disclosure, the display device 200 generally refers to a device with picture display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart television, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0036] FIG. 1 is a schematic diagram of an operating scenario between a display device and a control device, according to some embodiments. As shown in FIG. 1, a user can operate the display device 200 through the mobile terminal 300 and the control device 100. Among them, the control device 100 can be used to convert the user's operation actions into control instructions that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote controller, a stylus, a handle, etc.

[0037] The mobile terminal 300 can serve as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also serve as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install a software application with the display device 200, implement connection communication through a network communication protocol, and achieve the purpose of one-to-one control operation and data communication. The mobile terminal 300 can also display audio and video content on the display device 200 to achieve a synchronous display function.

[0038] In some embodiments, the display device 200 can communicate data with the server 400 through various communication methods. The display device 200 can be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0039] In some embodiments, the display device 200 can provide a broadcast receiving television function, and can additionally provide a smart network television function supporting computer functions, including but not limited to network television, smart television, Internet protocol television (IPTV), etc.

[0040] FIG. 2 is a hardware configuration block diagram of the display device 200 shown in FIG. 1. As shown in FIG. 2, the display device 200 can include at least one of a tuning demodulator 210, a communication device 220, a detector 230, a device interface 240, a processor 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0041] In some embodiments, the detector 230 can be used to collect signals of an external environment or interaction with the outside. For example, the detector 230 includes a light receiver for collecting ambient light intensity sensors.

[0042] In some embodiments, the detector 230 can include an image collector, such as a camera of various sizes, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. The detector 230 can also include a sound collector, such as a microphone, which can be used to receive external sounds.

[0043] In some embodiments, the display 260 can include display function components for presenting a picture, and driving components for driving image display. The display 260 can be used to receive image signals output from the processor 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface, as well as a user control UI interface, etc.

[0044] In some embodiments, the display 260 can support touch interaction, that is, a touch interaction module can be provided on the display device 200, which can be a built-in module of the display 260 or an external module of the display device 200 / display 260. For example, in order to realize touch interaction operation, a touch screen (TP) can be provided on the display 260 of the display device 200, which can receive touch signals input by fingers, contacts, styluses, etc., and perform touch interaction response according to a pre-set interaction strategy. Of course, the display device 200 can also be a touch integrated screen.

[0045] For the display device 200 with an external touch interaction module, a terminal, device, component with touch function can be connected through the device interface 240 of the display device 200. For example, the USB or USB-C interface of the display device 200 is connected to a touchpad, which can receive touch data input by the user in real time and transmit the touch data to the display device 200 through the USB channel. The display device 200 can perform interaction response in combination with touch application according to the received touch data, so that the display device 200 supports touch interaction operation.

[0046] In some embodiments, the communication device 220 is a component for communicating with external devices or servers 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 containing WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 containing Bluetooth function.

[0047] In some embodiments, the communication device 220 can connect the display device 200 with external devices or servers 400 through wireless or wired connection. The wired connection can connect the display device 200 with external devices through data lines, interfaces, etc. The wireless connection can connect the display device 200 with external devices through wireless signals or wireless networks. The display device 200 can directly establish a connection relationship with external devices, or indirectly establish a connection relationship through gateways, routers, connection devices, etc.

[0048] In some embodiments, the processor 250 can include at least one of a central processor, a video processor, an audio processor, a graphics processor, a power processor, and a first interface to an n-th interface for input / output. The processor 250 can control the operation of the display device and respond to the user's operation through various software control programs stored in the memory, or directly control the overall operation of the display device 200 through the processor 250.

[0049] In some embodiments, the processor 250 and the tuner-demodulator 210 can be located in different parts of the device, i.e., the tuner-demodulator 210 can also be located in an external device of the main device where the processor 250 is located, such as an external set-top box, etc.

[0050] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface (GUI).

[0051] In some embodiments, the audio output device 270 can be a native loudspeaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 can also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0052] In some embodiments, the user input interface 280 can be used to receive instructions from the user input.

[0053] In order to perform user interaction, in some embodiments, the display device 200 can run an operating system. The operating system is a computer program used to manage and control hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0054] It should be noted that the operating system can be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0055] The operating system can be divided into different modules or levels according to the functions implemented, for example, as shown in FIG. 3, the operating system can be divided into four layers, from top to bottom, the application layer (referred to as "application layer" for short), the application framework layer (referred to as "framework layer" for short), the system library layer and the kernel layer.

[0056] In some embodiments, the application layer can be used to provide services and interfaces to the applications so that the display device 200 can run the applications and interact with the user based on the applications. At least one application can be run in the application layer, which can be a Window program, a system setting program, a clock program, etc. provided by the operating system, or an application developed by a third party developer. In a specific implementation, the applications in the application layer are not limited to the above examples.

[0057] The framework layer provides the application programming interface (API) and programming framework for the applications. The application framework layer includes some pre-defined functions. The application framework layer is equivalent to a processing center that decides which application in the application layer to act. The application can access the resources in the system and obtain the services of the system through the API interface in the execution.

[0058] Continuing as shown in FIG. 3, the application framework layer in the embodiments of the present disclosure can include a view system, managers, a content provider, etc.

[0059] The view system can be designed and implemented to interface and interact with the application, and the view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: an activity manager for interacting with all the activities running in the system; a location manager for providing the system services or the applications with access to the system location services; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of the notification messages; and a window manager for managing the icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0060] In some embodiments, the activity manager is used to manage the life cycle of each application and the general navigation back function, such as controlling the exit, opening, back, etc. of the application. The window manager is used to manage all the window programs, such as obtaining the size of the display screen, determining whether there is a status bar, locking the screen, intercepting the screen, controlling the display window change, such as reducing the display window, shaking the display, twisting the display, etc.

[0061] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions implemented by the framework layer.

[0062] In some embodiments, the kernel layer is a functional layer between the hardware and the software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, memory management, etc. For example, as shown in FIG. 3, the kernel layer can be configured with hardware drivers. The drivers contained in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply driver, etc.

[0063] It should be noted that the above examples are only a simple division of the functions of the operating system, and do not constitute a limitation on the specific operating system form of the display device 200 in the embodiments of the present disclosure. According to the function of the display device, the type of operating system, and other factors, the number and specific type of layers contained in the operating system can be in other forms.

[0064] For the display device 200 supporting touch interaction operation, the touch interaction module can receive the touch interaction data input by the user. According to the interaction principle of the touch interaction module, the touch interaction module can detect the related parameters in the touch action. For example, the display 260 screen of the display device 200 is provided with a capacitive screen touch layer. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touch position will change. At this time, the touch interaction module can record the position of the capacitance change, and combine the timer in the processor to record the time when the capacitance change occurs, to generate the touch interaction data.

[0065] In some embodiments, the physical signals detected by the touch interaction module need to be converted before being read by the processor 250 of the display device 200. For example, the analog signals of the capacitance change need to be converted into digital signals before being transmitted to the processor 250 of the display device 200. In order to adapt to the screen display process of the display device 200, the touch data detected by the touch interaction module also needs to be mapped in coordinates, so that the position point corresponding to the touch data can have a corresponding relationship with the pixel point in the display 260 or the user interface of the display device 200. For example, the coordinate position detection accuracy of the touch interaction module is 65536x65536, and the resolution of the display 260 of the display device 200 is 3840x2160. Therefore, after the touch interaction module detects the touch data, the touch point position in the width and height directions can be mapped in coordinates according to the proportions of 65536 / 3840 and 65536 / 2160, so that the display device 200 can perform related operation responses based on the mapped touch point position coordinates.

[0066] The touch data can include different operation parameters according to the input mode of the user. For example, as shown in FIG. 4, the touch data can include the touch time, touch position, touch event type, and touch point number of the touch operation. In a touch interaction process, the touch event types that can be detected by the touch interaction module include the down event, move event, and up event of the touch action. Corresponding to the entire touch trajectory, the starting point, trajectory point, and ending point can be formed. Each touch event is recorded with the corresponding touch time and touch position.

[0067] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, touch point numbers, and touch event types, that is, the touch input gestures of the user can be detected, and different interaction responses can be executed for different input gestures. For example, when the touch point number in the touch data detected by the touch interaction module is 1, the touch event type only includes the down event and the up event, the time interval between the down event and the up event is less than the single-click event time threshold, and the touch position change distance between the down event and the up event is less than the jitter distance threshold, it can be determined that the touch input gesture of the current user is a click gesture. Then, based on the click gesture, the display device 200 executes the operation of starting an application program according to the icon in the area where the touch position corresponding to the click gesture is located.

[0068] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch time, touch position, touch point number, and touch event type, such as a double-click gesture, a slide gesture, a long-press gesture, a multi-finger click gesture, a multi-finger slide gesture, and the like. Different interaction gestures can derive different touch parameters, such as a slide distance, a slide speed, a slide direction, a slide track shape, and the like, which can be used to perform different interaction controls.

[0069] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection according to the dynamic change rules of the touch time, touch position, touch point number, and touch event type. For example, when the touch interaction module detects that the touch data input by the user includes multiple touch points, and the touch positions of each touch point are close to each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grab gesture. Based on complex gesture detection, the display device 200 can perform more interaction responses. For example, when a multi-finger slide touch gesture is detected and it is identified that the touch gesture is a grab operation on an image view, the display device 200 can respond to the grab gesture and perform a control action of moving the image display position.

[0070] In some embodiments, the touch gesture can also be combined with the touch position to form a touch interaction operation based on a specific position. For example, when the touch interaction module detects that the down event position in the touch operation process is at the top of the display 260, the touch point position corresponding to the move event moves downward, and when the up event is detected, the display device 200 can determine that the current touch gesture is a downward swipe from the top of the screen. Therefore, the display device 200 can respond to the touch gesture and control the display 260 to present a drop-down list.

[0071] Some touch interaction operation gestures can be used to perform interaction actions based on the operating system of the display device 200, which are referred to as global gestures. For example, the operation of swiping down from the top of the screen to call a drop-down list in the above example of the display device 200. Some touch interaction operation gestures can be used to perform interaction actions based on a specific touch application. For example, when the display device 200 runs a whiteboard application, a slide touch operation input by the user can be used to draw a track line on the whiteboard.

[0072] The touch application can be a system application or a third-party application developed based on a specific display device type and user needs. The touch application can be installed in the display device 200 and controlled to be started and run by the user. For example, the user can control the display device 200 to run a whiteboard application by clicking a "whiteboard" application icon in the application interface of the display device 200.

[0073] It should be noted that the whiteboard application described above can be a standalone application, can be an associated application, and can also be a functional module in a touch application. Different forms of whiteboard applications can have different starting modes. For example, a user can start a standalone whiteboard application through an application interface of the display device 200; the user can also start the whiteboard application in association by performing a "whiteboard" operation on a conference application; and the user can also control the conference application to start the whiteboard functional module by clicking a "whiteboard" icon on the conference application.

[0074] Different forms of whiteboard applications can also have different whiteboard interface forms after being started. For example, when a user starts a whiteboard application in the form of an associated application, the display device 200 can automatically enter a dual-screen (or split-screen) mode. In the dual-screen (or split-screen) mode, the display device 200 can display a user interface through two display areas, that is, a main screen displays an original application (such as a conference application) interface, and a sub-screen displays a whiteboard interface.

[0075] After the whiteboard application is running, the display device 200 can be switched to display the whiteboard interface. As shown in FIG. 5, the whiteboard interface can include drawing tool controls for drawing operations, such as pen-shaped, graphic, line-shaped, color, erasing, clearing, selecting, and the like. A user can click any control according to specific drawing needs to draw a figure.

[0076] As shown in FIG. 6, the display device 200 can render and display a drawing picture in real time through an image generation (Image Producer) module, an image rendering (surface flinger) module, a composition (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module, and a display (screen).

[0077] During the display of the drawing picture, a graphics interface (openGLES) of the image generation module can be connected with the whiteboard application to generate drawing image content and transmit the image content to the image rendering module. The image rendering module is used to control the display 260 to display content, that is, to render image content through a user interaction layer (UI layer) or a video layer according to the image content. In order to obtain better display effects, the image content of the UI layer can be rendered through a GPU component, so as to form a picture frame for display in the composition module. The composition module can form picture content through software or hardware, for example, the composition module can be a hardware composition (Hardware Composer, HWC) module.

[0078] The picture frames can be stored in the frame buffer and the video buffer respectively in the synthesis module, and the picture content can be formed in the corresponding layers by the video process module. That is, the image frames generated by the video playing process form the picture content through the video plane, while the image frames of other image pictures (such as touch track, graphics, etc.) form the picture content through the primary OSD plane. Finally, the display device 200 forms the final picture through the blend unit of the video process module and displays it on the display 260.

[0079] Since the whiteboard application presents the user's touch track in real time according to the user's touch operation, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. For this purpose, in some embodiments, the display device 200 can present a real-time drawing effect by setting an acceleration layer. That is, in the video process module of the display device 200, an external OSD plane can be additionally set on the basis of the primary OSD plane and the video plane. The external OSD plane can be configured to have different layer characteristics according to specific display effect requirements. For example, the bitmap refresh mode of the external OSD plane can be modified so that the external OSD plane is refreshed in real time according to the area, thereby improving the display speed of the graphics. At this time, the external OSD plane can be used as an acceleration layer in the scene of high dynamic display effect.

[0080] Taking the display device 200 running the whiteboard application as an example, in order to improve the real-time response speed of the line drawing process, the touch track can be drawn in real time on the external OSD plane (acceleration layer) after detecting the user's touch operation. As shown in FIG. 7, the display level of the acceleration layer can be higher than that of other layers, and the line drawn by the acceleration layer can be updated to other layers when the user completes the touch input, thereby reducing the input delay and improving the real-time display effect of the whiteboard application drawing process.

[0081] In some embodiments, the external OSD plane can also work together with other graphics to improve the display quality. For example, the acceleration layer can respond to the touch event input by the user at the same time as the primary OSD layer and draw graphics respectively. However, since the acceleration layer is displayed above the primary OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer, thereby maintaining the real-time response speed. After the user inputs the touch interaction action, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawn graphics, thereby reducing the graphics mixing time without the need for the acceleration layer to refresh the drawn graphics to the OSD layer. This improves the real-time display effect of the graphics drawing process and also improves the final presentation quality of the graphics drawing.

[0082] In some embodiments, the whiteboard application can also dynamically configure different interface contents to adapt to different user needs. For example, the whiteboard application can include a "picture import" option. When the user clicks the picture import option, a picture storage path selection interface can be popped up, and the user can select a locally stored picture through the picture storage path selection interface to import the picture into the whiteboard interface for display.

[0083] In some embodiments, the whiteboard application can also store the drawing objects drawn by each touch action of the user. One touch action refers to the process from when the user's finger (or stylus) contacts the touch interaction module to when the user's finger (or stylus) leaves the touch interaction module. If the user inputs a click action, one touch action should include a down event and an up event. If the user inputs a swipe action, one touch action should include a down event, a move event, and an up event. Each touch action can generate a drawing object, and the whiteboard application can record the drawing object, thereby facilitating the user to perform actions such as selection, undo, redo, and the like.

[0084] In the process of storing the drawing objects, the whiteboard application can also set associated information for the drawing objects according to their characteristics and input states. For example, after the user draws a straight line, the whiteboard application can set a shape identifier for the straight line object, which can be stored in association with the straight line object and can be called by other modules of the whiteboard application or other applications to facilitate object selection, object retrieval, object deletion, and the like.

[0085] In some embodiments, the whiteboard application can also support multiple gesture interaction operations and have different control functions for different gesture interaction operations. For example, the whiteboard application can move the whiteboard drawing area along the track of a multi-finger swipe operation in response to a multi-finger swipe operation of the user. The whiteboard application can also perform an erasing function to erase the drawing objects covered by the track of a palm swipe operation of the user in response to the palm swipe operation.

[0086] For ease of description, the graphics displayed on the whiteboard interface (such as imported pictures, drawing objects formed by drawing, and teaching tool graphics, etc.) are uniformly denoted as element objects in some embodiments of the present disclosure.

[0087] In some embodiments, when the display device 200 displays the whiteboard interface, the user can implement touch interaction through touch actions such as clicking, long pressing, swiping, and the like, to make the display device 200 perform corresponding operations and update the whiteboard interface to display the added element objects in the whiteboard interface.

[0088] In some embodiments, a base stack is configured in the display device 200. The base stack is used to record the adding order of the element objects, and the base stack is a last in first out (LIFO) data structure. When a newly added element object is recorded, the newly added element object is recorded at the top of the stack. For example, as shown in FIG. 8, after the whiteboard application is run, the user first imports a picture to form an element object PC1 (at time T1), then draws a track line on the upper layer of the element object PC1 to form an element object Path1 (at time T2), and then imports another picture to form an element object PC2 (at time T3), and then draws two track lines in the whiteboard interface to form an element object Path2 (at time T4) and an element object Path3 (at time T5). The above element objects are opaque colors, and after being drawn, the whiteboard interface of the display device 200 presents the display effect shown in FIG. 9. Correspondingly, the element objects written from bottom to top in the base stack in the display device 200 are: the element object PC1, the element object Path1, the element object PC2, the element object Path2, and the element object Path3, thereby recording the adding order of the element objects.

[0089] In some embodiments, the whiteboard interface can also provide auxiliary tools and menus for adjusting element objects. For example, the whiteboard interface can include a “select” tool, and when the user clicks the “select” tool icon, the whiteboard application can enter an object selection mode. In the object selection mode, the display device 200 can call element objects that have an intersection with the touch position based on the touch position in the down event and / or the move event and / or the up event in the user touch interaction operation, to achieve the purpose of simultaneously selecting multiple element objects.

[0090] In some embodiments, as shown in FIG. 9, the whiteboard interface adds multiple element objects, and in order to perform editing operations such as moving, scaling, or rotating on one or more element objects in the whiteboard interface, one or more element objects can be selected to perform the corresponding editing operation, that is, the display device 200 can respond to the selection instruction input by the user, detect the target element object indicated in the selection instruction, and perform a selection operation on one or more target element objects. In the process of selecting one or more target element objects in the drawing interface, in order to facilitate the user to distinguish between selected objects and unselected objects, when a certain target element object is selected, a selection effect is displayed, for example, the outline of the target element object is highlighted.

[0091] In some embodiments, the implementation of the selection instruction includes multiple methods. For a single element object, the user can implement the selection operation by a point selection method, such as a single click, double click, long press, and the like. For multiple element objects, the user can implement the selection operation by a line selection method, a circle selection method, a rectangular diagonal selection method, and the like.

[0092] In some embodiments, the display device 200 can perform the selection operation on the selected target element object based on the outlining method, and perform outlining on the target element object to display the selection effect of the target element object. The display device 200 can obtain a preset color value, and perform thickening processing on the target element object based on the preset color value. The preset color value is a color value with a color difference from the line color value of the target element object greater than a color difference threshold value, and the display device 200 uses the preset color value as the fill color of the target element object. After the display device 200 performs thickening processing on the target element object based on the preset color value, an intermediate object is generated, and the intermediate object is redrawn and the target element object is drawn on the upper layer of the intermediate object. Since the intermediate object and the target element object have different line thicknesses, when the display device 200 displays the drawn objects, the edges of the intermediate object will exceed the display area of the target element object, forming an outlining effect.

[0093] That is, an intermediate object is first drawn using a brush that is thicker than the target element object and has a different color from the target element object, and then the target element object is drawn on the upper layer of the intermediate object. Since the intermediate object is a graph formed after the target element object is thickened and rendered, it is thicker than the target element object and has a different color, which covers the target element object. Then the target element object is redrawn on the upper layer of the intermediate object, so that the intermediate object only displays the edges of the line, thereby achieving the effect of highlighting the target element object.

[0094] For example, as shown in FIG. 10, when the selection operation is performed on the element object Path in FIG. 9, an element object Path2` can be drawn at the position of the element object Path2 using a brush that is thicker than the element object Path2 and has a different color from the element object Path2. The drawn Path2` is the intermediate object.

[0095] Further, since the element object Path2` and the element object Path2 are located at the same position, and the element object Path2` is thicker than the element object Path2, the element object Path2` can cover the element object Path2. Then, the element object Path2 is drawn at the position of the element object Path2` using a brush that has the same thickness and the same color as the element object Path2. Since the element object Path2` is thicker than the element object Path2, the element object Path2` is not completely covered, thereby achieving the outlining effect of the element object Path2.

[0096] For ease of description, the display device 200 renders the graphical representation of the intermediate object and the target element object together as an identification object, i.e., the identification object is a graphical object formed by superimposing the intermediate object and the re-rendered target element object.

[0097] The above-described stroke method can render all the target element objects indicated by the selection instruction, thereby presenting a highlight effect. In some embodiments, when the number of target element objects is greater than or equal to 2, the display device 200 can read the addition order stored in the base stack and render the identification objects of the target element objects in the order recorded in the base stack. For example, the display device 200 can create a copy of the base stack, and then read each target element object from the copy to obtain the addition order of the target element objects, so that the content of the original base stack remains unchanged.

[0098] To keep synchronization with user operations, in some embodiments, the display device 200 can render the identification objects of the target element objects in the order of their addition, i.e., first render the identification object of the target element object added earlier and then render the identification object of the target element object added later. In this way, after the identification objects are rendered, the display levels of the identification objects and the original target element objects can be kept synchronized.

[0099] It should be noted that the display levels of the element objects and the identification objects described in the embodiments of the present disclosure can be the hierarchical relationship between the element objects on the same layer, the hierarchical relationship between the element objects on different layers, or the hierarchical relationship between the element objects on the same layer and on different layers. The present disclosure does not limit this.

[0100] Regardless of the same layer or different layers, when the display level of the target element object or the identification object is higher than that of the underlying target element object or identification object, and the upper object is opaque, the upper object will block the underlying object, making it difficult for the user to intuitively perceive the existence of the underlying object, and prone to misoperation problems, such as selecting other non-desired element objects (excess selection) or not fully selecting all desired element objects (omission selection), which will all cause the above misoperation problems.

[0101] For example, the element object shown in FIG. 8, the whiteboard interface of the display device 200 presents the display effect shown in FIG. 9. Among them, the display areas of the element object PC1, the element object Path1 and the element object PC2 overlap, and the element object PC1, the element object Path1 and the element object PC2 are all opaque colors, and the display level of the element object PC2 is higher than that of the element object PC1 and the element object Path1. Therefore, the element object PC2 completely covers the element object Path1 and the element object PC1, and at this time the user cannot intuitively see the element object Path1 and the element object PC1 in the whiteboard interface (as shown in FIG. 9).

[0102] Based on this, in the whiteboard interface shown in FIG. 9, if the user wants to select a single element object PC2, the element object PC2 needs to be selected. Since part of the selection instruction needs to be drawn by a touch operation to a certain area, such as an area including the element object PC2 can be drawn by a circle selection method, a line selection method or a rectangular diagonal method. For example, the user can draw the area β shown in FIG. 12 by a touch operation to select the element object PC2. However, as shown in FIG. 8, the element object PC1 and the element object Path1 are also in the area β, and the above operation will additionally select the element object PC1 and the element object Path1. At this time, as shown in the hatching effect in FIG. 12, since the display level of the identification object PC2 (the identification object of the element object PC2) is also higher than that of the identification object Path1 (the identification object of the element object Path1) and the identification object PC1 (the identification object of the element object PC1), the user cannot see the selected state of the element object Path1 and the element object PC1 after the selection is completed, thereby causing the problem of redundant selection.

[0103] Alternatively, in the whiteboard interface shown in FIG. 9, if the user wants to select two element objects PC2 and PC1, the element object PC2 and the element object Path1 need to be selected. However, since the element object PC1 is completely covered by the element object PC2, the user cannot accurately perceive the position of the element object PC1, and the problem of inaccurate area drawn by the selection instruction is likely to occur. For example, as shown in FIG. 13, since the position of the lower element object PC1 cannot be intuitively observed, the user can draw the area α based on the memory of the position of the lower element object. However, the position of the element object PC1 is not in the area α, and the element object PC1 and the area α only have a small distance. Similarly, since the display level of the identification object PC2 (the identification object of the element object PC2) is higher than that of the identification object PC1 (the identification object of the element object PC1), the user cannot see whether the element object PC1 is selected after the selection is completed, thereby causing the problem of missed selection.

[0104] It should be noted that, in order to facilitate the description, the lower element object is shown by a dashed line in FIG. 13, and the display device 200 actually displays the whiteboard interface, and the element object PC1 and the element object Path1 are in an invisible state due to the shielding of the element object PC2.

[0105] That is, if there is a covered target element object in the target element object indicated by the selection instruction, the user cannot intuitively perceive the position of the target element object in the lower layer, nor can the user intuitively perceive whether the target element object in the lower layer is selected, whether in the selection process or at the end of the selection, thereby affecting the accuracy of touch selection in the display device 200.

[0106] Based on the above application scenarios, in some embodiments, the display device 200 can include a display, a touch interaction module, a memory, and at least one processor. The display is configured to display a user interface including at least two element objects added in response to a touch instruction. The touch interaction module is connected to the display and is configured to receive a touch instruction input by a user. The memory is configured to store computer instructions and data associated with the display device. The at least one processor is connected to the display and the memory and is configured to execute the computer instructions to cause the display device to perform the steps shown in FIG. 14:

[0107] S1401: In response to a target selection instruction input by a user, detecting an object type of a target element object.

[0108] After the display device 200 receives a selection instruction input by a user based on a line selection method, a circle selection method, a rectangular diagonal selection method, etc., the display device 200 detects an object type of a target element object indicated by the target selection instruction. The target element instruction indicates at least two element objects.

[0109] In some embodiments, since some element objects do not have shielding properties, such as a track line (Path), text, etc. The display device 200 divides the element objects into a covering class object and a non-covering class object. For example, the non-covering class object can be a track line, text, and a hollow graphic object, etc. which do not have complete shielding properties, and the above non-covering class element object will not completely cover the element object in the lower layer. The covering class object fills a track line, a picture (PIC), etc. which have complete shielding properties, and the above covering class element object can completely cover the element object in the lower layer.

[0110] In some embodiments, the element object further comprises a graphic class, such as a Path class for representing a track line, a TextBox class for representing a text box, a picture class for representing a picture, and a Shape class for representing a filled track line and a filled graphic, etc. The display device 200 can read the graphic class of the element object, mark the element object of a target graphic class as an overlay class object, and mark a non-target graphic class as a non-overlay class object. For example, the target graphic class can comprise the picture class and the Shape class, and the non-target graphic class can comprise the TextBox class and the Path class.

[0111] It should be noted that the naming of the above graphic classes can also be other ways, such as the graphic class of the track line can also be a Line class, and the graphic class of the picture can also be an Image class. The present disclosure does not limit this.

[0112] In some embodiments, the element object can further comprise a preset identifier, which is used to represent whether the element object is an overlay class object or a non-overlay class object. For example, the preset identifier can comprise a first identifier and a second identifier, the first identifier is used to represent the overlay class object, and the second identifier is used to represent the non-overlay class object, and the display device 200 can determine the object type of the element object by reading the first identifier or the second identifier.

[0113] It should be noted that the preset identifier can be a combination of one or more of English characters, Chinese characters, or numbers, and the present disclosure does not limit this. In some embodiments, the preset identifier can only comprise one identifier, and the element object with a flag bit in the preset identifier is one of the overlay class object and the non-overlay class object, and the element object with an empty flag bit (default character) is the element object of the other object type.

[0114] In some embodiments, since the element object with transparency can penetrate the underlying element object, the display device 200 can also mark the element object with transparency as a non-overlay class object. For example, the display device 200 can read the Alpha (transparency) value of the element object, the value range of Alpha is 0-1, the larger the value of Alpha, the less transparent the element object; the smaller the value of Alpha, the more transparent the element object. The display device 200 can mark the element object with an Alpha value less than 1 as a non-overlay class object.

[0115] In some embodiments, since part of the selection instruction can be determined only when the termination point is monitored, i.e., the up event is detected, the complete selection instruction is formed. For example, the selection instruction input by the touch interaction action such as line selection, circle selection, rectangular diagonal selection, etc. To this end, the display device 200 can create a state change list, and then record the element objects that change from unselected state to selected state and from selected state to unselected state in the state change list during the process of receiving the target selection instruction, and draw the corresponding identification object based on the element objects in the state change list, so that the highlight effect of the selected element object can be presented in real time during the selection process.

[0116] That is, the display device 200 can perform the stroking operation on the object changing from unselected state to selected state and the unstroking operation on the object changing from selected state to unselected state in real time during the touch selection process of the user, so as to prompt the user about the selection state of each element object, and the user can intuitively perceive whether the element object is selected during the selection process, and the accuracy of touch selection can be improved.

[0117] In some embodiments, the display device 200 can distinguish the element objects that are stroked and unstroked as shown in FIG. 15, and specifically as shown in FIG. 15, the method can include the following steps:

[0118] S1501: receiving a touch point for forming a target selection instruction;

[0119] S1502: setting the selection state of the element object according to the element object indicated by the touch point;

[0120] For example, as shown in FIGS. 16-19, the process of inputting the target selection instruction by the user based on the rectangular diagonal selection method, as shown in FIG. 16, when the sliding event from the touch point A to the touch point B is detected, the element object PC2 is selected, and the display device 200 sets the element object PC2 to the selected state. Then continue to detect the touch point, when the sliding event from the touch point A to the touch point C is detected, as shown in FIG. 17, the element object PC1 is selected again, and the display device 200 sets the element object PC1 to the selected state. When the sliding event from the touch point C to the touch point D is detected, as shown in FIG. 18, the element object Path1 is selected again, and the display device 200 sets the element object Path1 to the selected state. At this time, the sliding event of the touch point D sliding back to the touch point C is detected again, as shown in FIG. 19, the selection of the element object Path1 is cancelled, and the display device 200 sets the element object Path1 to the unselected state.

[0121] S1503: creating a first state change list and a second state change list;

[0122] The display device 200 can create a first state change list and a second state change list after setting the selection state of the element object according to the touch point. The first state change list is used to record the selected element object, and the second state change list is used to record the element object whose selection state is changed.

[0123] S1504: record the first object in the first state change list and record the second object in the second state change list;

[0124] In the embodiments of the present disclosure, the newly selected element object is represented as the first object, and the element object whose selection state is changed is represented as the second object. The first object recorded in the first state change list and the second object recorded in the second state change list are updated in real time following the touch point of the newly added target selection instruction. That is, after the display device 200 creates the first state change list and the second state change list, the first object whose state is changed from unselected to selected is recorded in the first state change list, and the second object whose state is changed from selected to unselected is recorded in the second state change list.

[0125] S1505: draw a third identification object based on the first state change list, and draw a fourth identification object based on the second state change list.

[0126] The third identification object is generated by superimposing the element object formed by the first object after thickening and the first object, and the fourth identification object is generated by the element object formed by the second object after thickening. Then, the display 260 is controlled to display the third identification object and the fourth identification object, so as to realize the real-time outlining effect of the selected element object and the canceling outlining effect of the element object whose selection state is changed during the selection process.

[0127] For the outlining operation during the selection process, in some embodiments, the display device 200 obtains a preset color value. The preset color value is different from the line color value of the first object. Then, the line of the first object is thickened according to the preset color value to generate a first intermediate object. Then, the first intermediate object is drawn, and the first object is drawn on the upper layer of the first intermediate object to generate the third identification object corresponding to the first object.

[0128] That is, during the selection process, if one element object is selected, the display device 200 draws the intermediate object corresponding to the element object by using a color different from the original line color of the element object, and the thickness of the line of the intermediate object is thicker than that of the original element object. Then, the element object is redrawn on the upper layer of the intermediate object, so as to present the outlining effect by the line of the intermediate object which is not covered due to thickening and has a different color.

[0129] In some embodiments, the preset color value can be dynamically set based on the line color value of the element object, and the color difference between the preset color value and the line color value of the element object is greater than a default color difference threshold. For example, when the line color value is white, the preset color value used for the stroke can be yellow.

[0130] For the canceling stroke operation in the selection process, in some embodiments, the display device 200 acquires the line color value of the second object. Then, the thickening processing is performed on the line of the second object according to the line color value of the second object, to obtain the fourth identification object.

[0131] That is, when one of the element objects is unselected in the selection process, the display device 200 draws the identification object corresponding to the element object by using the same color as the line color of the element object, and the thickness of the line of the identification object is thicker than the thickness of the line of the original element object, so that the stroke effect (i.e., the third identification object) displayed when the element object is selected is covered by the line of the same color in the identification object that is not covered due to thickening, thereby achieving the canceling stroke effect. For example, when the line color value is white, the line color value of the fourth identification object drawn when the stroke is canceled is still white.

[0132] In some embodiments, in the selection process, when the display device 200 draws the first intermediate object of the element object, the thickness of the line of the first intermediate object is recorded as a reference thickness. When the fourth identification object of the element object is drawn, the recorded reference thickness is read, and the thickening processing is performed on the line of the element object according to the reference thickness, so as to ensure that the identification object drawn when the stroke is canceled can cover the identification object drawn when the stroke is performed.

[0133] Alternatively, in some embodiments, the display device 200 can also read a preset thickening threshold, and perform the thickening processing on the line of the element object according to the preset thickening threshold. In this way, whether it is the canceling stroke operation or the stroke operation, the thickness of the thickened line of the same element object is the same, and it can also be ensured that the identification object drawn when the stroke is canceled can cover the identification object drawn when the stroke is performed. For example, the thickening threshold can be 2, the thickness of the line is x, and the thickness of the thickened line is x+2.

[0134] Based on the above embodiments, the display device 200 displays the stroke and canceling stroke effects of the user in the process of inputting the target selection instruction as shown in FIGS. 16-19, which will not be described herein. It can be understood that, in order to highlight the display effect of the stroke, in the drawings provided by the embodiments of the present disclosure, the thickness of the line of the intermediate object is set to be relatively thick, and in actual applications, since the color difference between the preset color value and the line color value is large, the display device 200 can set a thinner thickness for the intermediate object. In this regard, the present disclosure is not limited.

[0135] After the user completes the input of the target selection instruction, the display device 200 also needs to determine the final selection result (i.e., the target element object) according to the target selection instruction, and draw the identification object corresponding to the target element object to realize the stroke effect. In order to improve the situation that the identification effect cannot be intuitively perceived when the target element object is covered, in some embodiments of the present disclosure, the display device 200 also detects the object type corresponding to the target element object. The object type of the target element object can be a covering class object, or a non-covering class object, or a combination of a covering class object and a non-covering class object.

[0136] S1402: If the target element object is a non-covering class object, draw a first identification object according to a first order, and control the display to display the first identification object.

[0137] For a non-covering class object, the display device 200 can draw a first identification object corresponding thereto according to a first order, and display the first identification object through the display 260. The first order is the adding order of the non-covering class object, and the first identification object is generated by superimposing the element object formed by thickening the non-covering class object and the non-covering class object. That is, since the non-covering class object does not completely cover the underlying element object, for example, the upper track line can only cover part of the lower track line, and cannot completely cover the lower track line and the stroke effect of the lower track line. Therefore, the display device 200 can draw the identification object corresponding to the non-covering class object according to the original adding order of the non-covering class object.

[0138] In some embodiments, when the display device 200 draws the first identification object, the non-covering class object is recorded to a first list (List), and then the adding order of the non-covering class element object is read in the base stack based on the first list. Then, the non-covering class element object is arranged according to the adding order to generate a first order. The first order is used to draw the first identification object according to the order from early to late according to the adding time, so that the first identification object corresponding to the non-covering class object with the latest adding time has the highest display level. After the display device 200 generates the first order, the first identification object is drawn according to the first order.

[0139] That is, for non-overlay type objects, the display device 200 can draw the identification objects corresponding to the non-overlay type objects in the order of addition of the non-overlay type objects. For example, as shown in FIG. 8, the element objects, the display device 200 displays the display effect shown in FIG. 9 in the whiteboard interface. The user selects the non-overlay type objects: the element object Path2 and the element object Path3, and the display device 200 draws the identification objects in the order of addition of the two. Since the element object Path2 is added earlier than the element object Path3, the display device 200 draws the identification object Path2` of the element object Path2 first, and then draws the identification object Path3` of the element object Path3, and displays the stroke effect shown in FIG. 11.

[0140] S1403: If the target element object is an overlay type object, draw a second identification object according to a second order, and control the display to display the second identification object.

[0141] For non-overlay type objects, the display device 200 can draw the second identification objects corresponding thereto according to the second order, and display the second identification objects through the display 260. The second order is generated based on the overlapping areas between the overlay type objects, and the second identification objects are generated by the element objects formed by the overlay type objects being thickened and the overlay type objects being superimposed. That is, since the overlay type objects can cover a larger area of the underlying element objects than the non-overlay type objects, the display device 200 can determine the drawing order (i.e., the second order) of the non-overlay type objects through the overlapping areas of the overlay type objects.

[0142] In some embodiments, for a target element object of a non-overlay type, the display device 200 can determine the drawing order of the identification objects according to one or more of the following principles: 1. Arranging the overlay type objects that completely cover other element objects in front, and arranging the overlay type objects that do not completely cover other element objects in back. 2. When the element objects have overlapping areas, arranging the element objects with larger areas in front and arranging the element objects with smaller areas in back. 3. Based on the target selection instruction input by the point selection method, arranging the uppermost element object indicated in the target selection instruction as the target element object. 4. When there are multiple target element objects, displaying a selection object list control, and when the user clicks the selection object list control, displaying a selection object list including all the target element objects, wherein the arrangement order of the selection object list is consistent with the generation order. 5. When the target element objects include overlay type objects and non-overlay type objects, arranging the overlay type objects and the non-overlay type objects according to at least one of the principles in 4 above, and then arranging the non-overlay type objects in front of the overlay type objects.

[0143] It should be noted that the first order and the second order are the order of drawing the identification objects, and the element object arranged in the front has a higher drawing order of the corresponding identification object. However, it is not limited thereto, and in some embodiments, the first order and the second order can be an order opposite to the above principle, and the display device 200 can first read the element object arranged in the rear and draw the identification object of the element object arranged in the rear.

[0144] In some embodiments, when the display device 200 draws the second identification object, the display device 200 can record the cover class object to a second list, and then parse the display area of the cover class object based on the second list. Then, the cover attribute of the cover class object is detected according to the overlapping area. The cover attribute includes a first attribute and a second attribute. For a cover class object, the first attribute of the cover class object represents the display area of other cover class objects (for example, the display area of another cover class object) contained in the display area of the cover class object; and the second attribute represents the remaining cover attribute except the first attribute.

[0145] In some embodiments, the first attribute can be represented as a complete cover attribute, and the second attribute can be represented as a non-complete cover attribute.

[0146] After the display device 200 detects the cover attribute of the cover class object, the display device 200 sets the arrangement order of the cover class object of the first attribute before the arrangement order of the cover class object of the second attribute to generate a second order. The second order is used to draw the second identification object according to the cover attribute, so that the display level of the second identification object corresponding to the cover class object of the first attribute is lower than the display level of the second identification object corresponding to the cover class object of the second attribute.

[0147] That is, for the cover class object, the upper cover class object in the selected state completely blocks the lower cover class object, the display device 200 first draws the identification object of the upper cover class object, and then draws the identification object of the lower cover class object, so that the identification object of the upper cover class object is displayed below the identification object of the lower cover class object.

[0148] In some embodiments, when there are multiple cover class objects of the first attribute in the target element object, the display device 200 can read the adding order of the cover class object, arrange the cover class object with a later adding order in front, and arrange the cover class object with an earlier adding order in rear, so that the identification object corresponding to the cover class object with a lower display level is displayed at a higher display level.

[0149] In some embodiments, when the display device 200 draws the second mark, the display device 200 can also record the overlay objects into a second list, resolve the display area of the overlay objects based on the second list, and detect the overlapping area of the overlay objects according to the display area. When the overlay objects have overlapping areas, the display device 200 detects the element object area of the overlay objects having the overlapping areas. Then, the display device 200 sets the arrangement order of the overlay objects according to the element object area from large to small to generate a second order. The second order is used to draw the second mark objects according to the element object area, so that the second mark object corresponding to the overlay object with a large element object area is displayed at a lower level than the second mark object corresponding to the overlay object with a small element object area. The display device 200 draws the second mark objects according to the second order.

[0150] That is, for the overlay objects, when the overlay object at a higher level blocks the overlay object at a lower level, the display device 200 draws the mark object with a large element object area first, and then draws the mark object with a small element object area, so that the mark object with a large element object area is displayed at a lower level than the mark object with a small element object area.

[0151] For the case where the target element object includes both the non-overlay object and the overlay object, in some embodiments, as shown in FIG. 20, the display device 200 superimposes the second order on the first order to generate a third order. The third order includes the first order and the second order, and the highest arrangement order in the first order is lower than the lowest arrangement order in the second order. The third order can be used to make the display level of the first mark object higher than the second mark object. Then, the display device 200 draws the mark objects of the target element object according to the third order, and the mark objects include the first mark object and the second mark object. The display device 200 controls the display 260 to display the mark objects, so that the mark object corresponding to the non-overlay object is displayed at a higher level than the mark object corresponding to the overlay object.

[0152] That is, when the target selection object indicated by the target selection instruction input by the user includes both the non-overlay object and the overlay object, the display device 200 can arrange the drawing order of the non-overlay object behind the overlay object, draw the mark objects of the overlay object according to the second order first, and then draw the mark objects of the non-overlay object according to the third order, so that all the first mark objects are displayed at a higher level than the second mark objects.

[0153] For example, as shown in FIG. 8, the whiteboard interface of the display device 200 presents the display effect shown in FIG. 9. The target selection instruction input by the user indicates the overlay class objects: the element object PC1 and the element object PC2, and the non-overlay class object: the element object Path1. The display device 200 determines the drawing order according to the order principle of the non-overlay class object and the overlay class object. Since the element object PC2 completely covers the element object PC1, the arrangement order of the element object PC2 is after the element object PC1. The element object Path1 is a non-overlay class object, and the element object Path1 is after the element object PC2. The determined drawing order is element object PC1-> element object PC2-> element object Path1. The display device 200 draws the identification object PC1` of the element object PC1 first, then draws the identification object PC2` of the element object PC2, and finally draws the identification object Path1` of the element object Path1, and displays the stroke effect as shown in FIG. 21. In this way, when there is partial overlap or complete overlap between the target element objects, the display device 200 can display the identification effect of all the target element objects, which can improve the user experience.

[0154] For the target selection instruction input by the user based on the point selection method, in some embodiments, the display device 200 can also analyze the input type of the target selection instruction. When the input type is the point selection type, that is, the target selection instruction is a touch instruction input based on the point selection method, the third object with the highest display level in the target element object is detected. Then the fifth identification object of the third object is drawn. The fifth identification object is generated by the element object formed by the third object being thickened and the third object being superimposed. Then the display 260 is controlled to display the fifth identification object.

[0155] That is, since the point selection method is commonly used when selecting a single element object, in order to be close to the touch intention of the user, when the user inputs the target selection instruction based on the point selection method, the display device 200 can only set the topmost element object indicated by the selection instruction to the selected state, and only draw the identification object of the topmost element object indicated by the target selection instruction.

[0156] Based on the display device 200 described above, some embodiments of the present disclosure also provide a touch selection method, which can be applied to the display device 200 provided in the above embodiments. The touch selection method includes the following program steps:

[0157] In response to a target selection instruction input by a user, an object type of a target element object is detected; the target element object is an element object indicated in the target selection instruction, and the target selection instruction indicates at least two element objects; if the target element object is a non-overlapping class object, a first identification object is drawn according to a first order and displayed; wherein the first order is determined based on an adding order of the non-overlapping class object; the first identification object is generated by superimposing an element object obtained by using the non-overlapping class object and performing bold processing on the non-overlapping class object; if the target element object is an overlapping class object, a second identification object is drawn according to a second order and displayed; wherein the second order is generated based on overlapping areas between the overlapping class objects, and the second identification object is generated by superimposing an element object obtained by using the overlapping class object and performing bold processing on the overlapping class object.

[0158] It can be known from the above process that the display device and the touch selection method provided by the embodiments of the present disclosure can detect an object type of a target element object indicated by a target selection instruction input by a user in response to the target selection instruction. If the target element object is a non-overlapping class object, an identification object is drawn according to an adding order of the target element object and displayed; if the target element object is an overlapping class object, the identification object is drawn according to a drawing order determined based on overlapping areas between the target element objects and displayed. The identification object is generated by superimposing an element object obtained by using the target element object and the target element object. The method classifies element objects, performs outline identification on the non-overlapping class object by determining the drawing order based on the adding order of the target element object, and performs outline identification on the overlapping class object by determining the drawing order based on the overlapping condition of the overlapping class object, so as to highlight a selection effect of each element object and improve the accuracy of touch selection.

[0159] Those skilled in the art can clearly understand that the technology in the embodiments of the present disclosure can be realized by means of software and necessary general hardware platforms. Based on such understanding, the implementation in the embodiments of the present disclosure or the part that contributes to the prior art in essence can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method of various embodiments or some parts of the embodiments of the present disclosure.

Claims

1. A display device, comprising: a display configured to display a user interface; at least two element objects added in the user interface in response to touch instructions; a touch interaction module connected with the display, configured to receive touch instructions input by a user; a memory configured to store computer instructions and data associated with the display device; at least one processor connected with the display and the memory, configured to execute the computer instructions to cause the display device to perform: detecting an object type of a target element object in response to target selection instructions input by the user; the target element object is an element object indicated in the target selection instructions, and the target selection instructions indicate at least two element objects; if the target element object is a non-overlapping class object, drawing a first identification object according to a first order and controlling the display to display the first identification object; wherein the first order is determined based on an adding order of the non-overlapping class object; and the first identification object is generated by superimposing the non-overlapping class object and an element object obtained by performing bold processing on the non-overlapping class object; if the target element object is an overlapping class object, drawing a second identification object according to a second order and controlling the display to display the second identification object; wherein the second order is generated based on overlapping areas between the overlapping class objects; and the second identification object is generated by superimposing the overlapping class object and an element object obtained by performing bold processing on the overlapping class object. 2.The display device of claim 1, wherein the at least one processor is specifically configured to execute the computer instructions to cause the display device to perform the drawing of the first identification object according to the first order by: recording the non-overlapping class object to a first list; reading the adding order of the non-overlapping class element object in a base stack based on the first list; sorting the non-overlapping class element object based on the adding order to generate the first order; and drawing the first identification object according to the first order. 3.The display device of claim 2, wherein the at least one processor is specifically configured to execute the computer instructions to cause the display device to perform the drawing of the first identification object according to the first order by: drawing the first identification object based on an order from early to late of adding time of the non-overlapping class element object, so that a display level of the first identification object corresponding to a non-overlapping class object with the latest adding time is the highest. 4.The display device of claim 1, wherein the at least one processor is specifically configured to execute the computer instructions to cause the display device to perform the drawing of the second identification object according to the second order by: recording the overlapping class object to a second list; analyzing display areas of the overlapping class object based on the second list. detecting a coverage property of the coverage class object according to the overlapping region; wherein The cover attributes include a first attribute and a second attribute; for one cover class object, the first attribute represents a display area of the one cover class object, and the second attribute represents a display area of another cover class object contained in the display area of the one cover class object; arranging the cover class objects in the first attribute in a first order, and arranging the cover class objects in the second attribute in a second order, to generate the second order; and rendering the second identification objects according to the second order.

5. The display device of claim 4, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the rendering the second identification objects according to the second order: rendering the second identification objects based on the cover attributes of the cover class objects in the second order, so that a display level of the second identification object corresponding to the cover class object in the first attribute is lower than a display level of the second identification object corresponding to the cover class object in the second attribute.

6. The display device of claim 1, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the rendering the second identification objects according to the second order: recording the cover class objects to a second list; analyzing display areas of the cover class objects based on the second list, and determining whether there is an overlapping area according to the display areas; if yes, arranging the cover class objects in an order from large to small according to the element object areas of the cover class objects, to generate the second order; and rendering the second identification objects according to the second order.

7. The display device of claim 6, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the rendering the second identification objects according to the second order: rendering the second identification objects based on the element object areas of the cover class objects in the second order, so that a display level of the second identification object corresponding to the cover class object with a large element object area is lower than a display level of the second identification object corresponding to the cover class object with a small element object area.

8. The display device of claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: when the target element object includes a non-overriding class object and an overriding class object, superimposing the second order onto the first order to generate a third order; wherein, the third order includes the first order and the second order, and a highest arrangement order in the first order is lower than a lowest arrangement order in the second order; the third order is used to make a display level of the first identification objects higher than a display level of the second identification objects; rendering identification objects of the target element object according to the third order, and controlling the display to display the identification objects; wherein the identification objects include the first identification objects and the second identification objects.

9. The display device of claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: receiving a touch point for forming the target selection instruction, and setting a selection state of an element object indicated by the touch point according to the element object, the selection state being a selected state or an unselected state; creating a first state change list and a second state change list; recording a first object changed from the unselected state to the selected state to the first state change list, and recording a second object changed from the selected state to the unselected state to the second state change list; drawing a third identification object based on the first state change list, and drawing a fourth identification object based on the second state change list; the third identification object is generated by the element object formed by the first object being bolded and the first object being superimposed, and the fourth identification object is generated by the element object formed by the second object being bolded; controlling the display to display the third identification object and the fourth identification object.

10. The display device of claim 9, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the drawing of the third identification object based on the first state change list by: obtaining a preset color value; the preset color value being different from a line color value of the first object; performing a bolding process on the line of the first object according to the preset color value to generate a first intermediate object; drawing the first object on an upper layer of the first intermediate object to generate the third identification object.

11. The display device of claim 9, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the drawing of the fourth identification object based on the second state change list by: obtaining a line color value of the second object; performing a bolding process on the line of the second object according to the line color value of the second object to generate the fourth identification object.

12. The display device of claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: parsing an input type of the target selection instruction; when the input type is a point selection type, detecting a third object with a highest display level in the target element object; drawing a fifth identification object of the third object, the fifth identification object being generated by the element object formed by the third object being bolded and the third object being superimposed; controlling the display to display the fifth identification object.

13. A touch selection method, comprising: in response to a target selection instruction input by a user, detecting an object type of a target element object; the target element object being an element object indicated in the target selection instruction, the target selection instruction indicating at least two element objects; if the target element object is a non-overlapping class object, drawing and displaying a first identification object according to a first order; wherein the first order is determined based on an adding order of the non-overlapping class object; and the first identification object is generated by element objects obtained by using the non-overlapping class object and performing a bolding process on the non-overlapping class object. If the target element object is an overlay class object, a second identification object is drawn and displayed according to a second order; wherein the second order is generated based on overlapping areas between the overlay class objects, and the second identification object is an element object superimposed by using the overlay class objects and performing bold processing on the overlay class objects.

Images