Display device, and trajectory line drawing method for display device

Abstract

Disclosed in the present application are a display device, and a trajectory line drawing method for a display device. The method may comprise: in response to a touch-control interaction instruction, which is inputted by a user, acquiring, in real time, a touch-control region generated by the touch-control interaction instruction; and generating a stroke model on the basis of the touch-control region, and then drawing a stroke trajectory line at the tail end of a touch-control trajectory on the basis of the stroke model, wherein the stroke model may be a dot matrix model generated on the basis of a touch-control point, a contour point and spare points in the touch-control region, the touch-control point may be at least one contact point in the touch-control region, the contour point may be at least one contact point that is located on an edge position of a stroke in the touch-control region, and the spare points may divide the distance between the touch-control point and the contour point into a preset number of equal parts. By means of the stroke model, a stroke effect can be maintained during the drawing of a stroke, and the stroke effect is uniform and smooth, thereby improving the user experience.

Description

Display device and trajectory drawing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on September 28, 2023, with application number 202311283411.1; filed on October 12, 2023, with application number 202311322911.1; and filed on May 31, 2024, with application number 202410698995.7, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a trajectory line drawing method for the display device. Background Art

[0004] Display devices are terminal devices capable of presenting user interfaces and supporting user interaction. These devices can include smart TVs, communication terminals, smart advertising screens, projectors, and other terminal devices. Taking smart TVs as an example, they are based on Internet application technologies, feature open operating systems and chips, and possess an open application platform. They enable two-way human-computer interaction and integrate multiple functions, including audio, video, entertainment, and data, to meet the diverse and personalized needs of users.

[0005] For display devices that support touch interaction, you can run a whiteboard application and implement writing or drawing based on user touch gestures. To enhance the writing or drawing effect, the display device can also add a pen stroke effect to the end of the trace when running the whiteboard application. The pen stroke effect creates a smooth, varying thickness effect on the writing or drawing, which can be used to decorate the writing or drawing.

[0006] In related technologies, when achieving pen stroke effects, display devices draw based on writing speed or contact area, or connect the latest path with the previous path based on the touch point. The latest path is usually drawn as a thin line, and the previous path is drawn as a thick line. However, drawing pen strokes based on writing speed or contact area can result in uneven and choppy pen strokes due to differences in writing speed or contact area. Connecting the latest path can also result in uneven and choppy pen strokes due to differences in line width between the latest and previous paths.

[0007] Summary of the Invention

[0008] According to some embodiments of the present application, a display device may include: a display, a memory and at least one processor, wherein the display may be configured to display a user interface; the display may be connected to a touch interaction module, and the touch interaction module may be used to obtain touch interaction instructions input by a user; the memory may be configured to store computer instructions and / or data associated with the display device; the at least one processor, connected to the display and the memory, may be configured to execute the computer instructions to enable the display device to execute: in response to the touch interaction instructions input by the user, a touch area and a touch trajectory may be obtained; a pen stroke model may be generated based on the touch area, and the pen stroke model may be a dot matrix model generated based on the touch points, contour points and spare points in the touch area; the touch point may be at least one contact point in the touch area, and the contour point may be at least one contact point located at the edge of the pen stroke in the touch area; the spare point may divide the distance between the touch point and the contour point into preset equal parts; according to the pen stroke model, a pen stroke trajectory line may be drawn at the end of the touch trajectory.

[0009] According to some embodiments of the present application, a trajectory line drawing method for a display device may include: in response to a touch interaction instruction input by a user, a touch area and a touch trajectory may be obtained; a pen tip model may be generated based on the touch area, and the pen tip model may be a dot matrix model generated based on the touch point, contour point and spare point in the touch area; the touch point may be at least one contact point in the touch area, and the contour point may be at least one contact point located at the edge of the pen tip in the touch area; the spare point may divide the distance between the touch point and the contour point into preset equal parts; according to the pen tip model, a pen tip trajectory line may be drawn at the end of the touch trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application;

[0011] FIG2 is a block diagram of a hardware configuration of a display device according to some embodiments of the present application;

[0012] FIG3 is a block diagram of a hardware configuration of a control device according to some embodiments of the present application;

[0013] FIG4A is a schematic diagram of a software configuration in a display device according to some embodiments of the present application;

[0014] FIG4B is a schematic diagram of another software configuration in a display device according to some embodiments of the present application;

[0015] FIG5 is a schematic diagram of a touch track according to some embodiments of the present application;

[0016] FIG6 is a schematic diagram of a whiteboard application interface according to some embodiments of the present application;

[0017] FIG7 is a schematic diagram of a line cap of a trajectory line according to some embodiments of the present application;

[0018] FIG8 is a schematic diagram of a whiteboard application display system according to some embodiments of the present application;

[0019] FIG9 is a schematic diagram of an acceleration layer according to some embodiments of the present application;

[0020] FIG10 is a schematic diagram of a teaching tool according to some embodiments of the present application;

[0021] FIG11 is a schematic diagram of a pen stroke effect according to some embodiments of the present application;

[0022] FIG12 is a schematic diagram of a line drawing with a decorative head according to some embodiments of the present application;

[0023] FIG13 is a schematic diagram of a dotted line according to some embodiments of the present application;

[0024] FIG14 is a schematic flow chart of a method for drawing a trajectory line for a display device according to some embodiments of the present application;

[0025] FIG15 is a schematic diagram of a touch area and touch points based on a touch track according to some embodiments of the present application;

[0026] FIG16 is a schematic diagram of a touch area and a touch point based on a contact area according to some embodiments of the present application;

[0027] FIG17 is a schematic diagram of a dot matrix structure of a pen stroke model according to some embodiments of the present application;

[0028] FIG18 is a schematic diagram of quasi-contour points of a pen stroke model according to some embodiments of the present application;

[0029] FIG19 is a schematic diagram of a pen tip shape according to some embodiments of the present application;

[0030] FIG20 is a schematic diagram of updating a stroke model according to some embodiments of the present application;

[0031] FIG21 is a schematic diagram of a line drawing with an arrowhead added to the end according to some embodiments of the present application;

[0032] FIG22 is a schematic flow chart of another method for drawing a trajectory line for a display device according to some embodiments of the present application;

[0033] FIG23 is a structural diagram of a line drawing with an arrowhead added to the end according to some embodiments of the present application;

[0034] FIG24 is a schematic diagram of drawing arrow endpoint graphics according to some embodiments of the present application;

[0035] FIG25 is a schematic diagram of drawing a T-shaped endpoint graph according to some embodiments of the present application;

[0036] FIG26 is a schematic diagram of drawing an X-shaped endpoint graph according to some embodiments of the present application;

[0037] FIG27 is a schematic diagram of a refresh area according to some embodiments of the present application;

[0038] FIG28 is a schematic diagram of another refresh area according to some embodiments of the present application;

[0039] FIG29 is a schematic diagram of a straight line according to some embodiments of the present application;

[0040] FIG30 is a schematic diagram of dotted lines drawn when the input speed of a touch action is different according to some embodiments of the present application;

[0041] FIG31 is a flowchart of drawing a dotted line according to some embodiments of the present application;

[0042] FIG32 is a schematic diagram of a dotted line according to some embodiments of the present application;

[0043] FIG33 is a dotted line diagram of adding an arrow decorative head to the end according to some embodiments of the present application;

[0044] FIG34 is a schematic flow chart of yet another method for drawing a trajectory line for a display device according to some embodiments of the present application;

[0045] FIG35 is a schematic diagram showing the effect of a round-head wire cap path pattern according to some embodiments of the present application;

[0046] FIG36 is a schematic diagram of an unsegmented pen stroke effect according to some embodiments of the present application;

[0047] FIG37 is a schematic diagram showing the principle of calculating and drawing points according to some embodiments of the present application;

[0048] FIG38 is a schematic diagram showing the effect of a pen stroke path graphic according to some embodiments of the present application;

[0049] FIG39 is a schematic diagram of a first pen tip filling area according to some embodiments of the present application;

[0050] FIG40 is an example diagram showing an effect in which a slow movement speed causes a less obvious pen stroke effect according to some embodiments of the present application;

[0051] FIG41 is a rendering of a drawing process of a pen stroke effect according to some embodiments of the present application;

[0052] FIG42 is a rendering of another stroke effect drawing process according to some embodiments of the present application;

[0053] FIG43 is a schematic diagram showing the principles of area rendering according to some embodiments of the present application. DETAILED DESCRIPTION

[0054] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all embodiments consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0055] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0056] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way are interchangeable under appropriate circumstances. The terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components is not necessarily limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices. The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that can perform the functions associated with the element.

[0057] In the embodiments of this application, a display device generally refers to a device capable of displaying images and processing data. For example, a display device may include, but is not limited to, a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, and the like. The display device shown in the subsequent figures is only one specific embodiment. It should be understood that this application does not limit the specific form of the display device.

[0058] Figure 1 is a schematic diagram illustrating an operational scenario between a display device and a control device according to some embodiments of the present application. As shown in Figure 1 , a user can operate a display device 200 through touch control, a mobile terminal 300, and a control device 100. The control device 100 can be configured to convert user actions into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a controller, or the like.

[0059] The mobile terminal 300 can function as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, enabling connection and communication via a network communication protocol, enabling one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronized display.

[0060] 1 , the display device 200 also communicates data with the server 400 via various communication methods. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks.

[0061] In addition to providing broadcast reception television functions, the display device 200 may also provide intelligent network television functions with computer support functions, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0062] FIG2 is a hardware configuration block diagram of the display device 200 in FIG1 according to some embodiments of the present application.

[0063] In some embodiments, the display device 200 may include at least one of a tuner 210 , a communication device 220 , a detector 230 , a device interface 240 , at least one processor 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface 280 .

[0064] In some embodiments, detector 230 can be used to collect signals from the external environment or external interactions. For example, detector 230 can include a light receiver, a sensor that can be used to collect information about ambient light intensity; or detector 230 can include an image collector, such as a camera, that can be used to collect information about the external environment, user attributes, or user interaction gestures; or detector 230 can include a sound collector, such as a microphone, that can be used to receive external sounds.

[0065] In some embodiments, the device interface 240 may include, but is not limited to, any one or more of the following: a High Definition Multimedia Interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a Universal Serial Bus (USB) input interface, an RGB port, and the like. It may also be a composite input / output interface formed by the above-mentioned multiple interfaces. For example, in order to enrich the display type of the display device 200, an HDMI interface and a USB interface may be provided on the display device 200, wherein the HDMI interface can be connected to a device having video and audio input / output functions, and can realize corresponding audio and video output / input functions through the HDMI driver. The USB interface can be connected to a device having signal and data transmission functions and interactive functions, and can access signals, transmit data, and perform specific interactive actions based on the USB driver.

[0066] In some embodiments, the display 260 may include a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0067] In some embodiments, the display 260 may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection display, or a projection device and a projection screen.

[0068] In some embodiments, the display device 200 may also support touch interaction, that is, a touch interaction module may be provided on the display device 200. The touch interaction module may be a built-in module or an external module. For example, in order to implement touch interaction operations, for a display device 200 with a built-in touch interaction module, a touch panel (TP) may be configured on the display 260. The touch panel can receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to pre-set interaction strategies. Of course, the display device 200 may also be an integrated touch screen.

[0069] For a display device 200 with an external touch interaction module, it can connect to a terminal, device, or component with touch functionality via the device interface 240 of the display device 200. For example, the USB or USB-C interface of the display device 200 can be connected to a touchpad. The touchpad can receive real-time touch data input by the user and transmit the touch data to the display device 200 via the USB channel. The display device 200 can then perform interactive responses based on the received touch data in conjunction with a touch application, enabling the display device 200 to support touch interaction operations.

[0070] In some embodiments, the communication device 220 is a component that can be used to communicate with an external device or server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 depending on the communication methods supported. For example, if the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 that includes WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 that includes Bluetooth functionality.

[0071] The communication device 220 can establish a communication connection between the display device 200 and an external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to an external device via a data cable, an interface, or other components. A wireless connection can connect the display device 200 to an external device via a wireless signal or wireless network. The display device 200 can establish a connection with an external device directly or indirectly through a gateway, router, or connection device.

[0072] In some embodiments, the processor may include a central processing unit, a video processor, an audio processor, a graphics processor, a random access memory (RAM), a read-only memory (ROM), and first to nth interfaces for input / output. At least one processor 250 may control the operation of the display device and respond to user operations through various software control programs stored in the memory. At least one processor 250 may control the overall operation of the display device 200.

[0073] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.

[0074] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0075] In some embodiments, the audio output interface 270 may be a local speaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may further be provided with an external audio output terminal, through which the audio output device may be connected to the display device 200 to output the sound of the display device 200.

[0076] In some embodiments, the user interface 280 is an interface for receiving control inputs, and can be used to receive instructions input from a user.

[0077] Figure 3 is a block diagram of the hardware configuration of the control device in Figure 1 according to some embodiments of the present application. As shown in Figure 3, the control device 100 may include at least one processor 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0078] The control device 100 can be configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, thereby acting as an interactive intermediary between the user and the display device 200 .

[0079] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0080] In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform similar functions as the control device 100 after installing an application for controlling the display device 200 .

[0081] The at least one processor 110 may include a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The at least one processor 110 may be used to control the operation and operation of the control device 100, as well as communication and coordination between internal components and external and internal data processing functions.

[0082] Under the control of at least one processor 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, a near field communication (NFC) module 133, or other near field communication modules.

[0083] The user input / output interface 140 , wherein the input interface may include at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 , and other input interfaces.

[0084] In some embodiments, the control device 100 may include at least one of a communication interface 130 and an input / output interface 140. The control device 100 may be configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0085] The memory 190 can be used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of at least one processor. The memory 190 can also store various control signal instructions input by the user.

[0086] The power supply 180 can be used to provide operating power support for various components of the control device 100 under the control of the controller.

[0087] To facilitate user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program that can be used to manage and control the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0088] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0089] The operating system can be divided into different modules or levels according to the functions implemented. For example, Figure 4A is a schematic diagram of a software configuration in the display device in Figure 1 according to some embodiments of the present application. As shown in Figure 4A, in some embodiments, the system is divided into four layers, from top to bottom, namely the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0090] In some embodiments, the application layer can be used to provide services and interfaces for applications, so that the display device 200 can run applications and interact with the user based on the applications. The application layer can run at least one application, which can be a window program included with the operating system, a system settings program, a clock program, or other applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0091] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining actions for applications in the application layer. Through the API, applications can access system resources and services during execution.

[0092] As shown in FIG4A , the application framework layer in some embodiments of the present application may include a view system, managers, content providers, etc., wherein the view system may design and implement the interface and interaction of the application, and the view system may include lists, grids, text boxes, buttons, etc. The manager may include at least one of the following modules: an activity manager may be used to interact with all activities running in the system; a location manager may be used to provide system services or applications with access to the system location service; a package manager may be used to retrieve various information related to the application packages currently installed on the device; a notification manager may be used to control the display and clearing of notification messages; and a window manager may be used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0093] In some embodiments, the activity manager can be used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager can be used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0094] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0095] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as shown in FIG4A , the kernel layer can be configured with hardware drivers, and the drivers included in the kernel layer can be at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0096] It should be noted that the above examples are merely a simple division of operating system functions and do not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0097] Figure 4B is a schematic diagram of another software configuration in the display device in Figure 1 according to some embodiments of the present application. In some embodiments, the system of the display device 200 can be divided into three layers, namely, application layer, middleware layer and hardware layer from top to bottom.

[0098] The application layer mainly includes commonly used applications on the TV and the application framework. Among them, commonly used applications can mainly be applications developed based on the browser, such as HTML5 APPs; and native applications (Native APPs); the application framework (Application Framework) is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0099] Native apps can support online or offline, message push or local resource access.

[0100] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can leverage the fundamental services (functions) provided by system software to connect various parts of the application system or different applications on the network, enabling resource and function sharing.

[0101] The hardware layer mainly includes the Hardware Abstraction Layer (HAL) interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, Wi-Fi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0102] The hardware layer of the display device 200 can also deploy other device interfaces, hardware, and drivers based on the specific display requirements of the display device 200. For example, to enrich the display types of the display device 200, the display device 200 can be provided with an HDMI interface and a USB interface. The HDMI interface can connect to devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver. The USB interface can connect to devices with signal and data transmission functions and interactive functions, and based on the USB driver, it can access signals, transmit data, and perform specific interactive actions.

[0103] For a display device 200 that supports touch interaction operations, the touch data input by the user can be received through the touch interaction module. According to the interaction principle of the touch interaction module, the touch interaction module can detect relevant parameters in the touch action. For example, the display 260 of the display device 200 is provided with a capacitive touch layer. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touched position will change. At this time, the touch interaction module can record the position of the capacitance change and record the time when the capacitance change occurs in combination with the timer in the controller to generate touch data.

[0104] In some embodiments, the physical signals detected by the touch interaction module need to be converted before they can be read by the controller 250 of the display device 200. For example, an analog signal of capacitance change needs to be converted into a digital signal before being transmitted to the controller 250 of the display device 200. To adapt to the screen display process of the display device 200, the touch data detected by the touch interaction module also needs to be coordinate mapped so that the location corresponding to the touch data can correspond to the pixel points in the display 260 of the display device 200 or the user interface. For example, if the coordinate position detection accuracy of the touch interaction module is 65536×65536, and the resolution of the display 260 of the display device 200 is 3840×2160, then after the touch interaction module detects the touch data, the touch point position in the width and height directions can be mapped according to the ratio of 65536 / 3840 and 65536 / 2160, respectively, so that the display device 200 can perform relevant operation responses based on the mapped touch point position coordinates.

[0105] Among them, the touch data may include different operation parameters according to the user's input method. For example, as shown in Figure 5, the touch data may include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. During a touch interaction process, the touch event types that can be detected by the touch interaction module may include: a down event, a move event, and an up event of a touch action. Corresponding to the entire touch trajectory, a starting point [down; x1, y1, t1], a trajectory point [move; x2, y2, t2], and an end point [up; x3, y3, t3] can be formed. Each touch event records the corresponding touch time t and touch position (x, y).

[0106] It should be noted that, since the user's finger or touch device is in surface contact with the touch interaction module when performing touch interaction operations, that is, the touch interaction module can detect a contact area, which may include multiple contact points. Therefore, when the display device 200 performs interactive control, it can detect (or specify) a contact point in the contact area as a touch point. For example, when the contact area is an elliptical area formed when the user's finger contacts the touch interaction module, the display device 200 can traverse the contact point coordinates in the elliptical area to determine the extreme values ​​of the coordinates in two mutually perpendicular directions to delineate the major axis and minor axis of the elliptical area, and use the contact point at the intersection of the major axis and the minor axis as the touch point.

[0107] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, number of touch points and touch event types, so as to detect the user's touch input gesture, thereby performing different interactive responses for different input gestures. For example, when the touch data detected by the touch interaction module has a touch point number of 1 and the touch event type can only include a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold. Moreover, if the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 executes the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0108] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch duration, touch location, number of touch points, and touch event type, such as a double-click gesture, a slide gesture, a long press gesture, a multi-finger click gesture, and a multi-finger slide gesture. Different interaction gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, and slide trajectory shape. Different touch parameters can be used to perform different interactive controls.

[0109] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection based on the dynamic change rules of touch time, touch position, number of touch points and touch event type. For example, when the touch interaction module detects that the touch data input by the user includes multiple touch points, and the touch position of each touch point is close to each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grab gesture. Based on complex gesture detection, the display device 200 can perform more interactive responses. For example, when a multi-finger sliding touch gesture is detected and the touch gesture is identified as a grab operation for the image view, the display device 200 can respond to the grab gesture and perform a control action to select and move the image display position.

[0110] In some embodiments, touch gestures can also be combined with touch positions to form touch interaction operations based on specific positions. For example, when the touch interaction module detects that the down event position during the touch operation is at the top of the display 260, the touch point position corresponding to the move event moves downward. When the up event is detected, the display device 200 can determine that the current touch gesture is a swiping down from the top of the screen. Therefore, the display device 200 can control the display 260 to present a drop-down list in response to the touch gesture.

[0111] Some touch interaction gestures can be used to perform interactive actions based on the operating system of the display device 200, and are referred to as global gestures. For example, swiping down from the top of the screen in the example above on the display device 200 to bring up a drop-down list. Some touch interaction gestures can be used to perform interactive actions based on specific touch applications. For example, when the display device 200 is running a whiteboard application, a user-inputted slide touch action can be used to draw a track line on the whiteboard.

[0112] Touch applications can be system applications or third-party applications developed based on the specific display device type and user needs. Touch applications can be installed on the display device 200 and launched and run under user control. For example, a user can control the display device 200 to run the whiteboard application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0113] It should be noted that a whiteboard application can be a standalone application, an associated application, or a functional module within a touch application. Different whiteboard applications can have different startup or invocation methods. For example, a user can launch a standalone whiteboard application through the application interface of the display device 200; a user can also launch a linked whiteboard application by performing a "whiteboard" operation on a conference application; and a user can control the conference application to launch the whiteboard functional module by clicking the "whiteboard" icon on the conference application.

[0114] In some embodiments, the whiteboard application is a graphic object drawn according to each touch action input by the user. Among them, for the display device 200 that supports touch interaction operations, the touch action input by the user can be received through the touch interaction module. For example, the touch interaction module is a touch layer set on the screen of the display 260 of the display device 200. A touch action can refer to the process from the user's finger (or stylus) contacting the touch layer to the user's finger (or stylus) leaving the touch layer. That is, if the user inputs a click action, a touch action should include a down event and an up event. If the user inputs a sliding action, a touch action should include a down event, a move event, and an up event. Corresponding to the entire touch track, a starting point, a track point, and an end point can be formed.

[0115] Different forms of whiteboard applications may present different whiteboard application interface forms after startup. For example, when a user starts a whiteboard application through an associated application form, the display device 200 may automatically enter dual-screen (or split-screen) mode. In dual-screen (or split-screen) mode, the display device 200 may display the user interface through two display areas, i.e., the main screen displays the original application (such as a conference application) interface, and the secondary screen displays the whiteboard application interface.

[0116] After the whiteboard application is running, the display device 200 can switch to displaying a whiteboard interface. As shown in FIG6 , the whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, shape, line shape, color, erase, clear screen, and selection. Users can click any control to draw graphics according to their specific drawing needs.

[0117] In some embodiments, when a user selects a drawing tool control for drawing a trajectory line in the whiteboard interface, the display device 200 generates a path graphic drawing instruction, also known as a touch interaction instruction, and in response to the touch interaction instruction, draws a trajectory line based on the detected touch points, forming a path-like graphic, i.e., a path graphic. For example, when a user selects an operation control such as a pen or line, the display device 200 generates a path graphic touch interaction instruction and, in response to the touch pouring instruction, draws the corresponding path based on the touch points.

[0118] In some embodiments, the display device 200 can call a graphics library to draw a trajectory line in response to a touch interaction instruction of a path graphic. For example, the graphics library can be a Canvas API. The trajectory line drawn by the display device 200 through the graphics library can use different types of line caps (line caps). The line caps are the line ends at the starting and ending points of the trajectory line and can be different shapes such as circles and squares. For example, taking the Android system as an example, as shown in Figure 7, line caps can include flat heads (Butt), round heads (Round), and square heads (Square). Among them, the default line cap of the graphics library can be a flat head line cap. When drawing a flat head line cap, the display device 200 does not need to do any additional processing, that is, the trajectory line drawn by the Canvas API defaults to a flat head line cap; when drawing a round head line cap, the display device 200 can use the Canvas API to draw a small circular or elliptical covering at the starting and ending points of the trajectory line; similarly, when drawing a square head line cap, the display device 200 can use the Canvas API to draw a small rectangular covering at the starting and ending points of the trajectory line.

[0119] It is understood that the caps of the track lines drawn by the display device 200 can also be other types of caps, and the shapes of the caps can be customized. For example, the display device 200 can also use the Path class to customize the shapes and draw these shapes at the starting point and end point of the track line. This application does not impose any restrictions on this.

[0120] In some embodiments, the display device 200 is configured with a specific settings object or structure that can be used to store trajectory drawing information. This drawing information may include attributes such as color values, baseline line width, and line caps. For example, a user can set the color, line width, and line cap of a trajectory using the drawing controls in the whiteboard application interface. The whiteboard application interface then stores the parameters set by the user based on the drawing controls in the settings object. Subsequently, when the display device 200 detects a touch interaction instruction for a path graphic, the parameters stored in the settings object are passed to a preset drawing function, which then calls the Canvas API based on these parameters to draw the trajectory of the path graphic.

[0121] In order to maintain the smoothness of the trajectory line, in some embodiments, the drawing function of the display device 200 is set according to the Bezier curve, that is, the display device 200 draws the trajectory line of the path graphic based on the Bezier curve. For example, the display device 200 can respond to the touch interaction instruction of the path graphic, set the starting point, end point and control point according to the detected touch point, and then construct a Bezier curve based on the set starting point, end point and control point. Then, the trajectory line is drawn based on the constructed Bezier curve to generate a path graphic. For ease of distinction, some embodiments of the present application represent the above-mentioned Bezier curve used to draw the trajectory line of the path graphic as a basic Bezier curve, that is, the display device 200 can draw the corresponding trajectory line according to the basic Bezier curve when drawing a path-type graphic.

[0122] As shown in FIG8 , the display device 200 can render and display a drawing picture in real time through an image generation (Image Producer) module, an image rendering (Surface Flinger) module, a synthesis (Composer) module, a video processing unit (VPU hardware) module, and a display (screen).

[0123] In the process of displaying the drawn picture, the graphics interface (openGLES) of the image generation module can be connected to the whiteboard application to generate the drawn image content and pass the image content to the image rendering module. The image rendering module can be used to control the display 260 display content, that is, to render the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, thereby forming a picture frame for display in the synthesis module. Among them, the synthesis module can form the picture content by software or hardware. For example, the synthesis module can be a hardware synthesis (Hardware Composer, HWC) module.

[0124] The synthesis module can store image frames in both the frame buffer and the video buffer, and the video processing module can generate image content on the corresponding layers. Specifically, the image frames generated during video playback are generated on the video plane, while the image frames of other image frames (such as touch traces and graphics) are generated on the primary OSD plane. Finally, the display device 200 uses the blending unit of the video processing module to generate the final image, which is then displayed on the display 260.

[0125] Since the whiteboard application will respond to the user's touch operation in real time and present the user's touch trajectory, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. To this end, in some embodiments, the display device 200 can present a real-time painting effect by setting an acceleration layer. That is, in the video process module of the display device 200, an additional extended display layer (external OSD plane) can be set on the basis of the main OSD layer and the video layer. Among them, the extended display layer can be configured with different layer characteristics according to the specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the area, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0126] For example, when display device 200 is running a whiteboard application, to improve the real-time response speed of line drawing, the touch track can be drawn in real time on an extended display layer (acceleration layer) after detecting a user touch operation. As shown in Figure 9, the acceleration layer can be displayed at a higher level than other layers. When the user completes the touch input, the line drawn by the acceleration layer is updated to other layers. This can reduce input latency and improve the real-time display effect of the whiteboard application drawing process.

[0127] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer, maintaining real-time response speed. After the user inputs the touch interaction action, the content displayed by the acceleration layer can be released directly, and the OSD layer directly forms the drawn graphics, so there is no need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0128] In some embodiments, in order to adapt to different user needs, the whiteboard application can also dynamically configure different interface contents. For example, as shown in Figure 10, a whiteboard application run by a display device 200 used in the field of education may include a "teaching tool" option. When the user clicks on the teaching tool option, a teaching tool list may pop up. The teaching tool list may include tool options of the type "ruler", "set square", "protractor", etc. After the user clicks on the "set square" tool option, the "set square" tool can be called to assist in drawing. At this time, the display device 200 can add a movable set square tool graphic to the whiteboard drawing interface, and when the user's touch operation trajectory approaches the edge of the set square tool graphic, a straight line segment is drawn along the edge of the set square tool graphic.

[0129] In some embodiments, the whiteboard application can also store the graphic objects drawn by each user touch action. Among them, a touch action refers to the process from the user's finger (or stylus) contacting the touch layer to the user's finger (or stylus) leaving the touch layer. That is, if the user inputs a click action, a touch action should include a down event and an up event. If the user inputs a sliding action, a touch action should include a down event, a move event, and an up event. Each touch action can generate a drawing object, and the whiteboard application can record the drawing object, making it easier for the user to perform actions such as selection, undo, and redo.

[0130] When storing graphics, the whiteboard application can also associate information with graphic objects based on the graphic's characteristics and input state. For example, after a user draws a line, the whiteboard application can assign a shape identifier to the line object. The shape identifier can be associated with the line object and stored, and can be used by other modules of the whiteboard application or other applications to facilitate operations such as object selection, object retrieval, and object deletion.

[0131] In some embodiments, the whiteboard application interface may also provide drawing assistance tools and menus for adjusting drawing content. For example, the whiteboard application interface may include a "select" tool. When the user clicks the "select" tool icon, the whiteboard application can be controlled to enter object selection mode. In object selection mode, the display device 200 can call drawing objects that intersect with the touch position in the user's touch interaction operation down event, and / or move event, and / or up event, to achieve the purpose of multi-graphic selection.

[0132] In some embodiments, the whiteboard application may also support multiple gesture interactions and provide different control functions for different gesture interactions. For example, in response to a multi-finger swipe operation by the user, the whiteboard application may move the whiteboard drawing area along the multi-finger swipe trajectory. The whiteboard application may also execute an erase function in response to a full palm swipe operation by the user, erasing the drawn graphics / objects covered by the palm swipe trajectory.

[0133] In some embodiments, the whiteboard application can also render and process the graphics drawn by the user to make the graphics drawn by the user more detailed or accurate. For example, as shown in Figure 11, the user selects the "pen stroke effect" option. When the user writes text or draws on the whiteboard application, the whiteboard application can set a pen stroke effect for the handwriting. Among them, the pen stroke effect refers to the effect that the end of the drawn line has a gradually tapering effect. The whiteboard application can dynamically adjust the pen stroke effect according to parameters such as the movement trend and movement speed of the touch track during the user's drawing process. For example, the faster the user writes, the longer the pen stroke length; the slower the user writes, the shorter the pen stroke length.

[0134] In some embodiments, in order to improve the diversity of line drawing, the whiteboard application can also set a variety of line drawing types, and then can draw different types of lines according to the touch actions input by the user. For example, the user can click the "Line Shape" control in the toolbar provided by the whiteboard application to call out the line shape selection menu, and click any line shape option according to the specific drawing needs to set the line drawing type. For example, as shown in Figure 12, the user selects the "Decorative Head" option. When the user writes text or draws on the whiteboard application, the whiteboard application can add a decorative head at the end of the handwriting. In addition, as shown in Figure 13, the user can also select the "Dotted Line" option. When the user writes text or draws on the whiteboard application, the whiteboard application can process the handwriting so that the handwriting presents a dotted line effect.

[0135] The process of rendering and processing the graphics drawn by the user by the whiteboard application can be implemented based on a preset graphics algorithm model. Among them, the graphics algorithm can be an algorithm built based on specific shapes such as Pascal's triangle, Bezier curve and knife shape. For example, the pen stroke effect can be drawn by a water drop-shaped pen stroke model or by generating a water drop-shaped pen stroke through a Bezier curve to draw a tip effect, and the tip effect is added to the end of the drawn graphic. For example, the whiteboard application can detect the shape of the user's drawn graphics within a preset drawing cycle, and replace the drawn shape with a standard graphics model to improve the standardization of the drawn graphics. To this end, a graphics recognition model can be built into the whiteboard application, and all the lines drawn by the user within the preset drawing cycle can form an image to be recognized, and the image to be recognized can be processed by the graphics recognition model to identify standard graphics similar to the drawn graphics within the drawing cycle, thereby replacing the drawn graphics displayed in the whiteboard application interface.

[0136] For the pen stroke effect rendered by the whiteboard application, the display device 200 may not add the pen stroke effect in real time during the drawing process, but may add the pen stroke effect after the touch is raised (i.e., a touch interaction is completed). The pen stroke effect can be drawn based on the movement speed or contact area of ​​the track point in the touch track. When the pen stroke is drawn based on the movement speed, the handwriting width will be too thin when the writing speed is too fast, and the pen stroke effect will not be obvious when the writing speed is too slow. When drawing lines based on the contact area, there will also be problems such as a large contact area when the pen is put down, the handwriting is too thick, and the pen stroke effect will not be obvious when the change is slow.

[0137] For example, after the user touches and raises his hand, the whiteboard application of the display device 200 can use the last N points or the trajectory points within the last N milliseconds (ms) to draw a pen stroke effect based on the speed or contact area. However, because the movement speed or contact area changes too drastically or too slowly, the pen stroke effect will be unsatisfactory, and there will be no pen stroke effect during the drawing process, resulting in a poor user experience.

[0138] In some embodiments, the display device 200 can also draw a pen stroke effect in real time during the drawing process. That is, the display device 200 can continuously draw the latest path as a thin line, then supplement the previous path with a thick line, and then use the method of filling in points to form the pen stroke. For example, the display device 200 draws the latest newPath as a thin line. If the touch device continues to move and a new touch point appears, forming a new path, the previous newPath will become the previous path, named lastPath. The lastPath needs to be redrawn as a thick line with a set line width. However, due to the different line widths of the latest path, newPath, and the previous path, lastPath, circular strokes need to be added to smooth them. The smaller the interval between the circular strokes, the smoother it is, but the slower the speed is, making it difficult to strike a balance between effect and speed. In addition, the above-mentioned method of drawing the pen stroke effect requires the latest path and the previous path to be connected. Due to the different line widths of the two, the trajectory line drawn by the display device 200 is prone to problems such as burrs or sudden changes, resulting in an uneven connection of the pen stroke effect, thereby affecting the pen stroke effect of the path graphic and reducing the user's interactive experience.

[0139] According to some embodiments of the present application, when the display device runs a whiteboard application, a pen stroke model can be created according to the touch area and a pen stroke effect can be drawn in real time based on the pen stroke model. Figure 14 is a flow chart of a trajectory line drawing method for a display device according to some embodiments of the present application. As shown in Figure 14, the display of the display device can be configured to display a user interface; the display can be connected to a touch interaction module, and the touch interaction module can be used to obtain touch interaction instructions input by the user; the memory can be configured to store computer instructions and / or data associated with the display device; at least one processor can be configured to execute computer instructions to enable the display device to perform some or all of the following method steps, so as to maintain the pen stroke effect during the drawing process through the pen stroke model, and the pen stroke effect is uniform and smooth, thereby improving the user experience:

[0140] S1401: Responding to a touch interaction instruction input by a user, obtaining a touch area and a touch track.

[0141] Among them, touch interaction instructions refer to touch actions output by the user through the touch interaction module. For example, when the user inputs a sliding touch action, the touch interaction module can generate a touch interaction instruction by detecting the sliding action. The touch trajectory can be the trajectory formed by the change in the contact position between the user's finger or touch device and the touch interaction component when performing a touch interaction operation. The touch area can be the area formed by the touch interaction action, which can be the entire touch trajectory or a section of the touch trajectory formed by the touch sliding process, or the contact area formed by a single touch process.

[0142] S1402: Generate a pen stroke model according to the touch area.

[0143] The pen tip model may be a dot matrix model generated based on touch points, contour points, and spare points in the touch area. A touch point may be at least one contact point in the touch area, a contour point may be at least one contact point located at the edge of the pen tip in the touch area, and a spare point may be a point that divides the distance between the touch point and the contour point into predetermined equal parts.

[0144] To generate the pen tip model, in some embodiments, the display device may extract the centerline of the touch area, where the centerline of the touch area is parallel to the direction line of the touch trajectory. Based on the centerline of the touch area, a preset number of touch points are extracted from the touch area. The preset number can be set based on the shape of the pen tip model. Contour points are then generated based on the touch points and the line width of the touch trajectory. Based on the shape of the pen tip model, spare points may be added between the touch points and the contour points.

[0145] For example, when a user controls a touch point to move on a touch interaction module of the display device 200, the touch interaction module may collect touch data and send the collected touch data to the at least one processor 250 of the display device 200. The at least one processor 250 may construct a pen stroke model using the last N points in the collected touch data, or N points predicted based on previously collected touch data, or points within N ms.

[0146] It should be noted that the touch area described above can be defined in a broad sense, i.e., the entire touch trajectory or a portion of a touch trajectory formed by a user's touch operation, or in a narrow sense, i.e., the contact area on the touch interaction module formed when a user's finger or touch device comes into contact with the touch interaction module. The methods for extracting touch points within the touch area vary depending on the meaning of the touch area.

[0147] In some embodiments, if the touch area is a touch track formed by multiple touch points, the display device can extract N consecutive touch points from the touch track. A touch point can be a point in the contact area formed on the touch interaction module when a user's finger or touch device contacts the touch interaction module.

[0148] The display device can extract a preset number of contact areas from the touch track according to a pen tip refresh cycle, wherein the pen tip refresh cycle can be greater than or equal to the sampling cycle of the touch interaction module. A direction line of the touch track can be delineated based on the line width of the touch track. The touch points are then sequentially extracted from the contact areas along the direction line of the touch track, wherein the touch points can be contact points within the contact areas that are located on the direction line.

[0149] For example, as shown in FIG15 , when the contact area is an ellipse, the display device 200 can use the intersection of the major and minor axes of the ellipse as the touch point. Furthermore, as the user's touch movement operation progresses, multiple contact areas can be formed along the touch trajectory in chronological order. A touch point can be determined in each contact area. The most recent touch point and the N-1 touch points preceding the current touch point can then be extracted to construct a pen stroke model.

[0150] By extracting the touch points from the touch trajectory, the constructed pen stroke model can be made to conform to the pen stroke effect at one end and the line width form at the other end. Therefore, during the user's touch movement, the pen stroke effect drawn by the pen stroke model can be better connected with the previously input touch point position, which can increase the continuity of the pen stroke effect.

[0151] In some embodiments, if the touch area is the contact area between the user's finger or the touch device and the touch interaction module, the display device can first determine the center line of the contact area according to the moving direction of the touch trajectory, and then extract N contact points in sequence along the center line at equal intervals or specific intervals, and use the extracted N contact points as touch points to construct a pen stroke model.

[0152] The display device may traverse the coordinates of the contact points in the contact area to obtain edge contact points. A centerline of the contact area may be delineated based on the edge contact points. A predetermined number of contact points may be extracted from the contact area along the centerline of the contact area to serve as the touch points. The centerline of the contact area is parallel to the direction line of the touch track.

[0153] For example, as shown in FIG16 , when a user inputs a touch interaction command by contacting the touch screen with a finger, the finger's contact with the touch interaction module forms an elliptical contact area. As the user's touch moves, the major axis of the contact ellipse becomes approximately parallel to the direction of movement of the touch trajectory. Display device 200 can then extract N equally spaced contact points within the contact ellipse as touch points, and construct a pen stroke model based on these N touch points.

[0154] The pen stroke model constructed by extracting touch points in the contact area can better adapt to the user's input of touch interaction commands. The contact shape of the finger or touch device and the touch interaction module can act on the pen stroke model, so that the pen stroke effect can be dynamically adjusted according to the contact method, thereby improving the flexibility of pen stroke drawing.

[0155] Because the more touch points, contour points, and spare points included in the pen stroke model, the more detailed the pen stroke graphic and the better the corresponding pen stroke effect, in some embodiments, the application ID of the calling whiteboard application can be obtained, and the current usage scenario can be queried based on the application ID. If the current usage scenario is the first scenario, a first number of touch points can be extracted from the touch area; if the current usage scenario is the second scenario, a second number of touch points can be extracted from the touch area. The second number can be smaller than the first number.

[0156] The display device can detect the current usage scenario based on the application tag that calls the whiteboard application. When the whiteboard application is used in a first scenario that requires a high pen stroke effect, the display device can increase the number of touch points collected in the touch data. For example, for a usage scenario with demonstration functions such as calligraphy and painting, the display device can increase the value of N when detecting this usage scenario to improve the fineness of the pen stroke graphics. When the display device detects that the current usage scenario is a second scenario that requires a low pen stroke effect, the display device can reduce the number of touch points collected in the touch data to reduce the data processing volume and increase the speed of calculating the pen stroke effect.

[0157] For example, the display device 200 can monitor the current application call status. When it is detected that the application calling the whiteboard application is a painting application, the application ID of the painting application can be read, and the usage scenario to which the painting application belongs can be detected according to the application type table pre-stored in the display device 200. That is, according to the application ID of the painting application, it is determined that the current usage scenario is drawing mode, which belongs to the first scenario. At this time, the number of touch points of the pen stroke model can be set to N=6, that is, 6 touch points can be extracted at the midline position of the touch area, and a more complex pen stroke model can be constructed based on the extracted 6 touch points.

[0158] If the application calling the whiteboard application is detected as a conference application, the display device 200 can read the application ID of the conference application and, based on the application ID, determine that the usage scenario of the conference application is meeting mode, which belongs to the second scenario. Therefore, the display device 200 can set the number of touch points N in the pen stroke model to 3, that is, extract three touch points at the center line of the touch area, and construct a simpler pen stroke model based on these three extracted touch points.

[0159] Since the pen tip effect is that the line width becomes thinner as it is drawn to the end, if only the pen tip part is used to build the model, the pen tip model will not be coherent during the movement of the touch point, and it will not be able to seamlessly connect with the previous model. Therefore, as shown in Figure 17, the display device 200 can generate a pen tip model through a dot matrix composed of touch points, contour points and spare points. For example, a pen tip model composed of 6 touch points is used, where P10, P20, ..., P60 are touch points located at the midline of the touch area. The points located on the contour line of the touch area are contour points. The contour points are the coordinates of the tangent points of two circles with two adjacent touch points such as P10 and P20 as the center of the circle and the line width as the diameter. That is, P11, P12, P21 and P22 are the contour points of touch point P10 and touch point P20 respectively. Among them, the line width can be set according to the long axis, short axis or fixed value of the touch point.

[0160] To achieve the effect of a pen tip tapering toward the tip, the pen tip model can be divided into N parts based on the Pascal's triangle principle. Each divided node can then be used as a backup point, stored in a two-dimensional array. For example, as shown in Figure 18, the contour point pairs P11', P11", P12', P12", P13', and P13" respectively occupy 4 / 10, 7 / 10, and 10 / 10 of the touch area diameter d (line width). This means that the thinnest part of the pen tip has a line width of 0, while the thickest part has the same line width.

[0161] S1403: Draw a pen tip trajectory line at the end of the touch trajectory according to the pen tip model.

[0162] After constructing the pen stroke model, the display device can render the pen stroke effect based on the pen stroke model. The display device can run a whiteboard application, and use the contour points involved in the contour line connection calculation based on the Pascal's triangle as quasi-contour points. The midpoints of adjacent contour points can be used as the starting and ending points of the Bezier curve respectively, and connected to form a pen stroke model.

[0163] Specifically, in some embodiments, quasi-contour points are extracted from the brush stroke model. These quasi-contour points may be touch points, contour points, or spare points located at the edge of the brush stroke shape. Midpoints between adjacent quasi-contour points are then calculated, and a Bezier curve is constructed using these midpoints and / or the quasi-contour points as starting and ending points to generate a brush stroke graphic.

[0164] For example, as shown in Figure 18, at the touch point P10, the outermost contour points P13' and P13" can be selected as quasi-contour points; at the touch point P20, the outermost contour points P23' and P23" can be selected as quasi-contour points; at the touch point P30, the outermost contour points P33' and P33" can be selected as quasi-contour points; at the touch point P40, the outer alternative points P42' and P42" can be selected as quasi-contour points; at the touch point P50, the inner alternative points P51' and P51" can be selected as quasi-contour points; at the touch point P60, the touch point P60 is directly selected as the quasi-contour point. Therefore, a dot matrix model of a pentagonal structure, that is, a pen tip model, can be constructed.

[0165] Based on the pen stroke model, as shown in Figure 19, the display device 200 can use the Pascal's triangle principle to draw pen stroke effects at the midpoints between adjacent quasi-contour points, such as the midpoint PP1 between P13' and P23', the midpoint PP2 between P23' and P33', the midpoint PP3 between P33' and P42', the midpoint PP4 between P42' and P51', the midpoint PP5 between P51' and P60... and based on the midpoints PP1, PP2...PP1', PP2'..., and the quasi-contour points as the starting and ending points of the Bezier curve.

[0166] It can be seen that in the above embodiment, by generating a dot matrix model based on the touch points, contour points and spare points in the touch area, the pen stroke style can be determined according to the shape of the touch area, and the pen stroke shape can be formed based on the midpoints of adjacent contour points as the starting and ending points of the Bezier curve, thereby adding a pen stroke effect to the touch trajectory in real time.

[0167] As the user's touch slides, the touch point continues to move forward, creating a new touch area. Based on this new touch area, a new touch point P can be determined. The pen tip model can then be sequentially advanced along the touch trajectory. The display device can then use a backup point or contour point, based on the positional relationship between the touch points, to replace the previously determined quasi-contour point to update the position of the pen tip model. Specifically, the display device can calculate the newly added point and its associated backup point as the new starting point, replacing the original starting point with a backup point at a preset equal inner position. Contour points that are also quasi-contour points are replaced by adjacent contour points.

[0168] That is, in some embodiments, the display device may obtain a new touch point in the touch trajectory, and then generate contour points and backup points associated with the new touch point based on the new touch point. The new touch point, the contour point, and the backup point are then used to replace the quasi-contour point to update the position of the pen stroke model.

[0169] For example, as shown in FIG20 , when the user forms a touch track through a touch operation, the display device 200 can detect a new touch point P, and the new touch point P can replace the original starting point P60 as the new starting point of the pen stroke model. Based on the position of the new first point P, the inner spare points P61 and P61' corresponding to the touch point P60 can be used as new quasi-contour points, that is, the original first point P60 is no longer used as a quasi-contour point, and is replaced by a contour point pair with a line width of 4 / 10 of the line width of the P60 position; the outer spare points P52 and P52' corresponding to the touch point P50 are used as new quasi-contour points, that is, the spare point pair at 4 / 10 of the line width of the P50 position in the original pen tip model is no longer used as a quasi-contour point, and is replaced by a spare point pair at 7 / 10 of the line width of the P50 position; the contour points P43 and P43' corresponding to the touch point P40 are used as new quasi-contour points, that is, the spare point at 7 / 10 of the line width of the original P40 position is no longer used as a quasi-contour point, and is replaced by a contour point pair at the P40 position, and so on and so forth, until the original end points P13 and P13' in the original pen tip model are removed from the pen tip model.

[0170] Based on the new pen stroke model, the display device can confirm the midpoints of adjacent quasi-contour points using the Pascal's triangle principle in the manner provided in the above-mentioned embodiment, and can draw pen stroke effects using the quasi-contour points in the newly generated pen stroke model as the start and end points of the Bezier curve. Because the contour point at the 100% line width position corresponding to the touch point is used as the quasi-contour point when the end of the pen stroke model replaces the old model with the new one, the pen stroke effects drawn by two adjacent pen stroke models can be connected using a connecting straight line, allowing the newly generated pen stroke model to seamlessly connect with the previous pen stroke model.

[0171] In some embodiments, the display device can preset a plurality of different forms of brush stroke models. Since the brush stroke model is a dot matrix model composed of touch points, contour points, and spare points, the number of touch points, contour points, and spare points contained in different forms of brush stroke models, the positional relationship between points, and the shape of the dot matrix composed of multiple points are different. For example, to obtain an ink brush stroke effect, the display device 200 can adjust the number, positional relationship, and shape of the touch points, contour points, and spare points according to the brush tip structure to generate a water droplet-shaped brush stroke model; to obtain a fountain pen effect, the display device 200 can adjust the number, positional relationship, and shape of the touch points, contour points, and spare points according to the brush tip structure to generate a triangular brush stroke model.

[0172] Since the input speed of the user when performing a touch action is different, and the sampling frame rate of the touch interaction module corresponding to the display device is fixed, the distance between two adjacent touch points in the touch trajectory will be non-fixed under different input speeds. For example, when the user performs a touch action on the touch interaction component, the finger movement speed is v1, and the sampling frame rate of the touch interaction module is F. Then the interval between two samplings of the touch interaction component is 1 / F. Based on this, the distance between two adjacent touch points is v1 / F. It can be seen that when the finger movement speed is different, the distance between two adjacent touch points is also inconsistent, that is, the greater the finger movement speed, the greater the distance between adjacent touch points.

[0173] In order to make the pen stroke effect seamless, when it is detected that the interval between the touch points is relatively large, the interpolation method can be used to fill in the points so that the pen stroke can be more seamlessly connected during the movement. In some embodiments, the display device can obtain the interval distance between the new touch point and the original starting point of the pen stroke model, wherein the original starting point can be a quasi-contour point at the edge position along the touch track direction in the pen stroke model. If the interval distance is greater than the distance threshold, an interpolation point can be added between the new touch point and the original starting point of the pen stroke model, and the contour points and spare points associated with the interpolation point can be generated based on the interpolation point, and the pen stroke model can be reconstructed based on the interpolation point. If the interval distance is less than or equal to the distance threshold, the pen stroke model can be constructed based on the new touch point, that is, the step of generating the contour points and spare points associated with the new touch point based on the new touch point is performed.

[0174] After detecting a new touch point, the display device can calculate the distance between the new touch point and the original starting point in the original pen stroke model. The device then determines the distance. If the distance is greater than or equal to a distance threshold, the device can interpolate at least one interpolation point based on the distance and determine associated contour points and backup points based on the line width. The interpolation point is then used as the new starting point of the new pen stroke model to construct the pen stroke model.

[0175] For example, as the user inputs a touch interaction instruction, when the display device 200 detects a new touch point P', it can calculate the interval distance L between the new touch point P' and the original touch point P60. When the interval distance L is less than the distance threshold ΔL, the new touch point P' can be used as the first point of the new pen stroke model, and the position of the pen stroke model can be updated in the manner provided in the above embodiment. When the interval distance L is greater than or equal to the distance threshold ΔL, the display device 200 can insert an interpolation point P" between the touch point P' and the touch point P60, and can use the interpolation point P" as the first point of the new pen stroke model.

[0176] After updating the position of the new stroke model using the interpolation point as the first point of the new stroke model, the display device 200 can draw the stroke graphic according to the updated stroke model position. After completing the stroke effect based on the stroke model drawn based on the interpolation point, the display device 200 can also use the new touch point (i.e., P') as the stroke model to update the stroke model to accommodate subsequent touch points.

[0177] In some embodiments, when an interpolation point is inserted between a new touch point and the original starting point of the pen tip model, the position of the interpolation point can be determined based on the spacing distance between the new touch point and the original starting point of the pen tip model. For example, when the spacing distance L is greater than the distance threshold ΔL, and the spacing distance L is less than or equal to twice the distance threshold (i.e., 2ΔL), the interpolation point can be set at the midpoint between the new touch point and the original starting point of the pen tip model. When the spacing distance L is greater than twice the distance threshold (i.e., 2ΔL), the interpolation point can be set at 1 / 3 of the spacing distance between the new touch point and the original starting point of the pen tip model.

[0178] In other embodiments, when interpolating an interpolation point between the new touch point and the original starting point of the stroke model, the position of the interpolation point may be determined based on a distance threshold. For example, when the separation distance L is greater than the distance threshold ΔL, the display device 200 may insert the interpolation point at a position that is the distance threshold ΔL away from the original starting point P60.

[0179] Since the whiteboard application requires the display device to display the pen stroke effect in real time, in order to obtain a better response speed, the display device 200 can use a regional refresh method during the writing process to display the pen stroke effect in the application interface.

[0180] In some embodiments, touch data is acquired in response to the touch interaction instruction. A touch track can be drawn on an acceleration layer based on the touch data, wherein the acceleration layer can be a display layer that performs regional refresh based on the coverage area of ​​the pen stroke model. Upon detecting a lift event in the touch track, the touch track drawn in the acceleration layer can be copied to an OSD layer, wherein the display level of the OSD layer is lower than that of the acceleration layer. Finally, the touch track and pen stroke effect drawn in the acceleration layer are cleared.

[0181] After receiving the touch interaction command input by the user, the display device can obtain touch data from the touch interaction module and call the acceleration layer. Based on the acceleration layer, the pen stroke graphics can be drawn in real time according to the pen stroke model. The acceleration layer can be set to use a regional refresh method, such as refreshing the area where the entire pen stroke model is located, thereby improving writing speed. The user can clearly see the blurring effect of the pen stroke moving forward, improving the user experience.

[0182] In some embodiments, for the above-mentioned line drawing with end effect, in order to ensure the display effect, during the drawing process, the display device needs to detect the newly added track points in the touch action according to the preset sampling frame rate, and continuously update the end effect of the line drawing based on the position of the newly added track points. For example, as shown in Figure 21, taking the line drawing with an "arrow" decorative head added at the end as an example, during the drawing process, the display device 200 detects through the touch interaction module that the track points of two adjacent sampling frames are points P1 and P2, that is, the touch action input by the user moves from point P1 to point P2. In order to display the line drawing effect with end effect, the display device 200 needs to erase the line drawing between the starting point P0 and the track point P1, redraw the line drawing between the starting point P0 and the track point P2, and refresh the redrawn line drawing on the display 260. As a result, the line drawing along the user's touch trajectory always has the "arrow" decorative head.

[0183] Similarly, for different line types, in order to ensure the display effect of the line, it is also necessary to continuously erase and redraw the drawn path line according to the changes in the trajectory points during the drawing process. For example, taking the drawing of a dotted line as an example, in order to maintain the consistency of the intervals between the dotted line points in the dotted line, during the drawing process, the display device 200 detects new trajectory points in the touch action input by the user at a preset sampling frame rate. When a new trajectory point is detected, the drawn dotted line is erased and the dotted line is redrawn according to the starting point and the new trajectory point. Finally, the redrawn dotted line is refreshed and displayed on the display 260.

[0184] Obviously, the above-mentioned trajectory line drawing method continuously erases and redraws the drawn line, resulting in a reduction in drawing speed. According to another trajectory line drawing method for a display device in some embodiments of the present application, as shown in Figure 22, the display device includes at least one processor that can be further configured to execute computer instructions stored in the memory to cause the display device to perform some or all of the following method steps, so as to achieve the use of a segmented method to separate the drawing parts that need to be erased and redrawn from the drawing parts that do not need to be erased and redrawn, thereby improving the drawing speed:

[0185] S2201: In response to a touch interaction instruction input by a user, detecting a newly added track point in a touch track.

[0186] As shown in Figure 22, when running a whiteboard application, display device 200 can respond to touch interaction commands input by the user via the touch interaction module by using a segmented approach to separate the drawn portions that need to be erased and redrawn from the drawn portions that do not need to be erased and redrawn. That is, as the user inputs touch interaction commands, display device 200 can detect newly added track points at a preset sampling frame rate.

[0187] S2202: When a new track point is detected, a historical endpoint graphic is erased. The historical endpoint graphic may be an endpoint graphic associated with a historical track point. The historical track point is a track point of a previous sampling frame.

[0188] S2203: drawing a path track based on the newly added track points and the historical track points, and adding an endpoint graphic at the end of the path track, where the path track is a line track between the historical track points and the newly added track points.

[0189] S2204: Delineate a refresh area, and refresh the refresh area in the user interface. The refresh area includes an area where the historical endpoint graphics are located, an area where the path track is located, and an area where the endpoint graphics at the end of the path track are located.

[0190] After adding an endpoint graphic at the end of the path track, the erased and redrawn content can be refreshed to the user interface presented on the display, thereby completing a line drawing update.

[0191] For example, as shown in FIG22 , as the user controls the touch point to move on the touch interaction module of the display device 200, the touch interaction module can collect trajectory points according to a preset sampling frame rate and transmit the collected trajectory point data to at least one processor 250 of the display device 200. Taking the arrow line drawing shown in FIG23 as an example, for a line drawing with an arrowhead at the end, as the user's touch point changes, the portion that does not need to be redrawn is the line portion Path1 between the starting point P0 and the trajectory point P1. The portions that need to be erased and redrawn are the arrow portion Path2 corresponding to the trajectory point P1, the arrow portion Path4 corresponding to the trajectory point P2, and the path portion Path3 between the trajectory point P1 and the trajectory point P2.

[0192] When the display device 200 detects the newly added track point P2, it can erase the historical endpoint graphic associated with the historical track point P1, namely, the arrow portion Path2. Then, it can draw the path track Path3 between track points P1 and P2, and add an endpoint graphic, namely, the arrow portion Path4, at the end of the path track Path3.

[0193] Similarly, when a new track point P3 is detected, the above process can be followed: the arrow portion of track point P2 is erased, the path between track points P2 and P3 is drawn, and the arrow portion is added at the end of the path. This allows the end of the line to be continuously updated with the position of the new track point.

[0194] For line drawing with end effects (endpoint graphics), only the position of the endpoint graphics is updated as the user inputs the touch trajectory. Therefore, in order to improve the drawing speed, for the drawing of endpoint graphics, since the endpoint graphics change direction according to the user's touch trajectory, the endpoints of the endpoint graphics can be calculated based on the touch trajectory and the trigonometric function relationship, and then the endpoints can be connected according to the types of different endpoint graphics to form corresponding endpoint graphics. In some embodiments, the display device can obtain the line drawing type set by the user before inputting the touch interaction instruction, and the endpoints of the endpoint graphics can be calculated according to the line drawing type, path trajectory and historical trajectory points, and the endpoints can be connected according to the line drawing type to complete the drawing of the endpoint graphics.

[0195] For example, as shown in FIG24 , taking the line drawing of the arrowhead decoration shown in FIG21 as an example, it is assumed that the endpoint graphics to be drawn are two straight lines with a side length of L and an angle of 30° with the path trajectory. Based on the touch trajectory and the trigonometric function relationship, the display device 200 can calculate the coordinates of the three endpoints (A, B, and C) of the endpoint graphic (arrow). Based on the line drawing type of the endpoint graphic, endpoint A can be connected to endpoint B, and endpoint C can be connected to endpoint B, thereby forming an arrow.

[0196] Similarly, as shown in FIG25 , for the line drawing of the T-shaped and X-shaped endpoint graphics shown in FIG12 , the T-shaped endpoint graphic to be drawn is a straight line with a side length of L and a 90° angle with the path trajectory. Based on the touch trajectory and the trigonometric function relationship, the display device 200 can calculate the coordinates of the endpoints (A, B, C) of the endpoint graphic. Based on the line drawing type of the endpoint graphic, the endpoints can be connected to form a T-shaped endpoint graphic.

[0197] As shown in FIG26 , for drawing the X-shaped endpoint graphic shown in FIG12 , the X-shaped endpoint graphic to be drawn is two straight lines with a side length of L and an angle of 45° with the path trajectory. Based on the touch trajectory and the trigonometric function relationship, the display device 200 can calculate the coordinates of the endpoints (A, B, C, D, and E) of the endpoint graphic. Based on the endpoint graphic line drawing type, the endpoints can be connected to form an X-shaped endpoint graphic.

[0198] Taking the arrow line drawing shown in FIG. 23 as an example, when drawing the arrow portion Path4, the display device 200 can directly search for the endpoint graphic of the arrow type in the endpoint graphic library and paste it to the end of the path track Path3.

[0199] In some embodiments, in order to achieve a real-time drawing effect, the display device can draw a touch track in real time on an extended display layer (acceleration layer). That is, in response to a down event generated by a touch interaction instruction input by the user, the extended display layer can be called to draw the touch track in real time in the extended display layer. In other words, the above-mentioned drawing operations of erasing historical endpoint graphics, drawing path tracks, and adding endpoint graphics at the end of the path track can all be drawn and refreshed in real time on the extended display layer. When an up event generated by a touch interaction instruction input by the user is detected, it indicates that a touch track has been drawn, and the touch track in the extended display layer can be refreshed to the display layer (OSD layer) in response to the up event. This improves the display speed of the graphics.

[0200] Since the entire line drawing process is executed and displayed on the extended display layer, for erasing endpoint graphics, if the extended display layer has no background and is transparent, you can use the redraw method. That is, you can use the brush clear mode (PorterDuff.Mode.CLEAR mode) to redraw the drawn path to achieve the purpose of erasing. If the extended display layer itself has a background, you can use the redraw method mentioned above or the background overlay method.

[0201] The clear mode refers to a blending mode that overlays the source path and the target path, so that the transparency value (alpha value) and color value (RGB value) of the intersection area of ​​the source path and the target path are both preset values. For example, the preset value can be set to 0, making the intersection area transparent. That is, the display device can create a brush, set the brush to clear mode, redraw the historical endpoint graphic on the historical endpoint graphic based on the clear mode, and set the alpha value and RGB value of the intersection area of ​​the redrawn historical endpoint graphic and the historical endpoint graphic to 0, so that the intersection area of ​​the redrawn historical endpoint graphic and the historical endpoint graphic is transparent, thereby achieving an erasing operation.

[0202] When erasing a historical endpoint graphic, the display device can set the drawing color to transparent and redraw the historical endpoint graphic using the transparent color over the historical endpoint graphic. In other words, a completely transparent historical endpoint graphic is drawn in the place of the historical endpoint graphic, so that the transparent historical endpoint graphic covers the original historical endpoint graphic, thus achieving the erasure operation.

[0203] In some embodiments, the above-mentioned step of refreshing the display of the erased and redrawn content can refresh the entire user interface. To increase the drawing speed, a refresh area can be set. In some embodiments, the display device can define a refresh area and refresh the refresh area in the user interface. The refresh area can include the area where the historical endpoint graphics are located, the area where the path trajectory is located, and the area where the endpoint graphics at the end of the path trajectory are located.

[0204] The refresh area can be a closed graphical area constructed based on the historical endpoint graphics, the path trajectory, and the endpoint graphics at the end of the path trajectory. For example, FIG27 is a schematic diagram of a refresh area according to some embodiments of the present application. Taking the arrow line shown in FIG23 as an example, the refresh area Rect can be a rectangular graphical area containing the arrow portion Path2, the path trajectory Path3, and the arrow portion Path4.

[0205] It is understandable that the refresh area may be a regular graphic area, such as a rectangular graphic, a circle, a triangle, etc. The refresh area may also be an irregular graphic area.

[0206] In order to more finely define the refresh area, the display device can separately define the area to be erased and the drawn area. In some embodiments, the display device can define a historical endpoint area, a newly added endpoint area, and a path area. And a union operation can be performed on the historical endpoint area, the newly added endpoint area, and the path area to obtain the refresh area. Among them, the historical endpoint area can be a closed graphic area constructed based on historical trajectory points and historical endpoint graphics, the newly added endpoint area can be a closed graphic area constructed based on the endpoint graphics at the end of the path trajectory, and the path area can be a closed graphic area constructed based on historical trajectory points, newly added trajectory points, and path trajectory.

[0207] For example, as shown in FIG28, a schematic diagram of another refresh area according to some embodiments of the present application is shown. Taking the arrow line shown in FIG23 as an example, after performing the erase and redraw operation, the display device 200 can construct a rectangular graphic area Rect1 based on the track point P1 and the arrow portion Path2, can construct a rectangular graphic area Rect2 based on the arrow portion Path4, and can construct a rectangular graphic area Rect3 based on the track point P1, the track point P2 and the path track Path3. Then, the union area of ​​area Rect1, area Rect2 and area Rect3 can be taken as the refresh area, and the refresh area can be refreshed to the user interface so that the user interface presents an interface effect of continuously updating the end effect along the touch track input by the user.

[0208] In this embodiment, during the drawing process, when the display device detects a newly added trajectory point, it may not erase and redraw the entire drawing line, but may separate the part that needs to be erased and redrawn from the part that does not need to be erased and redrawn. The area where the drawing line trajectory remains unchanged will not be processed, and only the area where the drawing line trajectory changes can be drawn and refreshed, thereby improving the drawing speed.

[0209] The above is for drawing lines with end effects. For drawing lines without end effects, there is no erasing operation involved. Therefore, the display device can also demarcate a new area based on the newly added track points and historical track points, and draw and refresh in the new area. In some embodiments, the display device can detect the newly added track points in the touch track in response to the touch interaction instructions input by the user. When the newly added track points are detected, the historical track points can be obtained, and a closed figure can be constructed based on the current track points and the historical track points to obtain the newly added area. A new line can be drawn in the newly added area, and the new line in the newly added area can be refreshed to the user interface.

[0210] For example, as shown in Figure 29, using a straight line as an example, during the drawing process, display device 200 detects, through the touch interaction module, that the trajectory points of two adjacent sampling frames are points P5 and P6, indicating that the user's touch action moved from point P5 to point P6. Based on trajectory points P5 and P6, display device 200 constructs a rectangular graphical region Rect5. A new line Path6 is drawn within Rect5, extending from trajectory points P5 to P6. This graphical region Rect5 is then refreshed so that the new line Path6 and the original line Path5 form a single line drawn along the touch trajectory input by the user.

[0211] Based on the above-mentioned trajectory line drawing method, dotted lines can also be drawn. However, when drawing dotted lines based on the above-mentioned area segmentation drawing and refreshing method, due to the different input speeds when the user performs the touch action, the intervals between the dotted lines will be inconsistent at different input speeds. For example, as shown in Figure 30, when the input speed when the user performs the touch action is too slow, the intervals between the dotted lines are small, making the drawn dotted line appear to be approximately a solid line. When the input speed when the user performs the touch action is uneven, the intervals between the dotted lines are uneven, thereby presenting the effect of combining dotted lines and solid lines.

[0212] To ensure that the interval between drawn dashed lines remains constant while increasing drawing speed, the display device 200 can employ both global redrawing and regional refresh methods for the lines. Global redrawing erases and redraws the previously drawn line trace, thereby ensuring that the interval between dashed lines remains constant. To increase drawing speed, only the newly added areas currently being drawn can be refreshed.

[0213] FIG31 is a flowchart illustrating the drawing of a dotted line according to some embodiments of the present application. Following a user input of a touch interaction command, the display device 200 may detect new track points in the touch trajectory at a preset sampling frame rate (S3101). Upon detecting a new track point, the historical line drawing may be erased (S3102). The historical line drawing may be a line drawn based on a starting point and historical track points. The historical track points may be track points from the previous sampling frame, and the starting point may be the touch location where a drop event occurred in the touch interaction command.

[0214] When erasing a historical line, all lines drawn from the starting point to the historical trajectory points can be erased. Therefore, erasure can be achieved by clearing the extended display layer. That is, when erasing a historical line, the display device 200 can clear the content on the extended display layer to achieve the purpose of erasing the historical line. Of course, historical lines can also be erased using the same method as for erasing historical endpoint graphics.

[0215] After erasing the historical line drawing, the line drawing indicated by the touch interaction instruction can be redrawn based on the starting point and the newly added track points (S3103), and a refresh area can be defined based on the historical track points and the newly added track points. Finally, the refresh area can be refreshed in the user interface to complete the line drawing update (S3104).

[0216] For example, as shown in FIG32 , a down event generated by a touch interaction instruction input by the user corresponds to a starting point P0 on the entire touch track. With subsequent move events generated by touch interaction instructions input by the user, track points can be formed on the entire touch track. As shown in FIG32 , when the touch interaction instruction input by the user moves from point P7 to point P8, a historical track point P7 and a newly added track point P8 can be formed. To ensure that the dashed line interval drawn between historical track point P7 and newly added track point P8 is evenly connected to the dashed line interval drawn between starting point P0 and historical track point P7, the display device 200 can erase the historical line Path7 between starting point P0 and historical track point P7 and redraw the line Path8 between starting point P0 and newly added track point P8. To increase drawing speed, after redrawing, a rectangular graphic area can be constructed based on historical track point P7 and newly added track point P8 to obtain a refresh area Rect7, and only this refresh area can be refreshed.

[0217] The above trajectory drawing method also applies to dashed lines with end effects. The difference between dashed lines with and without end effects lies in the definition of the refresh area. For dashed lines without end effects, a closed graph can be constructed based on the historical trajectory points and the newly added trajectory points to obtain the refresh area. For dashed lines with end effects, the area to be erased and the area to be drawn can be separately delineated, and the union of these areas is used as the refresh area.

[0218] In some embodiments, the display device can obtain the line drawing type set by the user before inputting a line drawing command. Line drawing types can include endpoint types and non-endpoint types. If the line drawing type is a non-endpoint type, a closed graph can be constructed based on the historical trajectory points and the newly added trajectory points to obtain a refresh area. If the line drawing type is an endpoint type, a historical endpoint area, a newly added endpoint area, and a path area can be delineated, and a union operation can be performed on the historical endpoint area, the newly added endpoint area, and the path area to obtain a refresh area.

[0219] For example, as shown in FIG33 , taking the dotted line with the arrowhead as shown in FIG23 as an example, based on the touch interaction command input by the user, the display device 200 can detect the starting point P0, the historical track point P1, and the newly added track point P2. To ensure that the dotted line interval drawn between the historical track point P1 and the newly added track point P2 is evenly connected to the dotted line interval drawn between the starting point P0 and the historical track point P1, the display device 200 can erase the path track Path1 between the starting point P0 and the historical track point P1, as well as the endpoint graphic Path2 associated with the historical track point P1, and redraw the path track Path1' between the starting point P0 and the newly added track point P2, and add the endpoint graphic Path4 at the end of the path track Path1'.

[0220] To improve drawing speed, as shown in Figure 28, the refresh area demarcation steps described above can be followed. After redrawing, the rectangular graphic area Rect1 can be constructed based on the historical track point P1 and the endpoint graphic Path2. The rectangular graphic area Rect2 can be constructed based on the newly added track point P2 and the endpoint graphic Path4. The rectangular graphic area Rect3 can be constructed based on the historical track point P1 and the newly added track point P2. The union of areas Rect1, Rect2, and Rect3 can then be taken as the refresh area and refreshed to the user interface.

[0221] In some embodiments, the whiteboard application can calculate the predicted point that can be used to draw the pen stroke effect based on the touch point in the touch interaction instruction (also known as the drawing instruction). The prediction algorithm can be at least one of the weighted least squares method and the Kalman filter. For example, the display device 200 can respond to the touch interaction instruction provided with the pen stroke effect path graphic, detect the touch point of the touch interaction instruction, and use the coordinates of the touch point as an input parameter, and then calculate a predicted point through the weighted least squares method or the Kalman filter. This predicted point is the end point of the pen stroke effect. Then, the display device 200 can draw the corresponding pen stroke effect at the end of the trajectory line based on the predicted point and the touch point.

[0222] It is understandable that the two prediction algorithms provided in the above embodiments are only for illustrative purposes and are not intended to limit the above. When drawing a pen stroke effect, the display device may also use one or more other algorithms to calculate the prediction point (i.e., the end point of the pen stroke effect).

[0223] To facilitate subsequent description, in some embodiments of the present application, a path graphic with a pen stroke effect can be expressed as a pen stroke path graphic, and the touch point with the latest touch time in the touch interaction instruction for drawing the pen stroke path graphic can be expressed as the target touch point, that is, the target touch point can be the touch point generated in real time based on the current touch position when the user draws the pen stroke path graphic.

[0224] FIG34 illustrates another trajectory drawing method for a display device according to some embodiments of the present application. As shown in FIG34 , at least one processor of the display device may be further configured to execute computer instructions to cause the display device to perform some or all of the following steps: predictive points and drawing points for drawing a pen stroke effect may be calculated based on a target touch point; and the pen stroke effect of a pen stroke path graphic may be drawn in segments based on the predictive points, drawing points, and target touch point. This eliminates the need for additional processing of the already drawn trajectory line, and allows for a better and more natural connection between trajectory lines without a pen stroke effect and trajectory lines with a pen stroke effect, thereby improving the pen stroke display effect of the pen stroke path graphic:

[0225] S3401: In response to a touch interaction instruction of a pen stroke path graphic, detecting a target touch point of the touch interaction instruction.

[0226] After detecting the touch interaction instruction of the pen stroke path graphic, the display device can respond to the touch interaction instruction and detect the target touch point in the touch interaction instruction. The target touch point can be the touch point with the latest touch time in the touch interaction instruction, that is, the touch point last reported by the current touch interaction module.

[0227] In some embodiments, the whiteboard interface of the display device may further include a target control for setting a stroke effect for a path graphic. When the display device detects a selection instruction for the target control, the target control may be recorded as a first state; when the display device detects a deselection instruction for the target control, the target control may be recorded as a second state. The first state may be used to indicate a stroke state, while the second state may be used to indicate a stroke-free state. The display device may then mark touch interaction instructions received in the first state as touch interaction instructions for a stroke path graphic and invoke the graphics library to draw a trajectory line with a stroke effect.

[0228] In some embodiments, the touch interaction instruction includes at least a starting point and a target touch point. The starting point may be the touch point with the earliest touch time in the touch interaction instruction, that is, the touch point corresponding to the down event. The display device may also receive the touch point of the touch interaction instruction and construct a basic Bezier curve based on the touch point. Then, the area without a brush stroke fill is obtained based on the basic Bezier curve. The area without a brush stroke fill may be the touch area between the starting point and the target touch point in the touch interaction instruction. Then, the color value of the area without a brush stroke fill may be set according to a preset fill color to draw a third trajectory line. The line width of the third trajectory line may be equal to the baseline line width of the brush stroke path graphic. Then, the display is controlled to display the third trajectory line in real time, thereby presenting a trajectory line without a brush stroke effect in the user interface. The trajectory line without a brush stroke effect is the part of the brush stroke path graphic without a brush stroke effect, that is, the third trajectory line.

[0229] For example, as shown in Figure 35, Figure 35 is a schematic diagram of a path graphic of a round-headed line cap according to some embodiments of the present application. Point A is the starting point of the path, and point T is the target touch point of the path graphic. The display device 200 can construct a basic Bezier curve through each touch point of the path to generate a drawing list that may include all pixel points in the area without pen tip filling. Then, according to the drawing list and the color, line width and line cap set by the user through the whiteboard interface, the color value of the pixel point in the area without pen tip filling can be set, thereby completing the drawing of the trajectory line (third trajectory line) without pen tip effect as shown in Figure 35.

[0230] To achieve the pen tip effect, in some embodiments, the display device needs to draw the portion of the trajectory line with the pen tip effect while drawing the third trajectory line. For ease of description, in some embodiments of this application, the trajectory line with the pen tip effect is referred to as the target trajectory line. The display device can calculate the predicted points of the pen tip path graphic using a prediction algorithm, and then draw the target trajectory line using the target touch point and the predicted points.

[0231] Because a pen stroke path graphic can have multiple line caps, the display device may experience unnatural transitions when drawing the target trajectory line through the target touch point and predicted point. For example, as shown in Figure 36, when the pen stroke path graphic has a rounded line cap, the target trajectory line α of the pen stroke effect is drawn directly through the target touch point T and predicted point P. The target trajectory line α cannot cover all the line caps, resulting in unnatural transitions in the pen stroke effect.

[0232] It should be noted that the target trajectory line in Figure 36 is shown as a dotted line and unfilled, but in the actual whiteboard interface, the target trajectory line is filled according to the color value set according to the pen stroke graphic path, that is, it is consistent with the color effect of the third trajectory line without the pen stroke effect (different line width).

[0233] In some embodiments, after detecting the target touch point, the display device can calculate the predicted point and the drawn point based on the target touch point, i.e., execute the subsequent step S3402, thereby dividing the target trajectory line with the pen stroke effect into a first trajectory line and a second trajectory line. The first trajectory line can be used to cover the line cap of the trajectory line, and the second trajectory line can be used to present a specific pen stroke trajectory line shape such as a water drop.

[0234] S3402: Calculating predicted points and drawn points based on the target touch point.

[0235] Among them, the predicted point can be calculated based on the target touch point, and the drawing point can be calculated based on the pen tip path graphic and the first inner circle of the pen tip path graphic, the center of the first inner circle can be the target touch point, and the diameter of the first inner circle can be the baseline width of the pen tip path graphic.

[0236] In some embodiments, the display device can read the first coordinates of the target touch point. A first inner circle can be constructed with the first coordinates as the center and half the reference line width as the radius. The basic Bezier curve can then be read and the intersection of the first inner circle and the basic Bezier curve can be calculated to obtain the drawing point.

[0237] For example, in the pen stroke path graph shown in FIG36 , the base line width of the pen stroke path graph is B, the target touch point is point T, and the predicted point is point P. Display device 200 can draw a third trajectory line according to the basic Bezier curve. When calculating the drawing point, display device 200 reads the coordinates of point T and constructs a first inner circle T' with point T as the center and B / 2 as the radius, as shown in FIG37 . Then, the intersection of first inner circle T' and the basic Bezier curve is calculated to obtain drawing point C.

[0238] In some embodiments, the display device may invoke a prediction algorithm to calculate a third coordinate based on the read first coordinate of the target touch point and the prediction algorithm, and may generate a predicted point based on the third coordinate. The prediction algorithm may be at least one of a weighted least squares method and a Kalman filter.

[0239] In a specific implementation, the display device can detect the touch point with the latest touch time in real time, and then calculate the above-mentioned prediction point through the change trend of each touch point and the prediction algorithm, so as to use the calculated prediction point as the end point of the pen stroke effect, that is, the end point of the second trajectory line.

[0240] S3403: Draw a first trajectory line according to the target touch point and the drawing point, and draw a second trajectory line according to the drawing point and the predicted point.

[0241] After the display device obtains the predicted point and the drawing point based on the target touch point, the target trajectory line with the pen stroke effect can be divided into a first trajectory line and a second trajectory line through the target touch point, the predicted point, and the drawing point. Among them, the first trajectory line can be the trajectory line between the target touch point and the drawing point. The line width of the first trajectory line can be equal to the baseline line width and can be used to cover the line cap of the trajectory line; the second trajectory line can be the trajectory line between the drawing point and the predicted point. The line width of the second trajectory line can be less than the baseline line width and can be used to present a pen stroke trajectory line in the shape of a water drop, a knife shape, a Bezier curve, or a Pascal's triangle. For example, as shown in Figure 38, the target touch point is point T, the predicted point is point P, and the drawing point is point C. Then the trajectory line from point T to point C can be the first trajectory line α1, and the trajectory line from point C to point P can be the second trajectory line α2.

[0242] In some embodiments, when drawing the first trajectory line, the display device may read the second coordinate of the drawing point and then construct a second inner circle with the second coordinate as the center and 1 / 2 the reference line width as the radius. A first circumferential tangent of the first inner circle and a second circumferential tangent of the second inner circle may then be calculated, and the intersection of the first and second circumferential tangents may be calculated to obtain a reference point. A first stroke fill area may then be obtained based on the reference point, and the color value of the first stroke fill area may be set according to a preset fill color to draw the first trajectory line.

[0243] For example, referring to the pen stroke path diagram shown in FIG38 , as shown in FIG39 , circle T' is the first inner circle, and circle C' is the second inner circle. We can then solve for the intersection points P1, P2, P3, and P4 of the circumtangent lines of circles T' and C'. Connecting P1, P2, P3, and P4, respectively, yields rectangle P1P2P3P4. Rectangle P1P2P3P4 represents the display area of ​​first trajectory line α1 in FIG14 .

[0244] In some embodiments, when the display device draws the second trajectory line, it can obtain a second stroke filling area between the drawing point and the predicted point based on the stroke drawing algorithm. Among them, the stroke drawing algorithm can be used to draw a trajectory line with a line width less than the baseline line width, such as that constructed based on the principles of Bezier curves, Pascal's triangles, etc. Then, the color value of the second stroke filling area can be set according to the preset fill color to draw the second trajectory line. For example, as shown in the stroke path graph of Figure 38, the drawing point is C and the predicted point is P. The display device 200 can construct a second-order Bezier curve based on the drawing point C and the predicted point P, and can draw the second trajectory line α2 shown in Figure 38 through the second-order Bezier curve.

[0245] S3404: Control the display to display the first trajectory line and the second trajectory line to present the pen stroke path graphic in the user interface.

[0246] After the display device completes drawing the first trajectory line and the second trajectory line, it can control the display to display the first trajectory line and the second trajectory line, thereby presenting a smooth pen stroke path graphic with a pen stroke effect, which may include the first trajectory line, the second trajectory line and the third trajectory line.

[0247] In some embodiments, the user's movement speed when inputting a touch interaction instruction is different, that is, the average distance between adjacent touch points per unit time is different, and the display device can calculate different distances between the predicted point and the target touch point based on the prediction algorithm. For example, the faster the movement speed when inputting the touch interaction instruction, the greater the distance between the predicted point and the target touch point; otherwise, the distance between the predicted point and the target touch point is smaller. Therefore, when the user's movement speed when inputting the touch interaction instruction is slower, the distance between the predicted point and the target touch point is smaller; accordingly, the distance between the drawing point and the predicted point is also smaller. At this time, when the first drawing line and the second drawing are drawn between the target touch point and the predicted point, as shown in FIG40 , it is easy to have the problem of no pen tip effect or a blunt pen tip effect.

[0248] In some embodiments, the display device may also be configured with a distance threshold Threshold, which may be M times the basic line width of the pen tip path graphic. Among them, M may be greater than 0.5, for example, M may take a value of 1, and Threshold may be equal to the basic line width. Before drawing the first drawing line and the second drawing line, the display device may calculate the actual distance between the predicted point and the target touch point. For ease of description, some embodiments of the present application express the actual distance between the predicted point and the target touch point as a first reference distance. When the first reference distance is less than or equal to the distance threshold, the display device may refresh the first trajectory line and the second trajectory line in reverse according to the distance threshold, so that the pen tip effect can be presented to the greatest extent when the movement speed is slower.

[0249] In some embodiments, after the display device calculates the first reference distance between the predicted point and the target touch point, if the first reference distance is less than or equal to the distance threshold, it indicates that the user's touch movement speed is slow, then the second reference distance between the predicted point and the starting point can be calculated, and the first trajectory line and the second trajectory line can be drawn according to the second reference distance and the distance threshold; if the first reference distance is greater than the distance threshold, it indicates that the user's touch movement speed is fast, then the display device can execute the steps of drawing the first trajectory line according to the target touch point and the drawing point, and drawing the second trajectory line according to the drawing point and the predicted point, i.e., step S3403.

[0250] For the case where the touch movement speed is slow, that is, when the first reference distance is less than or equal to the distance threshold, in some embodiments, the display device can also compare the second reference distance and the distance threshold. When the second reference distance is less than the distance threshold, the second trajectory line can be drawn according to the starting point and the predicted point; when the second reference distance is equal to the distance threshold, the first trajectory line and the second trajectory line can be drawn according to the starting point and the predicted point; when the second reference distance is greater than the distance threshold, the first trajectory line and the second trajectory line can be drawn according to the predicted point and the reverse cutoff point, wherein the distance between the reverse cutoff point and the predicted point can be equal to the distance threshold.

[0251] In this way, the display device can draw the first and second trajectory lines with a pen stroke effect according to the algorithm of steps S3401-S3403 above. The first trajectory line can be used to cover the line cap of the trajectory line, and the second trajectory line can be used to present a pen stroke trajectory line with effects such as a water drop shape or a knife shape. When the user's touch movement speed is slow, the distance between the target touch point and the predicted point will be short, and the first and second trajectory lines will be compressed within a shorter distance, which may easily lead to the problem of no pen stroke effect or a dull pen stroke effect. Therefore, in some embodiments of the present application, a distance threshold may be set, within which the first and second trajectory lines can present a minimum complete pen stroke effect. The minimum complete pen stroke effect can be set according to the specific application scenario and is not limited by this application. Then, when the first reference distance is less than or equal to the distance threshold, the display device can set different first drawing line starting points according to the distance threshold and the second reference distance (the distance between the predicted point and the starting point), so that the pen stroke effect of the pen stroke path graphic can be drawn to the greatest extent.

[0252] When the first reference distance is less than or equal to a distance threshold, a second reference distance between the predicted point and the starting point can be calculated. If the second reference distance is less than the distance threshold, indicating that the user has just begun drawing the pen stroke path graphic, only the portion between the starting point and the predicted point that can be used to present the pen stroke trajectory line, i.e., the portion from thick to thin, i.e., the portion of the second trajectory line, can be drawn. If the second reference distance is equal to the distance threshold, indicating that the distance required for a minimum complete pen stroke effect has been reached, the first and second trajectory lines can be drawn between the starting point and the predicted point, i.e., the minimum complete pen stroke. If the second reference distance is greater than the distance threshold, a reverse cutoff point equal to the distance threshold from the predicted point can be calculated backward from the predicted point, and the first and second trajectory lines can be drawn between the reverse cutoff point and the predicted point, thereby presenting a pen stroke path graphic that may include both trajectory lines without and with a pen stroke effect, forming the first, second, and third trajectory lines. In other words, when the second reference distance is greater than the distance threshold, the display device can draw the first and second trajectory lines for the distance from the predicted point backward from the threshold, refreshing the minimum complete pen stroke in the reverse direction.

[0253] For example, the distance threshold set in the display device 200 is Threshold, the user's touch movement speed is slow, and the first reference distance D1 between the predicted point and the target touch point is always less than or equal to Threshold. As shown in Figure 41, P0 in Figure 41 is the starting point of the drawing event. The display device 200 can calculate the predicted point P1 based on the target touch point at time T1 at time T1, and calculate the distance D2 between P1 and P0. If D2 < Threshold at this time, a trajectory line path1 from thick to thin can be drawn between P1 and P0; then, the display device 200 can calculate the predicted point P2 based on the target touch point at time T2 at time T2, and calculate the reference distance D3 between P2 and P0. If D3 = Threshold at this time, a trajectory line path2 of the minimum complete pen stroke can be drawn between P2 and P0; the display device 200 can calculate the predicted point P3 based on the target touch point at time T3 at time T3, Calculate the reference distance D4 between P3 and P0. If D4>Threshold at this time, the reverse cutoff point C1 can be determined at the distance Threshold backward from P3, the trajectory line path3 of the minimum complete pen stroke can be drawn between P3 and C1, and the trajectory line without pen stroke effect can be drawn from C1 to P0; then, the display device 200 can calculate the predicted point P4 according to the target touch point at time T4, and the reverse cutoff point C2 can be determined at the distance Threshold backward from P4, the trajectory line path4 of the minimum complete pen stroke can be drawn between P4 and C2, and the trajectory line without pen stroke effect can be drawn from C2 to C1.

[0254] In the case of a faster touch movement speed, that is, when the first reference distance is greater than the distance threshold, in some embodiments, the display device may read the first coordinate of the target touch point and then construct a first inner circle with the first coordinate as the center and 1 / 2 the reference line width as the radius. A basic Bezier curve may be read and the intersection of the first inner circle and the basic Bezier curve may be calculated to obtain a drawing point. A first trajectory line may then be drawn based on the target touch point and the drawing point, and a second trajectory line may be drawn based on the drawing point and the predicted point.

[0255] For example, if the distance threshold set in display device 200 is Threshold and the user's touch movement speed is slow, the first reference distance D1 between the predicted point and the target touch point is always greater than Threshold. As shown in FIG42 , P0 is the starting point of the drawing event, T is the target touch point, and P is the predicted point. Display device 200 can then draw path5, which may include the first trajectory line and the second trajectory line, from point T to point P according to steps S3401-S3403 described above.

[0256] It should be noted that regardless of the speed of the touch movement, the drawing algorithm of steps S3401-S3403 is used to draw the first and second trajectories. Different drawing starting points are set only when the comparison results of the first reference distance, the second reference distance, and the distance threshold are different. In other words, the drawing algorithm is the same, but the starting points of the first and second trajectories are different.

[0257] To improve processing efficiency, in some embodiments, the display device may render the trajectory of the stroke path graphic using regional rendering. That is, the display device may only render the region where the trajectory changes, thereby improving the processing speed of the display device.

[0258] In some embodiments, a display device may read a rectangular area for rendering a pen stroke path graphic. The rectangular area may include at least one of a first rectangular area, a second rectangular area, and a third rectangular area. The first rectangular area may be used to render a first trajectory line, the second rectangular area may be used to render a second trajectory line, and the third rectangular area may be used to render a third trajectory line. The pen stroke path graphic may then be rendered based on the rectangular area, so that the display displays the pen stroke path graphic in a user interface.

[0259] As shown in FIG43 , during the drawing process, when drawing from point T1 to point T2, it is necessary to clear the trajectory line with the pen stroke effect corresponding to the previous touch point (the first trajectory line and the second trajectory line). The display device 200 can then clear Rect1, the union of the first rectangular area and the second rectangular area at point T1. Then, the trajectory line with the pen stroke effect corresponding to point P6 (the first trajectory line and the second trajectory line corresponding to point T2) and the newly added trajectory line without the pen stroke effect (the third trajectory line added from T1 to T2) can be drawn, and the corresponding areas are Rect2 and Rcet3, respectively. Therefore, when drawing from point T1 to point T2, the display device 200 can perform regional rendering on the union of Rect1, Rect2, and Rect3.

[0260] In some embodiments, when the display device clears the trajectory lines corresponding to the first and second rectangular areas of the previous touch point, it can set the bitmap data of the area to a transparent bitmap. This can avoid overlapping and other uneven display effects in the final rendered stroke path graphic, thereby improving the user experience.

[0261] Finally, it should be noted that the above embodiments are only used to illustrate the solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding solutions from the scope of the embodiments of the present application.

Claims

1. A display device, comprising: a display configured to display a user interface; The display is connected to a touch interaction module, and the touch interaction module is used to obtain a touch interaction instruction input by a user; a memory configured to store computer instructions and / or data associated with the display device; At least one processor is configured to execute the computer instructions to cause the display device to perform: Responding to a touch interaction instruction input by a user, acquiring a touch area and a touch track; Generate a pen tip model according to the touch area, the pen tip model is a dot matrix model generated based on touch points, contour points and spare points in the touch area; the touch point is at least one contact point in the touch area, the contour point is at least one contact point in the touch area located at the edge of the pen tip; the spare point divides the distance between the touch point and the contour point into preset equal parts; According to the pen tip model, a pen tip trajectory line is drawn at the end of the touch trajectory.

2. The display device according to claim 1, wherein the at least one processor generates a pen stroke model according to the touch area, and is further configured to execute the computer instructions to enable the display device to execute: Extracting a center line of the touch area, wherein the center line of the touch area is parallel to a direction line of the touch track; Extracting a preset number of touch points in the touch area based on the center line of the touch area, wherein the preset number is set according to the shape of the pen tip model; Taking the touch point as a reference, generating contour points according to the line width of the touch track; The spare point is added between the touch point and the contour point according to the shape of the pen stroke model.

3. The display device according to claim 2, wherein the at least one processor is configured to extract a preset number of touch points in the touch area based on the center line of the touch area, and is further configured to execute the computer instructions to enable the display device to execute: Get the application ID that calls the whiteboard application; Query the current usage scenario according to the application ID; If the current usage scenario is the first scenario, extracting a first number of touch points in the touch area; If the current usage scenario is the second scenario, a second number of touch points is extracted from the touch area, and the second number is smaller than the first number.

4. The display device according to claim 2, wherein the touch area is a touch track formed during a touch sliding process, and the at least one processor is further configured to execute the computer instruction to enable the display device to execute: Extracting a preset number of contact areas from the touch track according to a pen tip refresh cycle, wherein the pen tip refresh cycle is greater than or equal to a sampling cycle of the touch interaction module; Delineating a direction line of the touch track according to the line width of the touch track; The touch points are extracted from the contact area in sequence along the direction line of the touch track, wherein the touch points are contact points in the contact area that are located on the direction line.

5. The display device according to claim 2, wherein the touch area is a contact area formed by a single touch process, and the at least one processor is further configured to execute the computer instructions to enable the display device to execute: Traversing the contact point coordinates in the contact area to obtain edge contact points; Delineating a center line of the contact area according to the edge contact point, wherein the center line of the contact area is parallel to a direction line of the touch track; According to the center line of the contact area, a preset number of contact points are extracted from the contact area to serve as the touch points.

6. The display device according to claim 1, wherein the at least one processor executes drawing a pen tip trajectory line at the end of the touch trajectory according to the pen tip model, and is further configured to execute the computer instructions to enable the display device to execute: Extracting quasi-contour points in the pen tip model, wherein the quasi-contour points are touch points, contour points and spare points located at the edge of the pen tip shape; Calculate the midpoint between adjacent quasi-contour points; A Bezier curve is constructed with the midpoint and / or the quasi-contour point as the starting and ending points to generate a stroke graphic.

7. The display device according to claim 6, wherein the at least one processor executes drawing a pen tip trajectory line at the end of the touch trajectory according to the pen tip model, and is further configured to execute the computer instructions to enable the display device to execute: Acquire a new touch point in the touch track; generating contour points and spare points associated with the new touch point based on the new touch point; The new touch point, the contour point and the spare point are used to replace the quasi-contour point to update the position of the pen stroke model.

8. The display device according to claim 7, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Acquire the interval distance between the new touch point and the original starting point of the pen tip model, where the original starting point is a quasi-contour point at the edge of the pen tip model along the touch track direction; If the interval distance is greater than the distance threshold, adding an interpolation point between the new touch point and the original first point of the pen stroke model, and generating contour points and spare points associated with the interpolation point based on the interpolation point; If the interval distance is less than or equal to the distance threshold, the step of generating contour points and spare points associated with the new touch point based on the new touch point is performed.

9. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to the touch interaction instruction, acquiring touch data; Draw a touch track on an acceleration layer according to the touch data, where the acceleration layer is a display layer that performs regional refresh based on the coverage area of ​​the pen stroke model; Upon detecting a lift event in the touch track, copying the touch track drawn in the acceleration layer to an OSD layer, wherein the display level of the OSD layer is lower than that of the acceleration layer; Clear the touch track drawn in the acceleration layer.

10. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a touch interaction instruction input by a user, detecting a newly added track point in a touch track; When the newly added trajectory point is detected, a historical endpoint graphic is erased, where the historical endpoint graphic is an endpoint graphic associated with the historical trajectory point, where the historical trajectory point is a trajectory point in a previous sampling frame; Drawing a path track based on the newly added track point and the historical track point, and adding an endpoint graphic at the end of the path track, the path track being a line drawing track between the historical track point and the newly added track point; A refresh area is defined, and the refresh area is refreshed on the user interface, wherein the refresh area includes an area where the historical endpoint graphics are located, an area where the path track is located, and an area where the endpoint graphics at the end of the path track are located.

11. The display device according to claim 10, wherein the at least one processor performs calculation refresh area, and is further configured to execute the computer instructions to cause the display device to perform: Delineating a historical endpoint area, a newly added endpoint area and a path area, wherein the historical endpoint area is obtained according to the historical track points and the historical endpoint graph, the newly added endpoint area is obtained according to the endpoint graph at the end of the path track, and the path area is obtained according to the historical track points, the newly added track points and the path track; A union operation is performed on the historical endpoint area, the newly added endpoint area, and the path area to obtain the refresh area.

12. The display device according to claim 11, wherein the at least one processor is further configured to execute the computer instructions to enable the display device to execute: Constructing a closed graph according to the historical trajectory points and the historical endpoint graph to obtain the historical endpoint region; Constructing a closed graph according to the endpoint graph at the end of the path trajectory to obtain the newly added endpoint area; A closed graph is constructed according to the historical trajectory points, the newly added trajectory points and the path trajectory to obtain the path area.

13. The display device according to claim 10, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a drop event generated by a touch interaction instruction input by a user, calling an extended display layer; Drawing a touch track in real time in the extended display layer; In response to a lift-up event generated by a touch interaction instruction input by a user, the touch track in the extended display layer is refreshed to the display layer.

14. The display device according to claim 13, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to execute: Redrawing the historical endpoint graphic on the historical endpoint graphic; The transparency value and color value of the intersection area of ​​the redrawn historical endpoint graphic and the historical endpoint graphic are set to preset values.

15. The display device according to claim 10, wherein the at least one processor performs adding an endpoint graphic at the end of the path trajectory, and is further configured to execute the computer instructions to cause the display device to perform: Obtaining the line drawing type set by the user before inputting the touch interaction instruction; Calculate the endpoints of the endpoint graph according to the line drawing type, the path trajectory and the historical trajectory points; The endpoints are connected according to the line drawing type to form the endpoint graphic.

16. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a touch interaction instruction input by a user, detecting a newly added track point in a touch track; When a new trajectory point is detected, a historical trajectory point is obtained, where the historical trajectory point is a trajectory point of a previous sampling frame; Constructing a closed graph according to the current trajectory point and the historical trajectory point to obtain a newly added area; Draw a new line in the newly added area; The new line in the newly added area is refreshed to the user interface.

17. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a touch interaction instruction input by a user, detecting a newly added track point in a touch track; When a new track point is detected, a historical drawing line is erased, where the historical drawing line is a drawing line drawn according to a starting point and a historical track point, where the historical track point is a track point of a previous sampling frame, and the starting point is a touch position where a drop event is generated in the touch interaction instruction; Redrawing the drawing line indicated by the touch interaction instruction according to the starting point and the newly added track point; Delimiting a refresh area according to the historical track points and the newly added track points; The refresh area is refreshed in the user interface.

18. The display device according to claim 17, wherein the at least one processor is configured to calculate a refresh area according to the historical track points and the newly added track points, and is further configured to execute the computer instructions to enable the display device to execute: Acquire the line drawing type set by the user before inputting the line drawing instruction, wherein the line drawing type includes an endpoint type and a non-endpoint type; If the line drawing type is a non-endpoint type, a closed graph is constructed according to the historical track points and the newly added track points to obtain the refresh area; If the line drawing type is an endpoint type, a historical endpoint area, a newly added endpoint area and a path area are delineated, and a union operation is performed on the historical endpoint area, the newly added endpoint area and the path area to obtain the refresh area, wherein the historical endpoint area is a closed graphic area constructed according to the historical trajectory points and the endpoint graphics associated with the historical trajectory points, the newly added endpoint area is a closed graphic area constructed according to the newly added trajectory points and the endpoint graphics associated with the newly added trajectory points, and the path area is a closed graphic area constructed according to the historical trajectory points and the newly added trajectory points. Close the graphics area.

19. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a touch interaction instruction of a stroke path graphic, detecting a target touch point of the touch interaction instruction, wherein the target touch point is a touch point with the latest touch time in the touch interaction instruction; Calculate predicted points and drawn points based on the target touch point; The predicted point is calculated based on the target touch point, and the drawing point is calculated based on the pen tip path graph and a first inner circle of the pen tip path graph, the center of the first inner circle is the target touch point, and the diameter of the first inner circle is the reference line width of the pen tip path graph; Draw a first trajectory line according to the target touch point and the drawing point, and draw a second trajectory line according to the drawing point and the predicted point; the line width of the first trajectory line is equal to the reference line width, and the line width of the second trajectory line is less than the reference line width; The display is controlled to display the first trajectory line and the second trajectory line, so as to present the pen stroke path graphic in the user interface.

20. The display device according to claim 19, wherein the touch interaction instruction at least comprises a starting point and a target touch point, the starting point being the touch point with the earliest touch time in the touch interaction instruction; and the at least one processor is further configured to execute the computer instruction to enable the display device to execute: receiving a touch point of the touch interaction instruction; Constructing a basic Bezier curve according to the touch points; Acquire a non-stroke filling area according to the basic Bezier curve, wherein the non-stroke filling area is a touch area between the starting point and the target touch point in the touch interaction instruction; Setting the color value of the non-pen-tip filling area according to a preset filling color to draw a third track line; The line width of the third track line is equal to the reference line width; The display is controlled to display the third trajectory line.

21. The display device according to claim 20, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to execute: Reading the first coordinate of the target touch point; Constructing the first inner circle with the first coordinate as the center and 1 / 2 of the reference line width as the radius; Reading the basic Bezier curve; The intersection point between the first inner circle and the basic Bezier curve is calculated to obtain the drawing point.

22. The display device according to claim 19, wherein the at least one processor executes drawing a first trajectory line according to the target touch point and the drawing point, and is further configured to execute the computer instruction to enable the display device to execute: Reading the second coordinate of the drawing point; Construct a second inner circle with the second coordinate as the center and 1 / 2 of the reference line width as the radius; Calculating a first external tangent line of the first inner circle and a second external tangent line of the second inner circle; Calculating an intersection point of the first external tangent line and the second external tangent line to obtain a reference point; Acquire a first pen tip filling area according to the reference point; The color value of the first pen tip filling area is set according to a preset filling color to draw the first trajectory line.

23. The display device according to claim 19, wherein the at least one processor executes drawing a second trajectory line according to the drawing point and the predicted point, and is further configured to execute the computer instruction to enable the display device to execute: Based on a pen tip drawing algorithm, obtaining a second pen tip filling area between the drawing point and the prediction point, the pen tip drawing algorithm being used to draw a trajectory line having a line width smaller than the reference line width; The color value of the second pen tip filling area is set according to a preset filling color to draw the second trajectory line.

24. The display device according to claim 19, wherein the touch interaction instruction at least comprises a starting point and a target touch point, and the starting point is the touch point with the earliest touch time in the touch interaction instruction; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: Calculating a first reference distance between the predicted point and the target touch point; If the first reference distance is less than or equal to a distance threshold, calculating a second reference distance between the prediction point and the starting point, and drawing the first trajectory line and the second trajectory line according to the second reference distance and the distance threshold; If the first reference distance is greater than the distance threshold, the steps of drawing a first trajectory line according to the target touch point and the drawing point and drawing a second trajectory line according to the drawing point and the predicted point are performed.

25. The display device according to claim 24, if the first reference distance is less than or equal to a distance threshold, the at least one processor is further configured to execute the computer instructions to cause the display device to execute: When the second reference distance is less than the distance threshold, drawing the second trajectory line according to the starting point and the predicted point; When the second reference distance is equal to the distance threshold, drawing the first trajectory line and the second trajectory line according to the starting point and the predicted point; When the second reference distance is greater than the distance threshold, the first trajectory line and the second trajectory line are drawn according to the predicted point and the reverse cutoff point, and the distance between the reverse cutoff point and the predicted point is equal to the distance threshold.

26. The display device of claim 20, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Reading a rectangular area for rendering the stroke path graphic, the rectangular area comprising at least one of a first rectangular area, a second rectangular area, and a third rectangular area, the first rectangular area being used to render the first trajectory line, the second rectangular area being used to render the second trajectory line, and the third rectangular area being used to render the third trajectory line; The pen stroke path graphic is rendered based on the rectangular area, so that the display displays the pen stroke path graphic in the user interface.

27. The display device of claim 19, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Calling a prediction algorithm, wherein the prediction algorithm includes at least one of a weighted least squares method and a Kalman filter; Reading the first coordinate of the target touch point; A third coordinate is calculated based on the first coordinate and the prediction algorithm, and the prediction point is generated according to the third coordinate.

28. A method for drawing a trajectory line for a display device, comprising: In response to a touch interaction instruction input by a user, acquiring a touch area generated by the touch interaction instruction in real time; Generate a pen tip model according to the touch area, the pen tip model is a dot matrix model generated based on touch points, contour points and spare points in the touch area; the touch point is at least one contact point in the touch area, the contour point is at least one contact point in the touch area located at the edge of the pen tip; the spare point divides the distance between the touch point and the contour point into preset equal parts; According to the pen tip model, a pen tip trajectory line is drawn at the end of the touch trajectory.

Images