Display device and auxiliary line drawing method for touch track

Abstract

The embodiment of the invention provides a display device and an auxiliary line drawing method for a touch track, and the method can respond to a touch event input by a user, detect a starting point of the touch track, and calculate a first distance between the starting point and an auxiliary edge. And querying the target auxiliary edge with the first distance smaller than or equal to the distance threshold according to the first distance. If the number of the target auxiliary edges is larger than 1, the target auxiliary edge with the shortest first distance is marked as a drawing edge, and a track line of the touch track is drawn according to the drawing edge. And detecting track points of the auxiliary edges, calculating second distances between the track points and the target auxiliary edges, and marking the target auxiliary edge with the shortest second distance as a drawing edge. According to the method, when a plurality of target auxiliary edges meeting drawing conditions exist, the trajectory is drawn according to the target auxiliary edge closest to the starting point. In the drawing process, the auxiliary edge for drawing the trajectory is converted according to the distance between the trajectory point and the target drawing edge, so that the problem of trajectory drawing errors is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a touch track auxiliary line drawing method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, and can be terminal devices such as smart TVs, communication terminals, smart advertising screens, and projectors. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. Display devices can be equipped with image interfaces, which can be used to connect monitors to form a multi-screen display mode.

[0003] In order to meet the needs of scenarios such as meetings and education, for display devices that support touch interactive operations, you can run a whiteboard application and implement writing or drawing functions based on the touch area of ​​the whiteboard application. The whiteboard application can respond to the touch track input by the user in real time and display the track line of the touch track. In order to ensure the standardization of the writing process, the whiteboard application of the display device can also be configured with an auxiliary line drawing view, which can include multiple auxiliary edges. When displaying the touch track, the display device can correct the touch track according to the position of the auxiliary edge, that is, the display device can display the touch track as a track line consistent with the auxiliary edge. When there are multiple auxiliary edges, the display device needs to calculate the distance between the hand landing position and each auxiliary edge, and then draw the track line of the touch track according to the auxiliary edge closest to the hand landing position.

[0004] However, since the distance between some auxiliary edges is relatively close, when the user inputs the touch track at the intersection of two auxiliary edges, it is difficult to accurately control the position of the hand to be closer to the desired auxiliary edge. If the user's hand position is closer to the undesired auxiliary edge, the display device will draw the track line of the touch track according to the undesired auxiliary edge, resulting in the problem of drawing errors of the track line of the touch track. Summary of the invention

[0005] The present application provides a display device and an auxiliary line drawing method for a touch track to solve the problem of track line drawing errors in a display device.

[0006] In a first aspect, the present application provides a display device, comprising a display and a controller. The display is configured to display a whiteboard application interface, the whiteboard application interface includes an auxiliary line drawing view, and the auxiliary line drawing view includes at least two auxiliary edges; the display is connected to a touch interaction module, and the touch interaction module is used to obtain a touch track input by a user, and the touch track includes at least one touch point; the controller is configured to execute the following program steps:

[0007] In response to a touch event input by a user, detecting a starting point of the touch track, wherein the starting point is a touch point of a drop event;

[0008] Calculating a first distance between the starting point and the auxiliary edge;

[0009] Querying a target auxiliary edge according to the first distance, the target auxiliary edge being an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to a preset distance threshold;

[0010] If the number of the target auxiliary edges is greater than 1, marking the target auxiliary edge with the shortest first distance as a drawing edge, and drawing a track line of the touch track according to the drawing edge;

[0011] Detecting a track point of the auxiliary edge, where the track point is a touch point of a moving event;

[0012] Calculating a second distance between the trajectory point and the target auxiliary edge;

[0013] The target auxiliary edge with the shortest second distance is marked as the drawing edge, so as to draw the trajectory line according to the drawing edge.

[0014] In a second aspect, some embodiments of the present application further provide a display device, including a display and a controller. The display is configured to display a whiteboard application interface, the whiteboard application interface includes an auxiliary line drawing view, the auxiliary line drawing view includes at least two auxiliary edges, the display is connected to a touch interaction module, the touch interaction module is used to obtain a touch track input by a user, the touch track includes at least one touch point; the controller is configured to execute the following program steps:

[0015] In response to a touch event input by a user, detecting a track point of the touch track;

[0016] Detecting a third distance between the trajectory point and the auxiliary edge;

[0017] When there is an adjacent auxiliary edge, calculating the change angle of the target trajectory point, and obtaining the standard angle between the adjacent auxiliary edge and the drawing edge, the adjacent auxiliary edge is an auxiliary edge whose third distance is less than the third distance of the drawing edge, the drawing edge is an auxiliary edge of a trajectory line for drawing a touch trajectory, the target trajectory point is a trajectory point with the latest touch time in the touch trajectory, and the change angle is obtained based on a trajectory foot point of the target trajectory point, and the trajectory foot point is a foot point of a perpendicular produced by a line connecting the trajectory point to the auxiliary edge;

[0018] If the changed angle is equal to the standard angle, the adjacent auxiliary edge is marked as a drawing edge, so as to draw the track line of the touch track according to the drawing edge.

[0019] In a third aspect, the present application further provides a touch track auxiliary line drawing method, which is applied to the display device described in the first aspect, and the method includes:

[0020] In response to a touch event input by a user, detecting a starting point of a touch track, wherein the starting point is a touch point of a drop event;

[0021] Calculate a first distance between the starting point and the auxiliary edge;

[0022] Querying a target auxiliary edge according to the first distance, the target auxiliary edge being an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to a preset distance threshold;

[0023] If the number of the target auxiliary edges is greater than 1, marking the target auxiliary edge with the shortest first distance as a drawing edge, and drawing a track line of the touch track according to the drawing edge;

[0024] Detecting a track point of the auxiliary edge, where the track point is a touch point of a moving event;

[0025] Calculating a second distance between the trajectory point and the target auxiliary edge;

[0026] The target auxiliary edge with the shortest second distance is marked as the drawing edge, so as to draw the trajectory line according to the drawing edge.

[0027] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and an auxiliary line drawing method for a touch track. The method can detect the starting point of the touch track in response to a touch event input by a user, and calculate the first distance between the starting point and the auxiliary edge. Then query the target auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to the preset distance threshold according to the first distance. If the number of target auxiliary edges is greater than 1, mark the target auxiliary edge with the shortest first distance as a drawing edge, and draw the track line of the touch track according to the drawing edge. Detect the track point of the auxiliary edge, and calculate the second distance between the track point and the target auxiliary edge. Then mark the target auxiliary edge with the shortest second distance as a drawing edge. The method can first draw the track line according to the target auxiliary edge closest to the starting point when there are multiple target auxiliary edges that meet the drawing conditions. During the drawing process, the auxiliary edge of the drawing track line is converted according to the distance between the track point and the target drawing edge, thereby improving the problem of track line drawing errors in the display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0030] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0031] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0032] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0033] Figure 5 A schematic diagram of touch tracks provided in some embodiments of the present application;

[0034] Figure 6 A schematic diagram of a whiteboard application interface provided in some embodiments of the present application;

[0035] Figure 7 A schematic diagram of a whiteboard application display system provided in some embodiments of the present application;

[0036] Figure 8 A schematic diagram of an acceleration layer provided for some embodiments of the present application;

[0037] Fig. 9 A schematic diagram of the effect of the auxiliary line drawing view provided in some embodiments of the present application;

[0038] Fig.10 A schematic diagram of the distance between the starting point and the auxiliary edge provided in some embodiments of the present application;

[0039] Fig.11 An example diagram of trajectory line drawing errors provided in some embodiments of the present application;

[0040] Fig.12 A schematic diagram of a flow chart of a touch track assisted line drawing method performed by a display device provided in some embodiments of the present application;

[0041] Fig.13 A schematic diagram of a flow chart of a touch track assisted line drawing method provided in some embodiments of the present application;

[0042] Fig.14 A schematic diagram of the perpendicular points of multiple target auxiliary edges provided in some embodiments of the present application;

[0043] Fig.15 A schematic diagram of a process of rendering trajectory lines provided in some embodiments of the present application;

[0044] Fig.16 A schematic diagram of the adsorption effect of the trajectory line provided in some embodiments of the present application;

[0045] Fig.17 A schematic diagram of a process for setting a calibration mark provided in some embodiments of the present application;

[0046] Fig.18 A schematic diagram of an option list for providing an auxiliary line drawing view for some embodiments of the present application;

[0047] Fig.19 An example diagram of the bending of the track line provided in some embodiments of the present application;

[0048] Fig. 20 A schematic diagram of a flow chart for calculating a change angle provided in some embodiments of the present application;

[0049] Fig.21 A schematic diagram of the foot point of the angle provided in some embodiments of the present application;

[0050] Fig. 22 A schematic diagram of the effect of drawing edges according to changing the angle conversion provided in some embodiments of the present application;

[0051] Fig.23 A schematic diagram of a process of drawing an edge according to a change in angle conversion provided in some embodiments of the present application;

[0052] Fig.24 A schematic diagram of the principle of generating a perpendicular foot point provided in some embodiments of the present application;

[0053] Fig.25 An example diagram of an early turning trajectory provided for some embodiments of the present application. DETAILED DESCRIPTION

[0054] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0055] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0056] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0057] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0058] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0059] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 may be a remote controller, a stylus pen, or the like.

[0060] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0061] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0062] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0063] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0064] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0065] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0066] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0067] In some embodiments, the display 260 is connected to a touch interaction module so that the display device 200 supports touch interaction operations. The touch interaction module can be a built-in module or an external module. For example, in order to implement touch interaction operations, for a display device 200 with a built-in touch interaction module, the display 260 can be a touch screen, which can receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy.

[0068] For the display device 200 with an external touch interaction module, a terminal, device, or component with a touch function can be connected through the device interface of the display device 200. For example, the USB or USB-C interface of the display device 200 is connected to a touch panel, and the touch panel can receive the touch track input by the user in real time and transmit the touch track to the display device 200 through the USB channel. The display device 200 can perform an interactive response based on the received touch track in combination with the touch application, so that the display device 200 supports touch interaction operations.

[0069] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0070] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0071] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0072] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0073] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0074] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0075] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0076] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0077] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0078] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0079] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0080] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0081] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0082] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0083] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0084] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0085] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0086] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0087] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0088] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0089] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0090] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0091] The display device 200 can also deploy other device interfaces, hardware, and drivers based on the kernel layer according to the specific display requirements of the display device 200. For example, in order to enrich the display types of the display device 200, an HDMI interface and a USB interface can be set on the display device 200, wherein the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver. The USB interface can connect devices with signal and data transmission functions and interactive functions, and based on the USB driver, access signals, transmit data, and perform specific interactive actions.

[0092] For a display device 200 that supports touch interaction operations, the display device 200 has a built-in touch interaction module, and can receive the touch track input by the user through the touch interaction module. According to the interaction principle of the touch interaction module, the touch interaction module can detect relevant parameters in the touch action. For example, a capacitive touch layer is provided on the display 260 of the display device 200. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touch position will change. At this time, the touch interaction module can record the position of the capacitance change, and record the time when the capacitance change occurs in combination with the timer in the controller to generate a touch track.

[0093] In some embodiments, the physical signal detected by the touch interaction module needs to be converted before it can be read by the controller 250 of the display device 200. For example, the analog signal of the capacitance change needs to be converted into a digital signal and then transmitted to the controller 250 of the display device 200. In order to adapt to the screen display process of the display device 200, the touch track detected by the touch interaction module also needs to be mapped to coordinates so that the position point corresponding to the touch track can have a corresponding relationship with the display 260 of the display device 200 or the pixel point in the user interface. For example, the coordinate position detection accuracy of the touch interaction module is 65536×65536, and the resolution of the display 260 of the display device 200 is 3840×2160. After the touch interaction module detects the touch track, the coordinate mapping of the touch point position in the width and height directions can be performed according to the ratio of 65536 / 3840 and 65536 / 2160, so that the display device 200 can perform related operation responses based on the mapped touch point position coordinates.

[0094] The touch track may include different operation parameters according to the user's input method. Figure 5 As shown, the touch trajectory may include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. In a touch interaction process, the touch event types that can be detected by the touch interaction module include: down events, move events, and up events of touch actions. Correspondingly, on the entire touch trajectory, a starting point [down; x1, y1, t1], a trajectory point [move; x2, y2, t2], and an end point [up; x3, y3, t3] may be formed. Each touch trajectory records the corresponding touch event type, touch time t, and touch position (x, y), that is, the touch points of the touch trajectory may include a starting point, a trajectory point, and an end point.

[0095] It should be noted that, since the user's finger or touch device is in surface contact with the touch interaction module when performing touch interaction operations, that is, the touch interaction module can detect a contact area, and the contact area includes multiple contact points. Therefore, when the display device 200 performs interactive control, it can detect (or specify) a contact point in the contact area as a touch point. For example, when the contact area is an elliptical area formed when the user's finger contacts the touch interaction module, the display device 200 can traverse the contact point coordinates in the elliptical area, determine the coordinate extremes in two mutually perpendicular directions, to delineate the major axis and minor axis of the elliptical area, and use the contact point at the intersection of the major axis and the minor axis as the touch point.

[0096] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, the number of touch points and the type of touch events, so as to detect the user's touch input gesture, thereby performing different interactive responses for different input gestures. For example, when the number of touch points in the touch trajectory detected by the touch interaction module is 1, the touch event type only includes a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold. In addition, if the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 executes the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0097] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch time, touch position, number of touch points and touch event type, such as double-click gesture, slide gesture, long press gesture, multi-finger click gesture, multi-finger slide gesture, etc. Different interaction gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, slide track shape, etc. Different touch parameters can be used to perform different interactive controls.

[0098] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection based on the dynamic change rules of touch time, touch position, number of touch points and touch event type. For example, when the touch interaction module detects that the touch trajectory of the user input includes multiple touch points, and the touch position of each touch point approaches each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grab gesture. Based on complex gesture detection, the display device 200 can perform more interactive responses. For example, when the detected touch gesture is a grab operation for an image view, the display device 200 can respond to the grab gesture and perform a control action to select and move the image display position.

[0099] In some embodiments, the touch gesture can also be combined with the touch position to form a touch interaction operation based on a specific position. For example, when the touch interaction module detects that the down event position during the touch operation is at the top of the display 260, the touch point position corresponding to the move event moves downward. When the up event is detected, the display device 200 can determine that the current touch gesture is to slide down from the top of the screen. Therefore, the display device 200 can respond to the touch gesture and control the display 260 to present a drop-down list.

[0100] Some touch interaction gestures can be used to perform interactive actions based on the operating system of the display device 200, which are called global gestures. For example, the operation of sliding down from the top of the screen to call out a drop-down list in the above example of the display device 200. Some touch interaction gestures can be used to perform interactive actions based on specific touch applications. For example, when the display device 200 runs a whiteboard application, the sliding touch operation input by the user can be used to draw a track line on the whiteboard.

[0101] In some embodiments, the display device 200 can generate bitmap data of the touch track in response to the touch track input by the user based on the touch application. When the end point of the touch track is detected, the bitmap data of the touch track is refreshed to the OSD layer of the display device 200, so that the display device 200 can present the display effect of the touch track.

[0102] The touch application can be a system application or a third-party application developed based on a specific display device type and user needs. The touch application can be installed in the display device 200 and started to run under user control. For example, the user can control the display device 200 to run the whiteboard application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0103] It should be noted that the whiteboard application can be an independent application, an associated application, or a functional module in a touch application. Different forms of whiteboard applications can have different startup or calling methods. For example, a user can start an independent whiteboard application through the application interface of the display device 200; a user can also start a whiteboard application by executing a "whiteboard" operation on a conference application; a user can also control the conference application to start the whiteboard functional module by clicking the "whiteboard" icon on the conference application.

[0104] Different forms of whiteboard applications may present different forms of whiteboard application interfaces after startup. For example, when a user starts a whiteboard application in an associated application form, the display device 200 may automatically enter a dual-screen (or split-screen) mode. In the dual-screen (or split-screen) mode, the display device 200 may display a user interface through two display areas, i.e., the main screen displays the original application (such as a conference application) interface, and the secondary screen displays the whiteboard application interface.

[0105] After the whiteboard application is running, the display device 200 can be switched to display a whiteboard interface. Figure 6 As shown, the whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, graphics, line shape, color, erase, clear screen, selection, etc. The user can click any control to draw graphics according to specific drawing needs.

[0106] In some embodiments, the display device 200 classifies the touch track input by the user based on the whiteboard application according to the selected state of the drawing tool control. For example, when the drawing tool control such as pen shape, graphic shape, line shape, etc. is in the selected state, the touch track input by the user is marked as a drawing event; when the erasing drawing tool is in the selected state, the touch track input by the user is marked as an erasing event. The display device 200 then responds to the touch track of the drawing event or the erasing event and executes different programs according to the preset interaction strategy.

[0107] like Figure 7As shown, the display device 200 can render and display the drawing picture in real time through an image generation (Image Producer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module and a display (screen).

[0108] In the process of displaying the drawing picture, the graphic interface (openGLES) of the image generation module can be connected to the whiteboard application to generate the drawing image content and pass the image content to the image rendering module. The image rendering module is used to control the display 260 to display the content, that is, to render the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the picture frames can be stored in the synthesis module through the frame buffer (framebuffer) and the video buffer (video buffer), respectively, and the picture content is formed in the corresponding layer through the video process module. That is, the image frames generated by the video playback process form the picture content through the video layer (video plane), and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the primary OSD layer (primary OSD plane). Finally, the display device 200 forms the final picture through the blending unit (blend) of the video process module and displays it on the display 260.

[0109] Since the whiteboard application will present the user's touch trajectory in real time according to the user's touch operation, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. To this end, in some embodiments, the display device 200 can present a real-time painting effect by setting an acceleration layer. That is, in the video process module of the display device 200, an additional extended display layer (external OSD plane) can be set on the basis of the main OSD layer and the video layer. Among them, the extended display layer can be configured with different layer characteristics according to the specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0110] Taking the display device 200 running a whiteboard application as an example, in order to improve the real-time response speed of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). Figure 8 As shown, the display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the lines drawn by the acceleration layer are updated to other layers, thereby reducing input delay and improving the real-time display effect of the whiteboard application drawing process.

[0111] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, that is, when the display device 200 detects the end point of the lift event, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawn graphics, thereby eliminating the need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0112] In some embodiments, in order to meet different user needs, the whiteboard application can also dynamically configure different views to implement different functions through different views. Fig. 9 As shown, in the whiteboard application run by the display device 200 applied to the field of education, a configurable auxiliary line drawing view can be included, and the teaching tool option can be started based on the "teaching tool" option. When the user clicks the teaching tool option, a teaching tool list can pop up. The teaching tool list can include tool options of types such as "ruler", "triangle", and "protractor". After the user clicks the "triangle" tool option, the auxiliary line drawing view of the "triangle" can be called for drawing. At this time, the display device 200 can add a movable triangle figure in the whiteboard drawing interface, and when the touch point of the touch track is close to the auxiliary edge of the triangle figure, the track line of the touch track is drawn along the auxiliary edge of the triangle figure, so that the track line presented by the touch track is more standardized.

[0113] Since the auxiliary line drawing view may include multiple auxiliary edges, after the user starts the auxiliary line drawing view, the display device 200 also needs to detect the starting point of the touch track to determine the auxiliary edge on which the drawing operation needs to be performed. Therefore, in some embodiments, the display device 200 detects the starting point of the touch track and calculates the distance between the starting point and the adjacent edge of each auxiliary edge in the auxiliary line drawing view. If there is an auxiliary edge in the auxiliary line drawing view whose adjacent edge distance is less than or equal to the preset distance threshold, the track line of the touch track is drawn according to the auxiliary edge.

[0114] For example, Fig.10 As shown, the preset distance threshold is M, and the user starts the auxiliary line drawing view of the set square in the display device 200. The set square includes auxiliary sides AB, AC and BC. Auxiliary sides AB and AC have a common intersection point P. When the user places his hand at point P and inputs a touch track to the display device 200, the display device 200 detects the initial point m1 of the touch track, and calculates the distance d1 between the initial point m1 and the auxiliary side AB, and the distance d2 between the initial point m1 and the auxiliary side AC. When M>d1>d2, the display device 200 draws the track line of the touch track according to the auxiliary side AB.

[0115] However, since the distance between some auxiliary edges is relatively close, when the user inputs a touch track near the intersection of two auxiliary edges, it is difficult to accurately control the position of the hand to be closer to the desired auxiliary edge. When the starting point of the touch track is closer to the undesired auxiliary edge, the display device 200 will draw the track line of the touch track according to the undesired auxiliary edge, resulting in the problem of drawing errors of the track line of the touch track. For example, Fig.11 As shown, the user wants to draw a track line through the auxiliary side AC of the set square, and puts his hand at the common intersection of the auxiliary side AB and AC to input the touch track to the display device 200. However, since the user's hand position is closer to the auxiliary side AB, that is, the distance between the starting point of the touch track and the auxiliary side AB is closer, the display device 200 draws the track line of the touch track according to the auxiliary side AB, resulting in the user being unable to draw the desired track line through the auxiliary side, which reduces the user experience.

[0116] Based on the above application scenarios, in order to improve the problem of trajectory line drawing errors, some embodiments of the present application provide a display device 200, such as Fig.12 As shown, it includes a display 260 and a controller 250. The display 260 is configured to display a whiteboard application interface, the whiteboard application interface includes an auxiliary line drawing view, and the auxiliary line drawing view includes at least two auxiliary edges; and the display 260 is also connected to a touch interaction module, and the touch interaction module is used to obtain a touch track input by a user, and the touch track includes at least one touch point.

[0117] like Fig.13 As shown, the controller is configured to perform the following program steps:

[0118] S110: In response to a touch event input by a user, detecting a starting point of a touch track.

[0119] After the user inputs a touch track to the display device 200, the display device 200 will detect the starting point of the touch track in response to the touch event of the touch track. The starting point is the touch point of the down event. The display device 200 can determine the distance between the starting point and each drawn edge by detecting the position coordinates of the starting point. For example, the starting point of the touch point can be stored in the display device 200 in the format of [down; x1, y1, t1]. When the display device 200 reads the down flag, it can be determined that the position coordinates of the starting point are (x1, y1).

[0120] S120: Calculate the first distance between the starting point and the auxiliary edge.

[0121] After the display device 200 detects the starting point, it calculates the first distance between the starting point and each auxiliary edge in the auxiliary line drawing view. When displaying the auxiliary line drawing view, the display device 200 will display the auxiliary line drawing view according to the preset layout file. The display device 200 can determine the real-time position of each auxiliary edge in the whiteboard application interface through the layout file. The display device 200 can calculate the distance between the starting point and the auxiliary edge based on the coordinates of the starting point and the real-time position of the auxiliary edge.

[0122] Since the auxiliary line drawing view includes multiple types of views, in order to facilitate the user to select the auxiliary line drawing view, in some embodiments, the display device 200 also controls the display 260 to display an option list of the auxiliary line drawing view in the whiteboard application interface. Among them, the option list includes a subview control. The display device 200 responds to the selection instruction of the subview control and obtains the layout file of the subview control. Among them, the layout file may include the view width and height, the view position and the attribute information of the auxiliary edge. The view position may include the coordinates of the upper left corner of the auxiliary line drawing view, etc., and the attribute information of the auxiliary edge may include the coordinates of the boundary point, length, etc. Then, the auxiliary line drawing view is rendered in the whiteboard application interface according to the layout file.

[0123] For example, Fig.18 As shown in FIG. 1 , the option list of the auxiliary line drawing view may include sub-view controls such as a set square, a protractor, and a ruler. When the user clicks the sub-view control of the set square, the display device 200 obtains the layout file of the set square and displays the following according to the layout file of the set square: Fig. 9 The whiteboard application interface shown.

[0124] In some embodiments, the layout file of the auxiliary line drawing view includes dynamic attributes. After the display device 200 starts the auxiliary line drawing view, if an event such as size change is performed on the auxiliary line drawing view, the display device 200 refreshes the dynamic attributes of the auxiliary line drawing view in response to the size change event. For example, when the user performs operations such as scaling, rotating, and moving the auxiliary line drawing view, the display device 200 refreshes the dynamic attributes of the auxiliary line drawing view.

[0125] Therefore, in some embodiments, when the display device 200 calculates the first distance between the starting point and the auxiliary edge, it also obtains the layout file of the auxiliary line drawing view. The layout file includes the attribute information of the auxiliary edge. After obtaining the layout file, the initial perpendicular foot point is generated according to the attribute information. The initial perpendicular foot point is the perpendicular foot point generated by the line connecting the starting point to the auxiliary edge. The distance between the initial perpendicular foot point and the starting point is calculated to obtain the first distance between the starting point and the auxiliary edge.

[0126] In some embodiments, limited by the size of the auxiliary line drawing view, when the display device 200 connects the auxiliary side, it generates an initial perpendicular foot point perpendicular to the auxiliary side from the starting point to the auxiliary side, and detects the boundary points of the auxiliary side. The boundary points of the auxiliary side include a first boundary point and a second boundary point. The first limiting distance between the first boundary point and the initial perpendicular foot point is calculated, and the second limiting distance between the second boundary point is calculated. If the first limiting distance or the second limiting distance is greater than the distance between the first boundary point and the second boundary point, the display device 200 marks the boundary point closest to the initial point as the initial perpendicular foot point of the starting point. That is, when the initial point exceeds the boundary range of the auxiliary side, the display device 200 limits the boundary point closest to the initial point to the perpendicular foot point of the starting point. .

[0127] It is understandable that when any touch point in the touch track exceeds the range of the auxiliary edge, the same logic as above is applied. Fig.24 As shown, the touch point p is outside the boundary range of the auxiliary side AB, and points A and B are the boundary points of the auxiliary side AB. When the display device 200 generates the foot point of the perpendicular point p, it first makes the foot point foot perpendicular to AB, and then calculates the distance between foot and point A, and the distance between foot and point B. If the display device 200 calculates that the distance between foot and point B is greater than the distance between point A and point B, then point A, which is closest to point p, is limited as the foot point of the perpendicular point P.

[0128] For example, the starting point is a1, such as Fig.14 When the display device 200 calculates the first distance between a1 and each auxiliary side of the set square, the initial perpendicular foot points of a1 on the auxiliary side AB and the auxiliary side AC are foot1 and foot1 respectively. ’ By calculating the distance between a1 and foot1, we can get the first distance between the starting point and the auxiliary side AB; by calculating the distance between a1 and foot1 ’ The distance between the starting point and the auxiliary side AC can be obtained.

[0129] S130: Query the auxiliary edge of the target according to the first distance.

[0130] When the starting point of the touch track input by the user is far away from the auxiliary line drawing view, it means that the user may not need to draw the track line through the target auxiliary edge at present. Therefore, after the display device 200 calculates the first distance between the starting point and each auxiliary edge, the display device 200 also determines the auxiliary edge that meets the drawing conditions in the auxiliary line drawing view according to the first distance. Among them, whether each auxiliary edge meets the drawing conditions can be determined by presetting a specific distance threshold. In some embodiments, when the first distance of the drawn edge in the auxiliary line drawing view is less than or equal to the distance threshold, it means that the auxiliary edge meets the drawing conditions and is marked as a target auxiliary edge; when the first distance of the drawn edge in the auxiliary line drawing view is greater than the distance threshold, it means that the auxiliary edge does not meet the drawing conditions and is not marked as a target auxiliary edge, that is, the target auxiliary edge is an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to the preset distance threshold.

[0131] S140: If the number of target auxiliary edges is greater than 1, mark the target auxiliary edge with the shortest first distance as a drawing edge, and draw a track line of the touch track according to the drawing edge.

[0132] After querying the target auxiliary edges that meet the drawing conditions, the display device 200 can also detect the number of target auxiliary edges. When the number of target auxiliary edges is greater than 1, it means that there are multiple auxiliary edges that meet the drawing conditions. At this time, the display device 200 first marks the target auxiliary edge closest to the starting point as the drawing edge, that is, marks the target auxiliary edge with the shortest first distance as the drawing edge, and then draws the track line of the touch track according to the drawing edge.

[0133] In some embodiments, Fig.15 As shown, when the display device 200 draws the track line of the touch track according to the drawing edge, it obtains the target initial perpendicular point from the starting point to the drawing edge. Then the track perpendicular point is generated according to the attribute information of the drawing edge. Among them, the track point is the touch point of the moving event, and the track perpendicular point is the perpendicular point generated by the track point to the drawing edge. Taking the target initial perpendicular point as the starting point, a track line is generated according to the track perpendicular point. Then the track line is rendered to the whiteboard application interface to present the display effect of the track line on the display 260. The display device 200 can convert the position of each touch point in the touch track into the coordinates of each perpendicular point by means of the perpendicular point, generate the track line of the touch track through each perpendicular point, and present the track line in the whiteboard drawing interface.

[0134] Since the auxiliary line drawing view is a two-dimensional view displayed in the whiteboard application interface, it is greatly different from three-dimensional auxiliary tools such as a set square and a protractor. When a user draws a line using a three-dimensional auxiliary tool, the three-dimensional characteristics of the auxiliary tool can be used to fix the handwriting of the line drawing. Therefore, the display device 200 provided in the embodiment of the present application determines the auxiliary edge that the user wants to use through the starting point of the touch track, and draws the track line of the touch track according to the auxiliary edge, thereby improving the jitter problem that may exist in the touch track and achieving the adsorption effect of the auxiliary edge. For example, Fig.16 As shown, when the display device 200 draws a track line by drawing an edge, if the user's finger shakes, the touch track can still be displayed in a standardized manner at the position where the auxiliary edge is located.

[0135] It should be noted that the process of generating perpendicular points from the trajectory point to the drawn edge line is the same as the process of generating perpendicular points from the starting point to each auxiliary edge line, which will not be described in detail here.

[0136] In order to meet the diversified display effects of the touch track, in some embodiments, the display device 200 also reads the touch color information selected by the user when inputting the touch track when generating the track line. The touch color information may include color values ​​of four channels of R (red), G (green), B (blue), and A (transparency). When generating the track line, the display device 200 may set it to a corresponding color value according to the touch color information.

[0137] S150: Detecting track points of the touch track.

[0138] When the number of target auxiliary edges is greater than 1, there are multiple auxiliary edges that meet the drawing conditions. In order to further determine the accuracy of the drawing edge, the display device 200 can also detect the track points of the touch track, and then determine whether the current drawing edge is the correct drawing edge according to the trend of the track points. For example, the track points in the touch point can be stored in the display device 200 in the format of [move; x1, y1, t1]. When the display device 200 reads the move flag, it can be determined that the position coordinates of the starting point are (x1, y1).

[0139] When the number of target auxiliary edges is equal to 1, there is only one auxiliary edge that meets the drawing condition, and there may be no problem of drawing edge errors. In order to reduce the judgment logic of the display device 200 in drawing the trajectory line and reduce the occupation of system resources, Fig.17As shown, in some embodiments, the display device 200 also detects the number of target auxiliary edges. When the number of target auxiliary edges is greater than 1, the calibration identifier is set to the first identifier, and the step of calculating the second distance between the trajectory point and the target auxiliary edge is performed, that is, step S600 is performed. When the number of target auxiliary edges is equal to 1, the calibration identifier is set to the second identifier, and the step of calculating the second distance between the trajectory point and the target auxiliary edge is no longer performed, that is, there is no need to calibrate the drawing edge. Among them, the first identifier is used to characterize the state of the drawing edge to be calibrated, and the second identifier is used to characterize the state of the drawing edge not to be calibrated. For example, when the number of target auxiliary edges is 1, the calibration identifier is set to flase; when the number of target auxiliary edges is greater than 1, the calibration identifier is set to true.

[0140] Furthermore, in some embodiments, when the number of target auxiliary edges is greater than 1, after the display device 200 marks the target auxiliary edge with the shortest first distance as a drawing edge, a timer is created to start timing, and / or the drawing length of the trajectory line is detected. After a preset time period or when the drawing length is greater than the target length, the display device 200 sets the mark to be calibrated to the second mark, and no longer calibrates the drawing edge. That is to say, after the display device 200 draws the trajectory line according to the target auxiliary edge with the shortest first distance for a period of time, and / or after the display device 200 draws the trajectory line according to the target auxiliary edge with the shortest first distance for a certain distance, there is no need to calibrate the drawing edge, then the mark to be calibrated is set to the second mark, and no longer calibrates the drawing edge.

[0141] In some embodiments, when the number of target auxiliary edges is equal to 0, there is no auxiliary edge that meets the drawing conditions in the auxiliary line drawing view, and the display device 200 directly generates a track line of the touch track according to the touch point of the touch track, and controls the display 260 to display the track line.

[0142] S160: Calculate the second distance between the trajectory point and the target auxiliary edge.

[0143] Since there are multiple target auxiliary edges in the display device 200, the display device 200 also needs to calculate the second distance between the trajectory point and the target auxiliary edge to determine the trend of the touch trajectory through the second distance, and determine whether the current drawing edge is the drawing edge expected by the user through the trend of the touch trajectory.

[0144] In some embodiments, when the display device 200 calculates the second distance between the track point and the target auxiliary edge, it obtains the layout file of the auxiliary line drawing view. After obtaining the layout file, it generates a track perpendicular point according to the attribute information of the target auxiliary edge, and calculates the distance between the track perpendicular point and the track point to obtain the second distance between the track point and the target auxiliary edge.

[0145] S170: Mark the target auxiliary edge with the shortest second distance as the drawing edge, so as to draw the trajectory line according to the drawing edge.

[0146] After calculating the second distance between the trajectory point and the target auxiliary edge, the display device 200 marks the target auxiliary edge with the shortest second distance as the drawing edge, thereby calibrating the drawing edge of the drawing trajectory line. Among them, the target auxiliary edge is an auxiliary edge whose first distance is less than or equal to the distance threshold, that is, the target auxiliary edge is an auxiliary edge that meets the drawing conditions. When it is determined through the starting point that there are multiple target auxiliary edges that meet the drawing conditions, the display device 200 determines the subsequent trend of the touch trajectory according to the distance between the trajectory point and the target auxiliary edge. If the trajectory point is closer to another target auxiliary edge other than the current drawing edge, it means that the drawing edge marked in step S400 is incorrect, and a new drawing edge is re-marked.

[0147] For example, Fig.14 As shown, the starting point is a1, the trajectory point is a2, and the distance threshold is M. The initial perpendicular points of a1 on the auxiliary side AB and the auxiliary side AC are foot1 and foot1 respectively. ’ . The distance d1 between a1 and foot1 is calculated. ’ When M>d1>d2, the trajectory line is drawn according to the drawn side AC. The foot points of the trajectory point a2 on the auxiliary side AB and the auxiliary side AC are foot2 and foot2 respectively. ’ , calculate the distance d3 between a2 and foot2, ’ When M>d4>d3, the trajectory line is drawn according to the drawing edge AB. In this way, when the number of target auxiliary edges is greater than 1, the display device 200 can convert the drawing edge of the auxiliary line drawing view according to the distance between the trajectory point and the target auxiliary edge, thereby improving the drawing accuracy of the trajectory line.

[0148] In addition, since the auxiliary line drawing view may have multiple intersecting auxiliary edges, the user may also input a turning touch track when inputting a touch track, so that the display device 200 displays the turning track line. In some embodiments, the display device 200 further detects a third distance between the track point and the auxiliary edge. If the third distance is less than the conversion distance threshold, the auxiliary edge with the shortest third distance is marked as a drawing edge, so as to draw the track line according to the drawing edge.

[0149] However, due to the two-dimensional characteristics of the auxiliary line drawing view, it is difficult for the user to stabilize the finger or stylus or other devices on a straight line when inputting the touch track. Fig.19 As shown, when the display device 200 draws the track line through the auxiliary side AB, if the finger slides to the position of point N, the display device 200 will suddenly switch to the auxiliary side AC to draw the track line, and the track line displayed by the display device 200 will appear Fig.19 Alternatively, in some embodiments, when the display device 200 detects that the distance between a track point and a certain auxiliary edge is smaller than the distance between the track point and the drawing edge, the drawing track line is converted to the auxiliary edge with a closer distance. Fig.25 As shown, there is still a certain distance between the trajectory point where the user's hand falls and the auxiliary edge AC. At this time, the user may not want to draw a trajectory line through the auxiliary edge AC, causing the display device 200 to turn the trajectory line prematurely, reducing the user's interactive experience.

[0150] In order to improve the bending and early turning problems of the above-mentioned trajectory line, some embodiments of the present application further provide a display device 200, including a display 260 and a controller 250. The display 260 is configured to display a whiteboard application interface, the whiteboard application interface includes an auxiliary line drawing view, and the auxiliary line drawing view includes at least two auxiliary edges; the display 260 is also connected to a touch interaction module, and the touch interaction module is used to obtain a touch trajectory input by a user, and the touch trajectory includes at least one touch point.

[0151] like Fig.23 As shown, the controller 250 is configured to perform the following program steps:

[0152] S210: In response to a touch event input by a user, detecting a track point of a touch track.

[0153] After the user inputs the touch track to the display device 200, the display device 200 will detect the track points of the touch track in response to the touch event of the touch track. For example, the track points in the touch track can be stored in the display device 200 in the format of [move; x1, y1, t1]. When the display device 200 reads the move flag, it can determine that the position coordinates of the starting point are (x1, y1).

[0154] S220: Detect the third distance between the trajectory point and the auxiliary edge.

[0155] The display device 200 detects the distances between all auxiliary edges and the track point in the auxiliary line drawing view, that is, the third distance between the track point and each auxiliary edge, so as to determine the distance relationship between the touch track and each auxiliary edge through the third distance. The third distance between the track point and the auxiliary edge can be calculated in the same way as the first distance between the starting point and the auxiliary edge. For example, the track point generates a perpendicular foot point to each auxiliary edge connection line, and the third distance between the track point and the auxiliary edge is obtained by calculating the distance between the track point and the perpendicular foot point.

[0156] S230: When there is an adjacent auxiliary edge, calculate the change angle of the target trajectory point, and obtain the standard angle between the adjacent auxiliary edge and the drawing edge.

[0157] The display device 200 calculates the third distance between the trajectory point and each auxiliary edge in real time, and determines the adjacent auxiliary edge closest to the trajectory point through the third distance. The adjacent auxiliary edge is an auxiliary edge whose third distance is less than the third distance of the drawing edge, the drawing edge is an auxiliary edge of the trajectory line for drawing the touch trajectory, the target trajectory point is the trajectory point with the latest touch event in the touch trajectory, and the change angle is obtained based on the trajectory foot point of the target trajectory point, and the trajectory foot point is the foot point of the perpendicular produced by the line connecting the trajectory point to the auxiliary edge. The standard angle is the angle between two auxiliary edges in the auxiliary line drawing view, which can be obtained based on the layout file of the auxiliary line drawing view. For example, Fig.19 In the set square shown, the standard angle between the auxiliary side AB and the auxiliary side AC is 90°.

[0158] S240: If the changed angle is equal to the standard angle, the adjacent auxiliary edge is marked as a drawing edge, so as to draw the track line of the touch track according to the drawing edge.

[0159] After determining that the changed angle is consistent with the standard angle, the display device 200 converts the drawing edge, marks the adjacent auxiliary edge as the drawing edge, and draws the trajectory line of the touch trajectory through the drawing edge, thereby displaying the auxiliary edge with a standard angle, improving the problem of trajectory line bending or premature turning.

[0160] like Fig. 20 As shown, in some embodiments, the display device 200 queries the reference point of the target trajectory point when calculating the change angle of the trajectory point. The reference point is the touch point whose touch time has the smallest touch time difference with the target trajectory point. The foot point of the angle perpendicular to the target trajectory point is obtained. The foot point of the angle perpendicular includes the first foot point of the target trajectory point to the adjacent auxiliary edge connection line, the second foot point of the reference point to the drawn edge connection line, and the initial foot point of the touch trajectory from the starting point to the drawn edge connection line, and the starting point is the touch point of the lift event. Then, an angle vector is generated according to the foot point of the angle perpendicular, and the angle vector includes a first vector and a second vector, the first vector is generated based on the first foot point and the second foot point, and the second vector is generated based on the second foot point and the initial foot point. The angle between the first vector and the second vector is calculated to obtain the change angle of the trajectory point.

[0161] For example, Fig.21 As shown in , the starting point is a1, the trajectory points are a2, a3, a4, a5, the distance threshold is M, and the standard angle between the auxiliary side AB and the auxiliary side AC is 90 degrees. Fig.21 , the initial foot points of a1 on the auxiliary sides AB and AC are foot1 and foot1 respectively ’ The foot points of the trajectory point a2 on the auxiliary side AB and the auxiliary side AC are foot2 and foot2 respectively. ’, the display device 200 is drawing a track line according to the auxiliary side AB, and the track point a3 is at the foot points of the auxiliary side AB and the auxiliary side AC, which are foot3 and foot3 respectively. ’ The distance d5 between a3 and foot3 is calculated. ’ The distance d6. At this time, d6<d5, calculate the angle ∠foot3 ’ , foot2, foot1; ∠foot3 ’ , foot2, foot1≠90 degrees. Similarly, when the track point a5 is detected, the angle angle ∠foot5 ’ , foot4, foot1 = 90 degrees, switch the drawing edge, and draw the trajectory line according to the auxiliary edge AC, such as Fig. 22 Therefore, the display device 200 provided in the embodiment of the present application can determine whether to convert the drawing line of the auxiliary line drawing view by the angle between the two intersecting auxiliary edges, and can display a trajectory line with a standardized angle when converting the drawing edge, thereby improving the user's interactive experience.

[0162] Furthermore, in order to enable the display device 200 to display the trajectory line in real time, in some embodiments, the whiteboard application interface is displayed through the graphics layer, and the graphics layer is the OSD layer described in the above embodiments. When the display device 200 draws the trajectory line according to the drawing edge, it also calls the acceleration layer. Among them, the acceleration layer is a display layer that refreshes the display content according to the unit area, and the display level of the acceleration layer is higher than the display level of the graphics layer when the user inputs the touch trajectory. The trajectory line is refreshed to the acceleration layer and the graphics layer, and the display 260 is controlled to display the superimposed display screen of the acceleration layer and the graphics layer. The display level of the acceleration layer is higher than the graphics layer, and the trajectory line is refreshed to the acceleration layer according to the unit area, so that the acceleration layer can display the trajectory line of the touch trajectory in real time, achieving a hand-following effect.

[0163] In some embodiments, when the display device 200 refreshes the track line to the acceleration layer and the graphics layer, it also responds to the detection of an operation event of a newly added touch point in the touch track and refreshes the track line to the acceleration layer. When the end point of the touch track is detected, the track line is refreshed to the graphics layer, and the track line of the acceleration layer is released. Among them, the end point is the touch point of the lift event in the touch track, that is, after the display device 200 detects the hand-lifting event, the entire track line of the touch track is refreshed to the graphics layer, and at the same time, whether to accelerate the track line displayed in the layer to reduce the amount of data in the acceleration layer and reduce resource usage.

[0164] Based on the above display device 200, some embodiments of the present application also provide a touch track auxiliary line drawing method, which is applied to the display device 200 provided in the above embodiment, such as Fig.13 As shown, the method comprises the following procedural steps:

[0165] S110: In response to a touch event input by a user, detecting a starting point of a touch track, where the starting point is a touch point of a drop event;

[0166] S120: Calculate a first distance between the starting point and the auxiliary edge;

[0167] S130: querying a target auxiliary edge according to the first distance, where the target auxiliary edge is an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to a preset distance threshold;

[0168] S140: If the number of the target auxiliary edges is greater than 1, mark the target auxiliary edge with the shortest first distance as a drawing edge, and draw the track line of the touch track according to the drawing edge;

[0169] S150: Detecting a track point of the auxiliary edge, where the track point is a touch point of a moving event;

[0170] S160: Calculating a second distance between the trajectory point and the target auxiliary edge;

[0171] S170: Mark the target auxiliary edge with the shortest second distance as the drawing edge, so as to draw the trajectory line according to the drawing edge.

[0172] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and an auxiliary line drawing method for a touch track. The method can detect the starting point of the touch track in response to a touch event input by a user, and calculate the first distance between the starting point and the auxiliary edge. Then query the target auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to the preset distance threshold according to the first distance. If the number of target auxiliary edges is greater than 1, mark the target auxiliary edge with the shortest first distance as a drawing edge, and draw the track line of the touch track according to the drawing edge. Detect the track point of the auxiliary edge, and calculate the second distance between the track point and the target auxiliary edge. Then mark the target auxiliary edge with the shortest second distance as a drawing edge. The method can first draw the track line according to the target auxiliary edge closest to the starting point when there are multiple target auxiliary edges that meet the drawing conditions. During the drawing process, the auxiliary edge of the drawing track line is converted according to the distance between the track point and the target drawing edge, thereby improving the problem of track line drawing errors in the display device.

[0173] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0174] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display device, characterized in that: include: A display configured to display a whiteboard application interface, wherein the whiteboard application interface includes an auxiliary line drawing view, and the auxiliary line drawing view includes at least two auxiliary edges; The display is connected to a touch interaction module, and the touch interaction module is used to obtain a touch track input by a user, and the touch track includes at least one touch point; The controller is configured as: In response to a touch event input by a user, detecting a starting point of the touch track, wherein the starting point is a touch point of a drop event; Calculating a first distance between the starting point and the auxiliary edge; Querying a target auxiliary edge according to the first distance, the target auxiliary edge being an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to a preset distance threshold; If the number of the target auxiliary edges is greater than 1, marking the target auxiliary edge with the shortest first distance as a drawing edge, and drawing a track line of the touch track according to the drawing edge; Detecting a track point of the touch track, where the track point is a touch point of a moving event; Calculating a second distance between the trajectory point and the target auxiliary edge; The target auxiliary edge with the shortest second distance is marked as the drawing edge, so as to draw the trajectory line according to the drawing edge.

2. The display device according to claim 1, characterized in that The controller calculates a first distance between the starting point and the auxiliary edge, and is configured to: Acquire a layout file of the auxiliary line drawing view, wherein the layout file includes attribute information of the auxiliary edge; Generate an initial perpendicular foot point according to the attribute information, where the initial perpendicular foot point is a perpendicular foot point generated from the starting point to the auxiliary edge connection line; The distance between the initial perpendicular foot point and the starting point is calculated to obtain the first distance.

3. The display device according to claim 2, characterized in that The controller executes drawing the track line of the touch track according to the drawing edge, and is configured to: Obtaining the target initial perpendicular point from the starting point to the drawn edge line; Generate a track foot point according to the attribute information of the drawn edge, where the track foot point is a point of perpendicularity generated from the track point to the line connecting the drawn edge; Taking the target initial perpendicular foot point as the starting point, generating the trajectory line according to the trajectory perpendicular foot point; The trajectory line is rendered to the whiteboard application interface.

4. The display device according to claim 1, characterized in that The whiteboard application interface is displayed through a graphics layer, and the controller executes drawing the trajectory line according to the drawing edge, and is further configured to: Calling an acceleration layer, wherein the acceleration layer is a display layer that refreshes display content according to a unit area, and the display level of the acceleration layer is higher than the display level of the graphic layer when the user inputs a touch track; Refreshing the trajectory line to the acceleration layer and the graphics layer; The display is controlled to display a superimposed display image of the acceleration layer and the graphics layer.

5. The display device according to claim 4, characterized in that The controller executes refreshing of the trajectory line to the acceleration layer and the graphics layer, and is configured to: In response to detecting an operation event of adding a touch point in the touch track, refreshing the track line to the acceleration layer; When the end point of the touch track is detected, the track line is refreshed to the graphics layer, and the track line of the acceleration layer is released, and the end point is the touch point of the lift event.

6. The display device according to claim 1, characterized in that The controller is also configured to: Detecting the number of auxiliary edges of the target; When the number of the target auxiliary edges is greater than 1, setting the calibration mark to a first mark, and performing a step of calculating a second distance between the trajectory point and the target auxiliary edge, the first mark being used to characterize a state of the drawn edge to be calibrated; When the number of the target auxiliary edges is equal to 1, the calibration mark is set to a second mark, and the second mark is used to represent the non-to-be-calibrated state of the drawn edge.

7. The display device according to claim 1, characterized in that The controller is also configured to: Controlling the display to display an option list of an auxiliary line drawing view in the whiteboard application interface, wherein the option list includes a subview control; In response to a selection instruction of the subview control, acquiring a layout file of the subview control, the layout file including attribute information of a view width and height, a view position, and the auxiliary edge; The auxiliary line drawing view is rendered in the whiteboard application interface according to the layout file.

8. A display device, characterized in that: include: A display configured to display a whiteboard application interface, wherein the whiteboard application interface includes an auxiliary line drawing view, and the auxiliary line drawing view includes at least two auxiliary edges; The display is connected to a touch interaction module, and the touch interaction module is used to obtain a touch track input by a user, and the touch track includes at least one touch point; The controller is configured as: In response to a touch event input by a user, detecting a track point of the touch track; Detecting a third distance between the trajectory point and the auxiliary edge; When there is an adjacent auxiliary edge, calculating the change angle of the target trajectory point, and obtaining the standard angle between the adjacent auxiliary edge and the drawing edge, the adjacent auxiliary edge is an auxiliary edge whose third distance is less than the third distance of the drawing edge, the drawing edge is an auxiliary edge of a trajectory line for drawing the touch trajectory, the target trajectory point is a trajectory point with the latest touch time in the touch trajectory, and the change angle is obtained based on a trajectory foot point of the target trajectory point, and the trajectory foot point is a foot point of a perpendicular produced by a trajectory point connecting the auxiliary edge; If the changed angle is equal to the standard angle, the adjacent auxiliary edge is marked as a drawing edge, so as to draw the track line of the touch track according to the drawing edge.

9. The display device according to claim 8, characterized in that The controller performs calculation of the change angle of the trajectory point and is configured as follows: Querying a reference point of the target trajectory point, the reference point being a touch point having a minimum touch time difference with the touch time of the target trajectory point; Obtaining a foot point of the included angle of the target track point, wherein the foot point of the included angle includes a first foot point of the target track point to the adjacent auxiliary edge line, a second foot point of the reference point to the drawn edge line, and an initial foot point of the starting point of the touch track to the drawn edge line, wherein the starting point is the touch point of the lift event; Generate an angle vector according to the foot point of the angle, the angle vector comprising a first vector and a second vector, the first vector is generated based on the first foot point of the angle and the second foot point of the angle, and the second vector is generated based on the second foot point of the angle and the initial foot point of the angle; The angle between the first vector and the second vector is calculated to obtain a changed angle of the target trajectory point.

10. A touch track auxiliary line drawing method, characterized in that: Applied to the display device according to any one of claims 1 to 7, the method comprising: In response to a touch event input by a user, detecting a starting point of a touch track, wherein the starting point is a touch point of a drop event; Calculate a first distance between the starting point and the auxiliary edge; Querying a target auxiliary edge according to the first distance, the target auxiliary edge being an auxiliary edge in the auxiliary line drawing view whose first distance is less than or equal to a preset distance threshold; If the number of the target auxiliary edges is greater than 1, marking the target auxiliary edge with the shortest first distance as a drawing edge, and drawing a track line of the touch track according to the drawing edge; Detecting a track point of the auxiliary edge, where the track point is a touch point of a moving event; Calculating a second distance between the trajectory point and the target auxiliary edge; The target auxiliary edge with the shortest second distance is marked as the drawing edge, so as to draw the trajectory line according to the drawing edge.

Images