Display device and touch track editing method

Abstract

The embodiment of the invention provides a display device and a touch track editing method, and the method can respond to a touch event that a user inputs a target touch track, and detect a touch point of the target touch track. Generating a track line of the target touch track according to the touch point, controlling a display to display the track line, and querying the path of the track line. And if the path comprises the editing features, querying the target writing graph according to the path, and obtaining a bounding box of the target writing graph. Wherein the bounding box is used for representing the boundary range of the target writing graph. And editing the target writing graph according to the bounding box. According to the method, the written graph can be edited directly according to the target touch track comprising the editing features and the path of the target touch track, and a user does not need to erase the existing written graph, so that the problem that the written graph is inconvenient to edit is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a touch track editing method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] In order to meet the needs of scenarios such as conferences and education, for display devices that support touch interactive operations, you can run a whiteboard application and implement writing or drawing functions based on the touch area of ​​the whiteboard application. The whiteboard application can respond to the touch track input by the user in real time, and display the touch track through the application interface to form a corresponding writing graphic or drawing graphic. In addition, in order to improve the standardization of the writing process, the whiteboard application can also convert the touch track input by the user into a standard body based on text recognition technology, and render the touch track in the form of a standard body.

[0004] However, since the written graphics are rendered at a fixed position in the application interface, if the user makes a mistake or misses a word when writing, the user needs to erase all the related written graphics and rewrite them to correct the touch track. In this way, when editing the written graphics, the user may repeatedly perform the erasing and rewriting operations, which increases the complexity of editing the written graphics and reduces the editing efficiency of the written graphics. Summary of the invention

[0005] The present application provides a display device and a touch track editing method to solve the problem of low editing efficiency of written graphics in a display device.

[0006] In a first aspect, some embodiments of the present application provide a display device, including a display and a controller. The display is configured to display a whiteboard application interface, the display is connected to a touch interaction module, the touch interaction module is used to detect a touch track input by a user, the whiteboard application interface includes a writing graphic, and the writing graphic is a track line generated based on the touch track;

[0007] The controller is configured as:

[0008] In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track;

[0009] Generate a track line of the target touch track according to the touch point;

[0010] Controlling the display to display the trajectory line, and querying the path of the trajectory line;

[0011] If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic;

[0012] Editing is performed on the target written graphic according to the bounding box.

[0013] In a second aspect, some embodiments of the present application further provide a method for editing a touch track, which is applied to the display device provided in the first aspect, wherein the display device includes a display and a controller, wherein the display is configured to display a whiteboard application interface, wherein the display is connected to a touch interaction module, wherein the touch interaction module is used to detect a touch track input by a user, wherein the whiteboard application interface further includes a writing graphic, wherein the writing graphic is a track line generated based on the touch track; the method includes:

[0014] In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track;

[0015] Generate a track line of the target touch track according to the touch point;

[0016] Controlling a display to display the trajectory line, and querying the path of the trajectory line;

[0017] If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic;

[0018] Editing is performed on the target written graphic according to the bounding box.

[0019] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a method for editing a touch track, and the method can detect the touch point of the target touch track in response to a touch event of a target touch track input by a user. Generate a trajectory line of the target touch track according to the touch point, control the display to display the trajectory line, and query the path of the trajectory line. If the path includes an editing feature, query the target writing graphic according to the path, and obtain the bounding box of the target writing graphic. Among them, the bounding box is used to characterize the boundary range of the target writing graphic. Then edit the target writing graphic according to the bounding box. The method can directly edit the writing graphic according to the target touch track including the editing feature and the path of the target touch track, without the user having to erase the existing writing graphic, thereby improving the problem that the writing graphic is inconvenient to edit. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0022] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0023] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0024] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0025] Figure 5 A schematic diagram of touch tracks provided in some embodiments of the present application;

[0026] Figure 6 A schematic diagram of a whiteboard application interface provided in some embodiments of the present application;

[0027] Figure 7 A schematic diagram of a whiteboard application display system provided in some embodiments of the present application;

[0028] Figure 8 A schematic diagram of an acceleration layer provided for some embodiments of the present application;

[0029] Fig. 9 A schematic diagram of the rendering process of the trajectory line when the standard body provided in some embodiments of the present application is in the open state;

[0030] Fig.10 A schematic diagram of the rendering effect of the trajectory line when the standard body provided in some embodiments of the present application is in the open state;

[0031] Fig.11 An example diagram of editing a writing graphic when writing incorrectly provided in some embodiments of the present application;

[0032] Fig.12 A schematic diagram of the effect of the touch track editing method provided in some embodiments of the present application;

[0033] Fig.13A schematic diagram of the principle of editing features by angle detection provided in some embodiments of the present application;

[0034] Fig.14 A schematic diagram of the principle of editing features by detecting distance differences provided in some embodiments of the present application;

[0035] Fig.15 A schematic diagram of a process of marking a touch track provided in some embodiments of the present application;

[0036] Fig.16 A schematic diagram of a process of editing a target written graphic under a deletion track provided in some embodiments of the present application;

[0037] Fig.17 A schematic diagram of a process of editing a target written graphic under an insertion track provided in some embodiments of the present application;

[0038] Fig.18 A schematic diagram of a process of editing a target written graphic under a replacement track provided in some embodiments of the present application;

[0039] Fig.19 A schematic diagram of a process of scaling a target writing graphic under a replacement trajectory provided in some embodiments of the present application;

[0040] Fig. 20 A schematic diagram of the effect of replacing a trajectory provided in some embodiments of the present application. DETAILED DESCRIPTION

[0041] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0042] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0043] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0044] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0045] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0046] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 may be a remote controller, a stylus pen, or the like.

[0047] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0048] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0049] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0050] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0051] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0052] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0053] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0054] In some embodiments, the display 260 is connected to a touch interaction module so that the display device 200 supports touch interaction operations. The touch interaction module can be a built-in module or an external module. For example, in order to implement touch interaction operations, for a display device 200 with a built-in touch interaction module, the display 260 can be a touch screen, which can receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy.

[0055] For the display device 200 with an external touch interaction module, a terminal, device, or component with a touch function can be connected through the device interface of the display device 200. For example, the USB or USB-C interface of the display device 200 is connected to a touch panel, and the touch panel can receive the touch track input by the user in real time and transmit the touch track to the display device 200 through the USB channel. The display device 200 can perform an interactive response based on the received touch track in combination with the touch application, so that the display device 200 supports touch interaction operations.

[0056] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0057] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0058] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0059] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0060] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0061] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0062] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0063] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0064] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0065] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0066] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0067] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0068] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0069] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0070] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0071] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0072] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0073] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0074] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0075] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0076] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0077] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0078] The display device 200 can also deploy other device interfaces, hardware, and drivers based on the kernel layer according to the specific display requirements of the display device 200. For example, in order to enrich the display types of the display device 200, an HDMI interface and a USB interface can be set on the display device 200, wherein the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver. The USB interface can connect devices with signal and data transmission functions and interactive functions, and based on the USB driver, access signals, transmit data, and perform specific interactive actions.

[0079] For a display device 200 that supports touch interaction operations, the display device 200 has a built-in touch interaction module, and can receive touch data input by the user through the touch interaction module. According to the interaction principle of the touch interaction module, the touch interaction module can detect relevant parameters in the touch action. For example, a capacitive touch layer is provided on the display 260 of the display device 200. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touch position will change. At this time, the touch interaction module can record the position of the capacitance change, and record the time when the capacitance change occurs in combination with the timer in the controller to generate touch data.

[0080] In some embodiments, the physical signal detected by the touch interaction module needs to be converted before it can be read by the controller 250 of the display device 200. For example, the analog signal of the capacitance change needs to be converted into a digital signal and then transmitted to the controller 250 of the display device 200. In order to adapt to the screen display process of the display device 200, the touch data detected by the touch interaction module also needs to be coordinate mapped so that the position point corresponding to the touch data can have a corresponding relationship with the display 260 of the display device 200 or the pixel point in the user interface. For example, the coordinate position detection accuracy of the touch interaction module is 65536×65536, and the resolution of the display 260 of the display device 200 is 3840×2160. After the touch interaction module detects the touch data, the coordinate mapping of the touch point position in the width and height directions can be performed according to the ratio of 65536 / 3840 and 65536 / 2160, so that the display device 200 can perform related operation responses based on the mapped touch point position coordinates.

[0081] The touch data may include different operation parameters according to the user's input method. Figure 5As shown, the touch data may include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. In a touch interaction process, the touch event types that can be detected by the touch interaction module include: down events, move events, and up events of touch actions. Corresponding to the entire touch track formed by the touch data, it can be the starting point [down; x1, y1, t1], the track point [move; x2, y2, t2], and the end point [up; x3, y3, t3]. Each touch data records the corresponding touch event type, touch time t, and touch position (x, y), that is, the touch points of the touch track can include the starting point, track point, and end point.

[0082] It should be noted that, since the user's finger or touch device is in surface contact with the touch interaction module when performing touch interaction operations, that is, the touch interaction module can detect a contact area, and the contact area includes multiple contact points. Therefore, when the display device 200 performs interactive control, it can detect (or specify) a contact point in the contact area as a touch point. For example, when the contact area is an elliptical area formed when the user's finger contacts the touch interaction module, the display device 200 can traverse the contact point coordinates in the elliptical area, determine the coordinate extremes in two mutually perpendicular directions, to delineate the major axis and minor axis of the elliptical area, and use the contact point at the intersection of the major axis and the minor axis as the touch point.

[0083] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, the number of touch points and the type of touch events, so as to detect the user's touch input gesture, thereby performing different interactive responses for different input gestures. For example, when the number of touch points in the touch trajectory detected by the touch interaction module is 1, the touch event type only includes a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold. In addition, if the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 executes the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0084] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch time, touch position, number of touch points and touch event type, such as double-click gesture, slide gesture, long press gesture, multi-finger click gesture, multi-finger slide gesture, etc. Different interaction gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, slide track shape, etc. Different touch parameters can be used to perform different interactive controls.

[0085] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection based on the dynamic change rules of touch time, touch position, number of touch points and touch event type. For example, when the touch interaction module detects that the touch trajectory of the user input includes multiple touch points, and the touch position of each touch point approaches each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grab gesture. Based on complex gesture detection, the display device 200 can perform more interactive responses. For example, when the detected touch gesture is a grab operation for an image view, the display device 200 can respond to the grab gesture and perform a control action to select and move the image display position.

[0086] In some embodiments, the touch gesture can also be combined with the touch position to form a touch interaction operation based on a specific position. For example, when the touch interaction module detects that the down event position during the touch operation is at the top of the display 260, the touch point position corresponding to the move event moves downward. When the up event is detected, the display device 200 can determine that the current touch gesture is to slide down from the top of the screen. Therefore, the display device 200 can respond to the touch gesture and control the display 260 to present a drop-down list.

[0087] Some touch interaction gestures can be used to perform interactive actions based on the operating system of the display device 200, which are called global gestures. For example, the operation of sliding down from the top of the screen to call out a drop-down list in the above example of the display device 200. Some touch interaction gestures can be used to perform interactive actions based on specific touch applications. For example, when the display device 200 runs a whiteboard application, the sliding touch operation input by the user can be used to draw a track line on the whiteboard.

[0088] The touch application can be a system application or a third-party application developed based on a specific display device type and user needs. The touch application can be installed in the display device 200 and started to run under user control. For example, the user can control the display device 200 to run the whiteboard application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0089] It should be noted that the whiteboard application can be an independent application, an associated application, or a functional module in a touch application. Different forms of whiteboard applications can have different startup or calling methods. For example, a user can start an independent whiteboard application through the application interface of the display device 200; a user can also start a whiteboard application by executing a "whiteboard" operation on a conference application; a user can also control the conference application to start the whiteboard functional module by clicking the "whiteboard" icon on the conference application.

[0090] Different forms of whiteboard applications may present different forms of whiteboard application interfaces after startup. For example, when a user starts a whiteboard application in an associated application form, the display device 200 may automatically enter a dual-screen (or split-screen) mode. In the dual-screen (or split-screen) mode, the display device 200 may display a user interface through two display areas, i.e., the main screen displays the original application (such as a conference application) interface, and the secondary screen displays the whiteboard application interface.

[0091] After the whiteboard application is running, the display device 200 can be switched to display the whiteboard application interface. Figure 6 As shown, the whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, graphics, line shape, color, erase, clear screen, selection, etc. The user can click any control to draw graphics according to specific drawing needs.

[0092] In some embodiments, the display device 200 classifies the touch track input by the user based on the whiteboard application according to the selected state of the drawing tool control. For example, when the drawing tool control such as pen shape, graphic shape, line shape, etc. is in the selected state, the touch track input by the user is marked as a drawing event; when the erasing drawing tool is in the selected state, the touch track input by the user is marked as an erasing event. The display device 200 then responds to the touch track of the drawing event or the erasing event and executes different programs according to the preset interaction strategy.

[0093] In order to distinguish the touch track, the whiteboard application includes a drawing area and a menu area. The drawing area is used to display the track line formed by the touch track and execute the drawing or erasing event of the touch track; the menu area is used to display the above-mentioned drawing tool control and execute the click event of the drawing tool control. When the user inputs the touch track in the drawing area, the display device 200 draws the track line of the touch track and renders the track line to the whiteboard application interface; when the user inputs the touch track in the menu area, the display device 200 modifies the selection state of the drawing tool control according to the position of the touch track.

[0094] like Figure 7As shown, the display device 200 can render and display the drawing picture in real time through an image generation (Image Producer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module and a display (screen).

[0095] In the process of displaying the drawing picture, the graphic interface (openGLES) of the image generation module can be connected to the whiteboard application to generate the drawing image content and pass the image content to the image rendering module. The image rendering module is used to control the display 260 to display the content, that is, to render the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the picture frames can be stored in the synthesis module through the frame buffer (framebuffer) and the video buffer (video buffer), respectively, and the picture content is formed in the corresponding layer through the video process module. That is, the image frames generated by the video playback process form the picture content through the video layer (video plane), and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the primary OSD layer (primary OSD plane). Finally, the display device 200 forms the final picture through the blending unit (blend) of the video process module and displays it on the display 260.

[0096] Since the whiteboard application will present the user's touch trajectory in real time according to the user's touch operation, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. To this end, in some embodiments, the display device 200 can present a real-time painting effect by setting an acceleration layer. That is, in the video process module of the display device 200, an additional extended display layer (external OSD plane) can be set on the basis of the main OSD layer and the video layer. Among them, the extended display layer can be configured with different layer characteristics according to the specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0097] Taking the display device 200 running a whiteboard application as an example, in order to improve the real-time response speed of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). Figure 8 As shown, the display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the track line drawn by the acceleration layer is updated to other layers, thereby reducing the input delay and improving the real-time display effect of the whiteboard application drawing process.

[0098] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, that is, when the display device 200 detects the end point of the lift event, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawn graphics, thereby eliminating the need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0099] Based on the above display device 200, in some embodiments, the user can input the touch track of various characters in the display device 200, so that the display device 200 presents the corresponding writing graphics. That is, the user can write the corresponding touch track in the display device 200 according to the strokes of each character, and the display device 200 maps the touch track to a track line according to the preset mapping relationship, and renders the track line to the OSD layer, so that the display 260 displays the track line. Among them, the track lines of each stroke in the character can constitute a complete writing graphic.

[0100] like Fig. 9 As shown, in order to improve the standardization of the written graphics, the display device 200 can also convert the trajectory line of the written graphics into a standard body, that is, in some embodiments, the display device 200 also detects the enabled state of the standard body. If the standard body is in the enabled state, the display device 200 detects the end point of the touch trajectory. Among them, the end point is the touch point of the lift (up) event. The end point is used as the timing starting point, and the touch point is received within the touch event cycle. If the touch point is not received within the touch time period, the standard body model is called, and the position coordinates of the converted touch point are obtained. The converted touch point includes a touch point whose touch time is earlier than the touch time of the end point. Then the position coordinates of the converted touch point are input into the standard body model to generate a standard body data object. Then, based on the standard body data object, the trajectory line of the converted touch point is rendered in the whiteboard application interface to obtain the written graphics.

[0101] That is to say, the whiteboard application of the display device 200 can be set with a conversion function for standard fonts. Among them, the standard fonts can include a variety of different fonts. When the standard font function is in an enabled state, the display device 200 can detect the end point of the touch trajectory, and when the end point of a certain touch trajectory is detected and no new touch points are detected within a preset time period, the original touch data of the touch points that have not been processed before the end point are input into the trained standard font model. The standard font model will convert these touch points into corresponding standard font coordinate points, and output the converted standard font coordinate points to generate a new standard font data object. Among them, the standard font data object and the touch data of the touch points are two different data objects, which can be stored separately in the display device 200, for example, stored in the memory or file of the display device 200. After generating the standard font data object, the display device 200 renders the trajectory line of the touch trajectory based on the standard font data object, so that the whiteboard application interface can present the writing graphics in standard fonts. On the contrary, if the standard font function is in a disabled state, the trajectory line is directly rendered according to the touch data of the touch points.

[0102] For example, the preset touch time period is 3 seconds. When the display device 200 detects the end point a of a certain touch trajectory and no new touch points are detected within 3 seconds, it inputs all the touch points whose touch time is earlier than the touch time of point a and have not been processed into the standard font model, so that the standard font model outputs a standard font data object. The display device 200 then renders a more regular writing graphic in the whiteboard application interface, as Fig.10 shown.

[0103] However, since the writing graphic is rendered at a fixed position on the OSD layer, if the user wants to modify the writing graphic, all relevant and already rendered writing graphics need to be erased, and then the touch trajectory of the erased part needs to be input again. For example, as Fig.11 shown, when the user writes the writing graphic of "Good morning" on the display device 200, it is miswritten as "Good". At this time, the user needs to erase the writing graphic of "Good" and then rewrite the writing graphics of "Up" and "Good". In this way, when the user edits the writing graphic, the operations of erasing and rewriting need to be repeatedly performed, resulting in an increase in the complexity of the writing graphic editing process, thereby reducing the editing efficiency of the writing graphic.

[0104] Based on the above application scenarios, in order to improve the problem of low writing graphic editing efficiency, some embodiments of the present application provide a display device 200, including a display 260 and a controller 250. Among them, the display 260 is configured to display a whiteboard application interface; the display 260 is connected to a touch interaction module, and the touch interaction module is used to detect the touch trajectory input by the user; the whiteboard application interface includes a writing graphic, and the writing graphic is a trajectory line generated based on the touch trajectory; as Fig.12As shown, the controller 250 is configured to perform the following program steps:

[0105] S100: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track.

[0106] After detecting a touch event of a target touch track input by a user, the display device 200 detects a touch point of the target touch track in response to the touch event. The touch track includes at least one touch point, and the touch point may include the starting point, track point, and end point described in the above embodiment.

[0107] In some embodiments, when the display device 200 detects the end point of the touch track, it also generates a bounding box of the touch track based on all the touch points of the touch track. The bounding box is used to characterize the boundary range of the touch track, that is, the closed area formed by the bounding box can cover all the touch points of the touch track. For example, the bounding box can be a rectangular box, etc. The display device 200 can determine the area where the touch track is located through the bounding box of the touch track.

[0108] It should be noted that the boundary box of the touch track is the attribute data stored when the display device 200 stores the touch track, and will not be displayed in the whiteboard application interface.

[0109] In order to facilitate the recognition of the writing pattern, in some embodiments, the display device 200 can also mark multiple touch tracks constituting a standard writing pattern as a group according to the standard writing pattern, and use the union of the bounding boxes corresponding to the touch tracks in the same group as the bounding box of the writing pattern. The display device 200 can determine the area where the writing pattern is located through the bounding box of the writing pattern.

[0110] Similarly, for writing graphics that have not been converted into standard fonts, in some embodiments, the display device 200 can identify the touch tracks based on OCR (Optical Character Recognition) technology, mark the touch tracks that constitute a font as a group, and use the union of the corresponding bounding boxes of the touch tracks in the same group as the bounding box of a writing graphic.

[0111] S200: Generate a track line of a target touch track according to the touch point.

[0112] After detecting the touch points of the target touch track, the display device 200 generates the track line of the touch track in real time according to each touch point, that is, the display device 200 can map the touch points to corresponding bitmap data and refresh the bitmap data to the OSD layer in real time, thereby forming the track line of the target touch track.

[0113] In some embodiments, when generating the trajectory line, the display device 200 also obtains the touch color information of the target touch trajectory. The touch color information can be the color information selected by the user based on the whiteboard application interface, or it can be the default color information in the display device 200. The touch color information includes a color value. When the display device 200 generates the bitmap data, it is necessary to set the pixel point to the color value in the touch color information.

[0114] In some scenarios, the display device 200 will enable the acceleration layer to achieve the hand-following effect. Therefore, in some embodiments, the display device 200 also detects the activation status of the acceleration layer. If the display device 200 enables the acceleration layer, the display device 200 refreshes the bitmap data to the acceleration layer in real time according to the touch point of the target touch track, and refreshes the bitmap data of the entire target touch track to the OSD layer when the end point of the touch track is detected. If the display device 200 does not enable the acceleration layer, the display device 200 refreshes the bitmap data to the OSD layer in real time according to the touch point of the target touch track.

[0115] S300: Control the display to display the trajectory line, and query the path of the trajectory line.

[0116] After generating the target touch track, the display device 200 controls the display 260 to generate the target touch track, queries the path of the track in the whiteboard application interface, and determines the relative position of the target touch track and each written figure through the path.

[0117] The display device 200 can identify the characteristics of the path by querying the path of the trajectory line. For example, the editing of the written graphics can be triggered by the path of the approximate straight line. Therefore, in some embodiments, the display device 200 can detect whether the path of the trajectory line meets the characteristics of the approximate straight line through the angle, that is, when the display device 200 queries the path of the trajectory line, it reads the touch information of the touch point. Among them, the touch information includes the position coordinates and touch time of the touch point. Then, the touch points adjacent in time are connected according to the touch time to generate the trajectory segment of the target touch trajectory. The sub-angle of the trajectory segment is calculated. If the difference between the sub-angle and the standard angle is less than or equal to the angle threshold, the marked path includes the editing feature; if the difference between the sub-angle and the standard angle is greater than the angle threshold, the marked path does not include the editing feature. Among them, the standard angle is a preset angle, which can be set according to the characteristics of the straight line.

[0118] For example, the standard angle is 180°. When querying the path of the target touch track, the display device 200 reads the touch information of all touch points of the target touch track, including the touch time and touch coordinates. Then, the touch points adjacent in time are connected to generate a track segment. Fig.13 As shown, every three adjacent touch points on the path constitute Fig.13The line shown in FIG. 1 and the sub-angle α exists between the track segments. If the difference between the sub-angle α and 180° of all track segments in the target touch track is less than or equal to the angle threshold, the marked path includes the editing feature; if the difference between the sub-angle α and 180° in the target touch track is greater than the angle threshold, the marked path does not include the editing feature.

[0119] Alternatively, in some embodiments, the display device 200 can also detect whether the path of the trajectory line satisfies the characteristics of an approximate straight line through the distance difference, that is, each two adjacent touch points on the path constitute a trajectory segment, and the length of the trajectory segment is the distance between the two touch points. Calculate the sum of the lengths of all trajectory segments in the target touch trajectory as the judgment length value. At the same time, calculate the distance between the starting point and the end point in the target touch trajectory as a reference length value. Then calculate the difference between the above-mentioned judgment length value and the reference length value. If the difference between the judgment length value and the reference length value is less than or equal to a fixed ratio of the reference length value, such as 1 / 10, 1 / 15, etc., the marked path includes an editing feature; if the difference between the judgment length value and the reference length value is greater than a fixed ratio of the reference length value, the marked path does not include an editing feature.

[0120] For example, Fig.14 As shown, P0 and P2 are the starting point and the ending point in the target touch track, and P1 is the track point of the target touch track. The sum of the track segment P0-P1 and the track segment P1-P2 is d1, and the distance between the touch points P0-P2 is d2, so the judgment length value is d1 and the reference length value is d2. If d1-d2≤(1 / 10)*d2, the marked path includes the editing feature; if d1-d2>(1 / 10)*d2, the marked path does not include the editing feature.

[0121] It is understandable that the above approximate straight line method is only one way of implementing the present application, and the editing features of the path may also adopt more different features, and the present application does not limit this.

[0122] S400: If the path includes an editing feature, query the target writing graphic according to the path, and obtain a bounding box of the target writing graphic.

[0123] After the display device 200 finds that the path of the trajectory line includes the editing feature, it searches for the target writing figure according to the path and obtains the bounding box of the target writing figure. Among them, the target writing figure is the writing figure to be edited by the target touch trajectory, and the bounding box is used to characterize the boundary range of the target writing figure. For example, when the target touch trajectory is a deletion trajectory, the display device 200 needs to delete the writing figure, then it can search for the writing figure that intersects with the path as the target writing figure, etc.

[0124] The bounding box of the writing graphic is generated based on the data object that forms its trajectory line. For example, when the target writing graphic to be edited is a non-standard body, the display device 200 queries the bounding box of the writing graphic based on the data object of the original touch data; when the target writing graphic to be edited is a standard body, the display device 200 queries the bounding box of the writing trajectory based on the standard body data object.

[0125] S500: Editing the target written graphic according to the bounding box.

[0126] After querying the target writing graphic that needs to be edited, the display device 200 performs editing on the target writing graphic according to the boundary box of the target writing graphic, such as deletion, insertion, replacement, etc. The display device 200 provided in the embodiment of the present application can trigger the editing of the target writing graphic through the target touch track including the editing feature, which can improve the problem of erasing a large number of writing graphics when editing the writing graphics, thereby improving the editing efficiency of the writing graphics.

[0127] In order to realize the editing of different functions, such as Fig.15 As shown, in some embodiments, the display device 200 also obtains the bounding box of the writing graphic, and detects the relative position of the path and the writing graphic based on the bounding box. If there are at least two intersections between the path and the first target bounding box, and the intersections are located at opposite sides of the first target bounding box, that is, the path of the trajectory line runs through two opposite sides of the first target bounding box, the display device 200 marks the target touch trajectory as a deletion trajectory, and marks the writing graphic of the first target bounding box as the target writing graphic. For example, when the editing feature is an approximate straight line feature, if the display device 200 detects that the path of the trajectory line includes an approximate straight line feature, and the path runs through two opposite sides of a certain bounding box, the display device 200 marks the target touch trajectory as a deletion trajectory, and marks the writing graphic of the bounding box as the target writing graphic.

[0128] Since the writing graphics are rendered at a fixed position on the whiteboard application interface, when the user writes too much, if the excess writing graphics are erased, some blank areas will appear in the whiteboard application interface, making the remaining writing graphics irregularly arranged and affecting the display effect of the writing graphics.

[0129] Therefore, to improve the above-mentioned typesetting problem, regarding the deletion trajectory, in some embodiments, the display device 200 responds to the touch event of the deletion trajectory, clears the target writing graphic, and reads the first width and height information of the first target bounding box. The arrangement attribute of the writing graphic is detected, and the supplementary writing graphic of the target writing graphic is obtained according to the arrangement attribute. Then, the supplementary writing graphic is moved according to the first width and height information and the arrangement attribute. Among them, the first width and height information is the horizontal and vertical range (bounds) of the first target bounding box; the arrangement attribute is the attribute information when the writing graphics are arranged, which can be arranged horizontally or vertically, and the arrangement attribute also includes the position attributes of the front and rear positions.

[0130] For example, as Fig.16 shown, Fig.16 the writing graphics "Ni", "Hao", and "Ya" shown are arranged horizontally, and the editing feature is a feature similar to a straight line. When the user inputs a touch trajectory passing through "Hao" in the whiteboard application interface, the display device 200 responds to the touch trajectory, renders the trajectory line of the touch trajectory in real time, and marks it as a deletion trajectory according to the path of the trajectory line. Then, in response to the touch event of the deletion trajectory, the writing graphic of "Hao" is cleared, and the width and height information of the bounding box of "Hao" is detected, including the width B1 and the height H1. When it is detected that the arrangement attribute is horizontal arrangement, the supplementary writing graphic "Ya" located at the rear position of "Hao" is moved forward by the length of B1.

[0131] The arrangement attribute is determined based on the input time of the writing graphic. That is, in some embodiments, when the display device 200 detects the touch trajectory for generating the writing graphic, it also marks the horizontal or vertical arrangement attribute according to the input time of the touch trajectory, and marks the front and rear positions of the horizontal or vertical arrangement according to the input time of the touch trajectory. For example, when the display device 200 detects the second writing graphic input by the user, it obtains the positions of the second writing graphic and the first writing graphic. If the two are on the same horizontal line, the arrangement attribute is marked as horizontal arrangement, and according to their positions on the horizontal line, the side close to the first writing graphic is marked as the front position.

[0132] Such as Fig.15As shown, in some embodiments, the display device 200 obtains the bounding box of the written graphic and detects the relative position between the path and the written graphic according to the bounding box. If there is no intersection between the path and the bounding box of the written graphic, and the path is located between the second target bounding box and the third target bounding box, the target touch trajectory is marked as an insertion trajectory, and the written graphic in the second target bounding box is marked as the target written graphic. For example, when the editing feature is an approximate straight line feature, if the display device 200 detects that the path of the trajectory line includes the feature of an approximate straight line and the path is located between the bounding boxes of two written graphics, the display device 200 marks the target touch trajectory as an insertion trajectory and marks the written graphic in the second position among the two written graphics as the target written graphic.

[0133] Regarding the insertion trajectory, in some embodiments, the display and device 200 obtains the inserted written graphic in response to the touch event of the insertion trajectory. Among them, the inserted written graphic is generated based on the touch points whose touch time is later than the touch time of the termination point in the target touch trajectory, and the termination point is the touch point of the lift event. Then read the second width and height information of the second target bounding box, and scale the inserted written graphic according to the second width and height information. Detect the arrangement attribute of the written graphic, and obtain the associated written graphic of the target written graphic according to the arrangement attribute. Move the inserted written graphic to the first center position of the target written graphic, and move the target written graphic according to the first width and height information and the arrangement attribute. Among them, the first center position is the center coordinates of the second target bounding box.

[0134] For example, as Fig.17 shown, Fig.17 As shown, the written graphics "你 (nǐ)", "好 (hǎo)", and "呀 (ya)" are arranged horizontally, and the editing feature is an approximate straight line feature. The user inputs a touch trajectory between "你 (nǐ)" and "好 (hǎo)" in the whiteboard application interface. The display device 200 then responds to the touch trajectory, renders the trajectory line of the touch trajectory in real time, and marks it as an insertion trajectory according to the path of the trajectory line. Then, in response to the touch event of the insertion trajectory, obtain the inserted written graphic "们 (men)", and detect the line height information of the bounding box of "好 (hǎo)" and the width information width B1 after the inserted "们 (men)" is scaled proportionally. When it is detected that the arrangement attribute is horizontal arrangement, move "们 (men)" to the position of "好 (hǎo)", and move the written graphic "好 (hǎo)" in the second position after "好 (hǎo)" and its associated written graphic "呀 (ya)" backward by a length of B1.

[0135] In some embodiments, the display device 200 also obtains the bounding box of the written graphic, detects the relative position between the detection path and the written graphic according to the bounding box, and then calculates the proportion of the path in the bounding box according to the relative position. If the proportion of the path in the third target bounding box is greater than or equal to the replacement threshold, the target touch trajectory is marked as a replacement trajectory, and the written graphic in the third target bounding box is marked as the target written graphic. For example, when the replacement threshold is 75%, when the display device 200 detects that the proportion of the path in the bounding box of a certain written graphic is greater than or equal to 75%, the touch trajectory of the path can be marked as a replacement trajectory.

[0136] For the replacement trajectory, in some embodiments, the display device 200 responds to the touch event of the replacement trajectory, reads the third width and height information of the third target bounding box, and scales the trajectory line of the target touch trajectory according to the third width and height information. Then, the trajectory line of the target touch trajectory is moved to the second center position of the target written graphic. The second center position is the center coordinates of the third target bounding box.

[0137] For example, as Fig.18 shown, Fig.18 the written graphics "ni", "hao", "ya" as shown. The user inputs a touch trajectory above "ya" in the whiteboard application interface. The display device 200 then responds to the touch trajectory, renders the trajectory line of the touch trajectory in real time, forms the font graphic of "ma", and marks the touch trajectory as a replacement trajectory. The display device 200 then responds to the touch event of the replacement trajectory, detects the width and height information of the bounding box of "ya", including the width B1 and the height H1. Then, the trajectory line of the replacement trajectory is scaled by the height H1, and the trajectory line of "ma" is moved to the center position of the bounding box of "ya".

[0138] In some embodiments, the replacement trajectory can also be a touch trajectory including specific editing features. For example, the path can be a touch trajectory that encloses a circular area. The user can draw a touch trajectory that can enclose a circular area outside the written graphic to be replaced, and draw a new written graphic within the circular area to replace the new written graphic into the circular area.

[0139] It can be understood that the replacement trajectory can also be determined based on the time when the user draws the written graphic and other methods. The present application does not limit this.

[0140] Since the user may also input the writing graphics of punctuation marks when inserting or replacing writing graphics, such as commas, periods, greetings, quotation marks or exclamation marks, etc., different symbols need to be mapped to different positions. Therefore, in some embodiments, if the target touch track is an insertion track or a replacement track, the display device 200 performs pre-conversion on the updated writing graphics through the standard body model. Among them, the updated writing graphics are newly drawn by the user in response to the insertion track or the replacement track; the pre-conversion is to identify the insertion writing graphics or the replacement track in the database through the standard body model, but the converted track line is not drawn to the whiteboard application interface. If the updated writing graphics are the writing graphics of punctuation marks, the position attributes of the punctuation marks are queried, and the updated writing graphics are mapped to different positions according to the position attributes. For example, the comma is mapped to the lower right corner of the target writing graphic boundary box, etc.

[0141] In some embodiments, the insertion track may also be a touch track including specific editing features, such as a touch track whose path includes a preset angle range, a touch track whose path can enclose a closed area, etc. After identifying the editing features of the insertion track, the display device 200 may mark the touch track as the insertion track.

[0142] In one embodiment, when the display device 200 detects the insertion trajectory, it is also necessary to obtain the touch trajectory of the inserted writing figure. Therefore, after detecting the end point of the insertion trajectory, the display device 200 detects the touch point whose touch time is later than the end point of the insertion trajectory, and generates the trajectory line of the inserted writing figure through the touch point whose touch time is later than the insertion trajectory. For example, after detecting the end point of the insertion trajectory, the display device 200 collects the touch points of the touch trajectory; if no new touch points are detected within a certain touch cycle, the collection of the touch points is terminated, and a trajectory line is generated according to the touch trajectory composed of the collected touch points as the inserted writing figure.

[0143] It is understandable that the above-mentioned editing features can be any features of approximate straight lines, approximate angles or approximate figures, and the query target writing figure can also adopt different strategies according to different editing features. This application does not make any limitation on this.

[0144] In addition, since some non-standard writing graphics are formed by multiple touch tracks, there may not be a concept of a bounding box for the entire writing graphic. Therefore, in some embodiments, the display device 200 can detect the rendering position of the writing graphic in the whiteboard application interface, detect the distance between the writing graphics through the rendering position, and generate the width and height information of each row and / or column of the writing graphics through the distance.

[0145] For example, Fig.19As shown, the inserted writing graphic is "Li Si", and its height is insertBounds. By detecting the rendering position of "Zhang San", the height of the row where Zhang San is located is obtained as toBounds, and "Zhang San" is cleared. Set the height of insertBounds to be equal to the height of toBounds, scale the writing graphic "Li Si" proportionally, and move it to the position of "Zhang San".

[0146] Alternatively, in some embodiments, the display device 200 can also respond to an insertion trajectory whose path can enclose a closed area, detect the width and height information of the closed area, and scale the inserted writing graphic through the width and height information of the closed area. For example, as Fig. 20 shown, the editing feature of the insertion trajectory is a closed rectangle. The display device 200 can scale the inserted writing graphic "Li Si" according to the rectangle, clear "Zhang San" within the rectangle, and move the scaled "Li Si" to the position of "Zhang San".

[0147] However, since the writing graphic that has not been converted to the standard form may have situations such as connected strokes and small spacing, it is difficult to input an insertion trajectory that does not intersect with the bounding box of the writing graphic. To enable the insertion trajectory to be input normally, in some embodiments, if the path includes an editing feature, the line height is obtained from the width and height information of each line of the writing graphic. When the length of the path is greater than the line height of the writing graphic, the relative position between the path and the writing graphic is detected. If the path has at least two intersections with a certain bounding box and the two intersections are located on the opposite sides of the bounding box, the target touch trajectory is marked as a deletion trajectory; if the path has an intersection with a certain bounding box, but the intersection is not located on the opposite side of the bounding box, the target touch trajectory is marked as an insertion trajectory.

[0148] That is to say, the display device 200 can also determine whether the target touch trajectory includes an editing feature based on the length of the path. Only when the length of the path is greater than the line height of the writing graphic, the display device 200 marks that the path includes an editing feature. In this way, the display device 200 can accurately edit the writing graphic that has not been converted to the standard form, thereby improving the user's interaction experience.

[0149] It can be understood that when the editing features are different, the display device 200 can mark the target touch trajectory through different judgment logics. This application does not limit this.

[0150] Moreover, due to the characteristics of text, inserting a writing graphic may include different width-to-height ratios. For example, the width of a writing graphic such as "one" is much greater than its height. At this time, if it is scaled according to the length of the bounding box, the scaled writing graphic may have an inconsistent problem. Therefore, in some embodiments, the display device 200 detects the bounding box of the inserted writing graphic and obtains the line height from the width and height information of each line of writing graphics. Calculate the height of the inserted writing graphic according to the bounding box of the inserted writing graphic. If the height of the inserted writing graphic is less than the line height, no scaling is performed on the inserted writing graphic. To improve the user's interaction experience, if the target touch trajectory includes an editing feature, after the display device 200 performs editing on the target writing graphic, it releases the trajectory line of the target touch trajectory so that it disappears in the whiteboard application interface; if the target touch trajectory does not include an editing feature, the display device 200 continues to maintain the trajectory line of the target touch trajectory so that it is normally displayed in the whiteboard application interface.

[0151] Based on the above display device 200, some embodiments of the present application further provide a method for editing a touch trajectory, which is applied to the display device 200 provided in the above embodiments. The display device 200 includes a display 260 and a controller 250. Among them, the display 260 is configured to display a whiteboard application interface. The display 260 is connected to a touch interaction module, and the touch interaction module is used to detect a touch trajectory input by a user. The whiteboard application interface further includes a writing graphic, and the writing graphic is a trajectory line generated based on the touch trajectory. As Fig.12 shown, the method includes the following program steps:

[0152] S100: In response to a touch event of a user inputting a target touch trajectory, detect the touch points of the target touch trajectory;

[0153] S200: Generate a trajectory line of the target touch trajectory according to the touch points;

[0154] S300: Control the display to display the trajectory line and query the path of the trajectory line;

[0155] S400: If the path includes an editing feature, query the target writing graphic according to the path and obtain the bounding box of the target writing graphic. The bounding box is used to represent the boundary range of the target writing graphic;

[0156] S500: Edit the target writing graphic according to the bounding box.

[0157] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a method for editing a touch track, and the method can detect the touch point of the target touch track in response to a touch event of a target touch track input by a user. Generate a trajectory line of the target touch track according to the touch point, control the display to display the trajectory line, and query the path of the trajectory line. If the path includes an editing feature, query the target writing graphic according to the path, and obtain the bounding box of the target writing graphic. Among them, the bounding box is used to characterize the boundary range of the target writing graphic. Then edit the target writing graphic according to the bounding box. The method can directly edit the writing graphic according to the target touch track including the editing feature and the path of the target touch track, without the user having to erase the existing writing graphic, thereby improving the problem that the writing graphic is inconvenient to edit.

[0158] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0159] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display device, characterized in that: include: A display configured to display a whiteboard application interface, the display being connected to a touch interaction module, the touch interaction module being used to detect a touch track input by a user, the whiteboard application interface comprising a writing graphic, the writing graphic being a track line generated based on the touch track; The controller is configured as: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track; Generate a track line of the target touch track according to the touch point; Controlling the display to display the trajectory line, and querying the path of the trajectory line; If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic; Editing is performed on the target written graphic according to the bounding box.

2. The display device according to claim 1, characterized in that The controller executes the query of the path of the trajectory line and is configured as follows: Reading touch information of the touch point, wherein the touch information includes position coordinates of the touch point and touch time; Connecting touch points adjacent in time according to the touch time to generate a track segment of the target touch track; Calculating a sub-angle of the trajectory segment; If the difference between the sub-angle and the standard angle is less than or equal to the angle threshold, marking the path includes the editing feature, and the standard angle is a preset angle; If the difference between the sub-angle and the standard angle is greater than the angle threshold, the path is marked as not including the editing feature.

3. The display device according to claim 1, characterized in that The controller executes the query of the target writing graph according to the path, and is configured to: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; If the path has at least two intersections with the first target bounding box, and the intersections are located at opposite sides of the first target bounding box, the target touch track is marked as a deletion track, and the writing graphic of the first target bounding box is marked as the target writing graphic.

4. The display device according to claim 3, characterized in that The controller performs editing on the target written graphic according to the bounding box, and is configured to: In response to the touch event of deleting the track, clearing the target writing graphic, and reading first width and height information of the first target bounding box; detecting an arrangement property of the written pattern; Acquire a replacement writing pattern of the target writing pattern according to the arrangement attribute; The replacement writing graphic is moved according to the first width and height information and the arrangement attribute.

5. The display device according to claim 1, characterized in that The controller executes the query of the target writing graph according to the path, and is further configured to: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; If there is no intersection between the path and the bounding box of the written figure, and the path is located between the second target bounding box and the third target bounding box, the target touch trajectory is marked as an insertion trajectory, and the written figure of the second target bounding box is marked as the target written figure.

6. The display device according to claim 5, characterized in that The controller performs editing on the target written graphic according to the bounding box, and is configured to: In response to the touch event of the insertion track, acquiring an insertion writing graphic, wherein the insertion writing graphic is generated based on a touch point whose touch time is later than the touch time of the termination point in the target touch track, wherein the termination point is the touch point of the lift event; Reading second width and height information of the second target bounding box; scaling the inserted writing graphic according to the second width and height information; detecting an arrangement property of the written pattern; Acquire a related writing graphic of the target writing graphic according to the arrangement attribute; The inserted writing graphic is moved to a first center position of the target writing graphic, and the target writing graphic and the associated writing graphic are moved according to the first width and height information and the arrangement attributes, wherein the first center position is the center coordinate of the second target bounding box.

7. The display device according to claim 1, characterized in that The controller is also configured to: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; Calculate the proportion of the path in the bounding box according to the relative position; If the proportion of the path in the third target bounding box is greater than or equal to the replacement threshold, the target touch track is marked as a replacement track, and the writing graphic of the third target bounding box is marked as a target writing graphic.

8. The display device according to claim 7, characterized in that The controller is also configured to: In response to the touch event of the replacement track, reading third width and height information of the third target bounding box; Scaling the track line of the target touch track according to the third width and height information; The track line of the target touch track is moved to a second center position of the target writing figure, where the second center position is the center coordinate of the third target bounding box.

9. The display device according to claim 1, characterized in that: The controller is also configured to: Check the activation status of the standard body; If the standard body is in the open state, detecting the end point of the touch track, the end point being the touch point of the lift event; Taking the end point as the timing start point, receiving the touch point within the touch time period; If no touch point is received within the touch time period, calling the standard body model, and acquiring the position coordinates of the converted touch point, the converted touch point includes a touch point whose touch time is earlier than the touch time of the end point; Inputting the position coordinates of the converted touch point into the standard volume model to generate a standard volume data object; The trajectory line of the converted touch point is rendered in the whiteboard application interface based on the standard volume data object to obtain the written graphic.

10. A method for editing a touch track, characterized in that: The method is applied to a display device, the display device includes a display and a controller, the display is configured to display a whiteboard application interface, the display is connected to a touch interaction module, the touch interaction module is used to detect a touch track input by a user, the whiteboard application interface also includes a writing graphic, and the writing graphic is a track line generated based on the touch track; the method includes: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track; Generate a track line of the target touch track according to the touch point; Controlling a display to display the trajectory line, and querying the path of the trajectory line; If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic; Editing is performed on the target written graphic according to the bounding box.

Images