Display device, and touch trajectory editing method for display device

Abstract

A display device, comprising: a display, which is configured to display a whiteboard application interface and is connected to a touch interaction module, wherein the touch interaction module is used for detecting a touch trajectory that is input by a user, the whiteboard application interface comprises a writing graphic, and the writing graphic is a trajectory line generated on the basis of the touch trajectory; a memory, which is configured to store computer instructions and / or data associated with the display device; and at least one processor, which is configured to execute the computer instructions, such that the display device executes: in response to a touch event of the user inputting a target touch trajectory, detecting touch points of the target touch trajectory; generating a trajectory line of the target touch trajectory according to the touch points; controlling the display to display the trajectory line, and querying a path of the trajectory line; if the path comprises an editing feature, querying a target writing graphic according to the path, and acquiring a bounding box of the target writing graphic, wherein the bounding box is used for representing the boundary range of the target writing graphic; and executing editing on the target writing graphic on the basis of the bounding box.

Description

Display device and touch track editing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311451740.2, filed on November 2, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a touch track editing method for the display device. Background Art

[0004] Display devices refer to terminal devices capable of outputting specific display images. These devices can include smart TVs, communication terminals, smart advertising screens, projectors, and other terminal devices. Taking smart TVs as an example, they are based on Internet application technologies, feature open operating systems and chips, and possess an open application platform. They enable two-way human-computer interaction and integrate multiple functions, including audio, video, entertainment, and data, to meet the diverse and personalized needs of users.

[0005] To meet the needs of scenarios like conferences and education, touch-enabled interactive display devices can run whiteboard applications and implement writing or drawing functions based on the touch area. Whiteboard applications can respond to user input touch traces in real time, displaying the touch traces through the application interface to form corresponding writing or drawing graphics. Furthermore, to improve the standardization of the writing process, whiteboard applications can also convert user input touch traces into standard text format based on text recognition technology and render the touch traces in standard text format.

[0006] However, since the written graphics are rendered at a fixed position on the application interface, if the user makes a mistake or misses a stroke, they will need to erase the entire written graphics and rewrite them to correct the touch track. This will cause the user to repeatedly perform erasing and rewriting operations when editing the written graphics, increasing the complexity of editing the written graphics and reducing the editing efficiency.

[0007] Summary of the Invention

[0008] According to some embodiments of the present application, a display device may include a display, a memory and at least one processor; wherein the display may be configured to display a whiteboard application interface, the display may be connected to a touch interaction module, the touch interaction module may be used to detect a touch trajectory input by a user, the whiteboard application interface may include a writing graphic, and the writing graphic may be a trajectory line generated based on the touch trajectory; the memory may be configured to store computer instructions and / or data associated with the display device; the at least one processor may be configured to execute the computer instructions to enable the display device to perform: in response to a touch event of a target touch trajectory input by a user, detect the touch point of the target touch trajectory; generate a trajectory line of the target touch trajectory according to the touch point; control the display to display the trajectory line, and query the path of the trajectory line; if the path includes an editing feature, query the target writing graphic according to the path, and obtain a bounding box of the target writing graphic, the bounding box may be used to represent the boundary range of the target writing graphic; and perform editing on the target writing graphic according to the bounding box.

[0009] According to some embodiments of the present application, a touch trajectory editing method for a display device may include a display, which may be configured to display a whiteboard application interface. The display may be connected to a touch interaction module, which may be used to detect a touch trajectory input by a user. The whiteboard application interface may also include a writing graphic, which may be a trajectory line generated based on the touch trajectory. The method may include: detecting a touch point of the target touch trajectory in response to a touch event of a target touch trajectory input by a user; generating a trajectory line of the target touch trajectory according to the touch point; controlling the display to display the trajectory line, and querying the path of the trajectory line; if the path includes an editing feature, querying the target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent the boundary range of the target writing graphic; and performing editing on the target writing graphic according to the bounding box. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application;

[0011] FIG2 is a schematic diagram of a hardware configuration of a display device according to some embodiments of the present application;

[0012] FIG3 is a schematic diagram of a hardware configuration of a control device according to some embodiments of the present application;

[0013] FIG4 is a schematic diagram of software configuration of a display device according to some embodiments of the present application;

[0014] FIG5 is a schematic diagram of a touch track according to some embodiments of the present application;

[0015] FIG6 is a schematic diagram of a whiteboard application interface according to some embodiments of the present application;

[0016] FIG7 is a schematic diagram of a whiteboard application display system according to some embodiments of the present application;

[0017] FIG8 is a schematic diagram of an acceleration layer according to some embodiments of the present application;

[0018] FIG9 is a schematic diagram of a rendering process of a trajectory line when the standard body is in an open state according to some embodiments of the present application;

[0019] FIG10 is a schematic diagram showing the rendering effect of a trajectory line when the standard body is in an open state according to some embodiments of the present application;

[0020] FIG11 is an example diagram of editing a written graphic when a writing error occurs according to some embodiments of the present application;

[0021] FIG12 is a schematic diagram showing the effect of a touch track editing method for a display device according to some embodiments of the present application;

[0022] FIG13 is a schematic diagram illustrating the principle of editing features by angle detection according to some embodiments of the present application;

[0023] FIG14 is a schematic diagram illustrating the principle of detecting editing features by distance difference according to some embodiments of the present application;

[0024] FIG15 is a schematic diagram of a process for marking a touch track according to some embodiments of the present application;

[0025] FIG16 is a schematic diagram of a process of editing a target written graphic under a deletion track according to some embodiments of the present application;

[0026] FIG17 is a schematic diagram of a process of editing a target written graphic under an insertion track according to some embodiments of the present application;

[0027] FIG18 is a schematic diagram of a process of editing a target written graphic under a replacement trajectory according to some embodiments of the present application;

[0028] FIG19 is a schematic diagram of a process of scaling a target written graphic under a replacement trajectory according to some embodiments of the present application;

[0029] FIG20 is a schematic diagram showing the effect of a replacement trajectory according to some embodiments of the present application. DETAILED DESCRIPTION

[0030] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0031] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0032] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0033] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0034] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0035] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in Figure 1, a user can operate a display device 200 through touch operation, a mobile terminal 300, and a control device 100. The control device 100 can be a remote control, a stylus, etc.

[0036] In some embodiments, the mobile terminal 300 can install software applications on the display device 200 and connect to the display device 200 through a network communication protocol to achieve one-to-one control operations and data communication. It is also possible to transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronous display.

[0037] 1 , the display device 200 also communicates data with the server 400 via various communication methods. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks.

[0038] In addition to providing a broadcast receiving television function, the display device 200 may also provide an intelligent network television function that provides computer support functions, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0039] FIG2 is a block diagram of a hardware configuration of the display device in FIG1 according to some embodiments of the present application.

[0040] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0041] In some embodiments, detector 230 can be used to collect signals from the external environment or external interactions. For example, detector 230 can include a light receiver, a sensor that can be used to collect information about ambient light intensity; or detector 230 can include an image collector, such as a camera, that can be used to collect information about the external environment, user attributes, or user interaction gestures; or detector 230 can include a sound collector, such as a microphone, that can be used to receive external sounds.

[0042] In some embodiments, the display 260 may include a display screen component for presenting images and a driving component for driving image display, which can be used to receive image signals output from at least one processor, display video content, image content, and menu control interface components, as well as user control user interface (UI) interface, etc.

[0043] In some embodiments, the display 260 may be connected to a touch interaction module, enabling the display device 200 to support touch interaction operations. The touch interaction module may be a built-in module or an external module. For example, to implement touch interaction operations, for a display device 200 with a built-in touch interaction module, the display 260 may be a touch screen that can receive touch signals input by a finger, a contact, a stylus, or the like, and execute a touch interaction response according to a pre-defined interaction strategy.

[0044] For a display device 200 with an external touch interaction module, it can connect to a terminal, device, or component with touch functionality through the device interface of the display device 200. For example, the USB or USB-C interface of the display device 200 can be connected to a touchpad. The touchpad can receive the touch track input by the user in real time and transmit the touch track to the display device 200 through the USB channel. The display device 200 can then perform an interactive response based on the received touch track in conjunction with a touch application, enabling the display device 200 to support touch interaction operations.

[0045] In some embodiments, the communication device 220 is a component that can be used to communicate with an external device or server 400 according to various communication protocol types.

[0046] In some embodiments, the at least one processor 250 may include a processor, a video processor, an audio processor, a graphics processor, a random access memory (RAM), a read-only memory (ROM), and first to nth interfaces for input / output. The at least one processor 250 may control the operation of the display device and respond to user operations through various software control programs stored in the memory. The at least one processor 250 may control the overall operation of the display device 200.

[0047] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.

[0048] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface may receive the user input command through the graphical user interface (GUI).

[0049] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0050] FIG3 is a block diagram of a hardware configuration of a control device according to some embodiments of the present application. As shown in FIG3 , the control device 100 may include a control device 110 , a communication interface 130 , a user input / output interface 140 , a memory 190 , and a power supply 180 .

[0051] The control device 100 can be configured to control the display device 200 , and can receive user input operation instructions, and can convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0052] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0053] In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform similar functions as the control device 100 after installing an application for controlling the display device 200 .

[0054] The control device 110 may include a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The control device 110 may be used to control the operation and operation of the control device 100, as well as communication and coordination between internal components and external and internal data processing functions.

[0055] Under the control of the control device 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0056] The user input / output interface 140 , wherein the input interface may include at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 , and other input interfaces.

[0057] In some embodiments, the control device 100 may include at least one of a communication interface 130 and an input / output interface 140. The control device 100 may be configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0058] The memory 190 may be used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the control apparatus 110. The memory 190 may store various control signal instructions input by the user.

[0059] The power supply 180 is used to provide operating power support for various components of the control device 100 under the control of the control device 110.

[0060] As shown in Figure 4, in some embodiments, the system can be divided into four layers, from top to bottom: the application layer (referred to as the "application layer"), the application framework layer (referred to as the "framework layer"), the Android runtime and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0061] In some embodiments, at least one application runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0062] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Applications use the API to access system resources and services during execution.

[0063] As shown in FIG4 , the application framework layer in some embodiments of the present application may include a view system, managers, content providers, etc., wherein the view system may design and implement the interface and interaction of the application, and the view system may include lists, grids, text boxes, buttons, etc. The manager may include at least one of the following modules: an activity manager may be used to interact with all activities running in the system; a location manager may be used to provide system services or applications with access to the system location service; a package manager may be used to retrieve various information related to the application packages currently installed on the device; a notification manager may be used to control the display and clearing of notification messages; and a window manager may be used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0064] In some embodiments, the activity manager can be used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager can be used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (for example, shrinking the display window, shaking the display, distorting the display, etc.).

[0065] In some embodiments, the system runtime layer can provide support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0066] In some embodiments, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer may include at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, a High Definition Multimedia Interface (HDMI) driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0067] The display device 200 can also deploy other device interfaces, hardware, and drivers based on the kernel layer and the specific display requirements of the display device 200. For example, to enrich the display types of the display device 200, the display device 200 can be provided with an HDMI interface and a USB interface. The HDMI interface can connect to devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver. The USB interface is used to connect devices with signal and data transmission functions and interactive functions, and based on the USB driver, it receives signals, transmits data, and performs specific interactive actions.

[0068] For a display device 200 that supports touch interaction operations, the display device 200 has a built-in touch interaction module, and can receive touch data input by the user through the touch interaction module. According to the interaction principle of the touch interaction module, the touch interaction module can detect relevant parameters in the touch action. For example, the display 260 of the display device 200 is provided with a capacitive touch layer. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touched position will change. At this time, the touch interaction module can record the position of the capacitance change and, in conjunction with a timer in at least one processor, record the time when the capacitance change occurs to generate touch data.

[0069] In some embodiments, the physical signal detected by the touch interaction module needs to be converted before it can be read by the display device. For example, the analog signal of the capacitance change needs to be converted into a digital signal and then transmitted to the at least one processor 250 of the display device 200. In order to adapt to the screen display process of the display device 200, the touch data detected by the touch interaction module also needs to be coordinate mapped so that the location corresponding to the touch data can correspond to the pixel point in the display 260 of the display device 200 or the user interface. For example, the coordinate position detection accuracy of the touch interaction module is 65536×65536, and the resolution of the display 260 of the display device 200 is 3840×2160. After the touch interaction module detects the touch data, the touch point position in the width and height directions can be coordinate mapped according to the ratio of 65536 / 3840 and 65536 / 2160, so that the display device 200 can perform relevant operation responses based on the mapped touch point position coordinates.

[0070] Among them, the touch data may include different operation parameters according to the user's input method. For example, as shown in Figure 5, the touch data may include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. During a touch interaction process, the touch event types that can be detected by the touch interaction module may include: a down event, a move event, and an up event of the touch action. Corresponding to the entire touch trajectory formed by the touch data, it can be the starting point [down; x1, y1, t1], the trajectory point [move; x2, y2, t2], and the end point [up; x3, y3, t3]. Each touch data records the corresponding touch event type, touch time t, and touch position (x, y), that is, the touch point of the touch trajectory may include the starting point, the trajectory point, and the end point.

[0071] It should be noted that, since the user's finger or touch device is in surface contact with the touch interaction module when performing touch interaction operations, that is, the touch interaction module can detect a contact area, which may include multiple contact points. Therefore, when the display device 200 performs interactive control, it can detect (or specify) a contact point in the contact area as a touch point. For example, when the contact area is an elliptical area formed when the user's finger contacts the touch interaction module, the display device 200 can traverse the contact point coordinates in the elliptical area to determine the extreme values ​​of the coordinates in two mutually perpendicular directions to delineate the major axis and minor axis of the elliptical area, and use the contact point at the intersection of the major axis and the minor axis as the touch point.

[0072] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, number of touch points and touch event types, so as to detect the user's touch input gesture, thereby performing different interactive responses for different input gestures. For example, when the number of touch points in the touch trajectory detected by the touch interaction module is 1, the touch event type can only include a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold. In addition, if the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 executes the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0073] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch duration, touch location, number of touch points, and touch event type, such as a double-click gesture, a slide gesture, a long press gesture, a multi-finger click gesture, and a multi-finger slide gesture. Different interaction gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, and slide trajectory shape. Different touch parameters can be used to perform different interactive controls.

[0074] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection based on the dynamic change rules of touch time, touch position, number of touch points and touch event type. For example, when the touch interaction module detects that the touch trajectory of the user input includes multiple touch points, and the touch position of each touch point is close to each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grabbing gesture. Based on complex gesture detection, the display device 200 can perform more interactive responses. For example, when the detected touch gesture is a grabbing operation for an image view, the display device 200 can respond to the grabbing gesture and perform a control action to select and move the image display position.

[0075] In some embodiments, touch gestures can also be combined with touch positions to form touch interaction operations based on specific positions. For example, when the touch interaction module detects that the down event position during the touch operation is at the top of the display 260, the touch point position corresponding to the move event moves downward. When the up event is detected, the display device 200 can determine that the current touch gesture is a swiping down from the top of the screen. Therefore, the display device 200 can control the display 260 to present a drop-down list in response to the touch gesture.

[0076] Some touch interaction gestures can be used to perform interactive actions based on the operating system of the display device 200, and are referred to as global gestures. For example, swiping down from the top of the screen on the display device 200 in the above example to bring up a drop-down list. Some touch interaction gestures can be used to perform interactive actions based on specific touch applications. For example, when the display device 200 is running a whiteboard application, a user-inputted sliding touch gesture can be used to draw a track line on the whiteboard.

[0077] Touch applications can be system applications or third-party applications developed based on the specific display device type and user needs. Touch applications can be installed on the display device 200 and launched and run under user control. For example, a user can control the display device 200 to run the whiteboard application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0078] It should be noted that a whiteboard application can be a standalone application, an associated application, or a functional module within a touch application. Different whiteboard applications can have different startup or invocation methods. For example, a user can launch a standalone whiteboard application through the application interface of the display device 200; a user can also launch a linked whiteboard application by performing a "whiteboard" operation on a conference application; and a user can control the conference application to launch the whiteboard functional module by clicking the "whiteboard" icon on the conference application.

[0079] Different forms of whiteboard applications may present different whiteboard application interface forms after startup. For example, when a user starts a whiteboard application through an associated application form, the display device 200 may automatically enter dual-screen (or split-screen) mode. In dual-screen (or split-screen) mode, the display device 200 may display the user interface through two display areas, i.e., the main screen displays the original application (such as a conference application) interface, and the secondary screen displays the whiteboard application interface.

[0080] After the whiteboard application is running, the display device 200 can be switched to displaying the whiteboard application interface. As shown in FIG6 , the whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, graphics, line shape, color, erase, clear screen, and selection. The user can click any control to draw graphics according to specific drawing needs.

[0081] In some embodiments, the display device 200 categorizes the user's touch trajectory input via the whiteboard application based on the selected state of the drawing tool control. For example, when a drawing tool control such as a pen, shape, or line is selected, the user's touch trajectory is marked as a drawing event; when the erase drawing tool is selected, the user's touch trajectory is marked as an erase event. In response to the touch trajectory associated with a drawing event or erase event, the display device 200 executes different programs according to a preset interaction strategy.

[0082] To distinguish touch tracks, the whiteboard application can include a drawing area and a menu area. The drawing area can be used to display the track line formed by the touch track and execute the draw or erase events of the touch track; the menu area can be used to display the tool controls of the above-mentioned drawing tools and execute click events of the drawing tool controls. When the user enters a touch track in the drawing area, the display device 200 draws the track line of the touch track and renders the track line to the whiteboard application interface. When the user enters a touch track in the menu area, the display device 200 modifies the selection state of the drawing tool control according to the position of the touch track.

[0083] As shown in FIG7 , the display device 200 can render and display a drawing picture in real time through an image generation (Image Producer) module, an image rendering (Surface Flinger) module, a synthesis (Composer) module, a video processing unit hardware (VPU hardware) module, and a display (screen).

[0084] During the display of the drawn image, the graphics interface (OpenGLES) of the image generation module can be connected to the whiteboard application to generate the drawn image content and pass the image content to the image rendering module. The image rendering module can be used to control the display content of the display 260, that is, to render the image content through the user interaction layer (UI layer) or the video layer according to the image content. To achieve a better display effect, the image content of the UI layer can be rendered by the GPU component, thereby forming a picture frame for display in the synthesis module. Among them, the synthesis module can store the picture frames through the frame buffer and video buffer respectively, and the video processing module can form the picture content in the corresponding layers. That is, the image frames generated during the video playback process form the picture content through the video plane, while the image frames of other image screens (such as touch tracks, graphics, etc.) form the picture content through the primary OSD plane. Finally, the display device 200 forms the final picture through the blending unit (blend) of the video processing module and displays it on the display 260.

[0085] Since the whiteboard application will respond to the user's touch operation in real time and present the user's touch trajectory, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. To this end, in some embodiments, the display device 200 can present a real-time painting effect by setting an acceleration layer. That is, in the video process module of the display device 200, an additional extended display layer (external OSD plane) can be set on the basis of the main OSD layer and the video layer. Among them, the extended display layer can be configured with different layer characteristics according to the specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the area, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0086] Taking the display device 200 running a whiteboard application as an example, to improve the real-time response speed of the line drawing process, the touch track can be drawn in real time on an extended display layer (acceleration layer) after detecting the user's touch operation. As shown in Figure 8, the acceleration layer can be displayed at a higher level than other layers. When the user completes the touch input, the track line drawn by the acceleration layer is updated to other layers. This can reduce input latency and improve the real-time display effect of the whiteboard application drawing process.

[0087] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer, maintaining real-time response speed. After the user inputs the touch interaction action, that is, when the display device 200 detects the end point of the lift event, the content displayed by the acceleration layer can be released directly, and the OSD layer directly forms the drawn graphics, thereby eliminating the need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0088] Based on the above-described display device 200, in some embodiments, a user can input touch traces of various characters on the display device 200, causing the display device 200 to display corresponding handwritten graphics. Specifically, the user can write corresponding touch traces according to the strokes of each character on the display device 200. The display device 200 then maps the touch traces into track lines according to a preset mapping relationship and renders the track lines to the OSD layer, allowing the display 260 to display the track lines. The track lines of each stroke in the character can form a complete handwritten graphic.

[0089] As shown in FIG9 , to improve the standardization of written graphics, the display device 200 can also convert the trajectory of the written graphics into a standard body. Specifically, in some embodiments, the display device 200 further detects the activation status of the standard body ( S901 ). The activation status of the standard body is determined ( S902 ). If the standard body is enabled, the display device 200 detects the end point of the touch trajectory ( S903 ). The end point may be the touch point of an up event. A touch point is received within the touch event period, starting with the end point. If no touch point is received within the touch time period ( S904 ), the standard body model is invoked to obtain the position coordinates of the converted touch point. The converted touch point may include a touch point with a touch time earlier than the touch time of the end point. The position coordinates of the converted touch point are then input into the standard body model to generate a standard body data object ( S905 ). Based on the standard body data object, the trajectory of the converted touch point is rendered in the whiteboard application interface ( S906 ) to obtain the written graphics.

[0090] That is to say, the whiteboard application of the display device 200 can be set with a conversion function for standard fonts. Among them, the standard fonts can include a variety of different fonts. When the standard font function is in the enabled state, the display device 200 can detect the end point of the touch trajectory, and when the end point of a certain touch trajectory is detected and no new touch points are detected within a preset time period, the original touch data of the touch points that have not been processed before the end point is input into the trained standard font model. The standard font model will convert these touch points into corresponding standard font coordinate points and output the converted standard font coordinate points to generate a new standard font data object. Among them, the standard font data object and the touch data of the touch points can be two different data objects, which can be stored separately in the display device 200, such as stored in the memory or file of the display device 200. After generating the standard font data object, the display device 200 renders the trajectory line of the touch trajectory based on the standard font data object, so that the whiteboard application interface can present the writing graphics in standard fonts. On the contrary, if the standard font function is in the disabled state, the trajectory line is directly rendered according to the touch data of the touch points (S907).

[0091] For example, the preset touch time period is 3 seconds. When the display device 200 detects the end point a of a certain touch trajectory and no new touch points are detected within  3 seconds, all the touch points whose touch time is earlier than the touch time of point a and have not been processed are input into the standard font model, so that the standard font model outputs a standard font data object. The display device 200 then renders a more regular writing graphic in the whiteboard application interface, as shown in Figure 10.

[0092] However, since the writing graphic is rendered at a fixed position on the OSD layer, if the user wants to modify the writing graphic, all relevant and already rendered writing graphics need to be erased, and then the touch trajectory of the erased part needs to be input again. For example, as shown in Figure 11, when the user writes the writing graphic of "Good morning" on the display device 200, it is miswritten as "Good early". At this time, the user needs to erase the writing graphic of "early" and then rewrite the writing graphics of "up" and "early". In this way, when the user edits the writing graphic, the operations of erasing and rewriting need to be repeatedly performed, resulting in an increase in the complexity of the writing graphic editing process, thereby reducing the editing efficiency of the writing graphic.

[0093] Based on the above application scenarios, in order to improve the problem of low efficiency in editing written graphics, a display device according to some embodiments of the present application may include a display, a memory, and at least one processor 250. Specifically, the display 260 may be configured to display a whiteboard application interface; the display 260 may be connected to a touch interaction module, which may be used to detect a touch trajectory input by a user; the whiteboard application interface may include written graphics, which may be trajectory lines generated based on the touch trajectory; the memory may be configured to store computer instructions and / or data associated with the display device; as shown in FIG12 , the at least one processor 250 may be configured to execute the aforementioned computer instructions to cause the display device to perform some or all of the following program steps:

[0094] S100: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track.

[0095] After detecting a touch event of a target touch track input by a user, the display device 200 detects the touch point of the target touch track in response to the touch event. The touch track may include at least one touch point, which may include the starting point, track point, and end point described in the above embodiment.

[0096] In some embodiments, when the display device 200 detects the endpoint of a touch track, it also generates a bounding box for the touch track based on all touch points in the touch track. The bounding box can be used to represent the boundary range of the touch track, that is, the closed area formed by the bounding box can cover all touch points in the touch track. For example, the bounding box can be a rectangular box. The display device 200 can use the bounding box of the touch track to determine the location of the area where the touch track is located.

[0097] It should be noted that the boundary box of the touch track may be attribute data stored when the display device 200 stores the touch track, and will not be displayed in the whiteboard application interface.

[0098] To facilitate recognition of handwritten patterns, in some embodiments, the display device 200 may further group multiple touch traces that constitute a standard handwritten pattern based on the standard handwritten pattern, and use the union of the bounding boxes corresponding to the touch traces in the same group as the bounding box of the handwritten pattern. The display device 200 may use the bounding box of the handwritten pattern to determine the location of the region where the handwritten pattern is located.

[0099] Similarly, for written graphics that have not been converted into standard script, in some embodiments, the display device 200 can identify the touch track based on optical character recognition (OCR) technology, mark the touch tracks that constitute a font as a group, and use the union of the corresponding bounding boxes of the touch tracks in the same group as the bounding box of a written graphic.

[0100] S200: Generate a target touch track according to the touch point.

[0101] After detecting the touch points of the target touch track, the display device 200 can generate the track line of the touch track in real time according to each touch point, that is, the display device 200 can map the touch points to corresponding bitmap data, and refresh the bitmap data to the OSD layer in real time, thereby forming the track line of the target touch track.

[0102] In some embodiments, when generating a trajectory, the display device 200 may also obtain touch color information of the target touch trajectory. The touch color information may be color information selected by the user based on the whiteboard application interface, or it may be default color information in the display device 200. The touch color information may include color values. When generating bitmap data, the display device needs to set the pixel points to the color values ​​in the touch color information.

[0103] Because the display device may enable the acceleration layer to achieve a hand-following effect in some scenarios, in some embodiments, the display device 200 also detects the activation status of the acceleration layer. If the display device 200 enables the acceleration layer, the display device 200 may refresh the bitmap data to the acceleration layer in real time according to the touch points of the target touch track, and upon detecting the end point of the touch track, refresh the bitmap data of the entire target touch track to the OSD layer. If the display device 200 does not enable the acceleration layer, the display device 200 may refresh the bitmap data to the OSD layer in real time according to the touch points of the target touch track.

[0104] S300: Control the display to display the trajectory line, and query the path of the trajectory line.

[0105] After generating the target touch track, the display device 200 can control the display 260 to generate the target touch track, query the path of the track in the whiteboard application interface, and determine the relative position of the target touch track and each written graphic through the path.

[0106] By querying the path of the trajectory line, the display device 200 can identify the characteristics of the path. For example, the editing of a written graphic can be triggered by a path that is approximately a straight line. Therefore, in some embodiments, the display device 200 can detect whether the path of the trajectory line meets the characteristics of an approximate straight line by angle. That is, when querying the path of the trajectory line, the display device 200 can read the touch information of the touch point. The touch information may include the position coordinates of the touch point and the touch time. The touch points that are temporally adjacent can then be connected according to the touch time to generate a trajectory segment of the target touch trajectory. The sub-angle of the trajectory segment is calculated. If the difference between the sub-angle and the standard angle is less than or equal to the angle threshold, the path is marked as including the editing feature; if the difference between the sub-angle and the standard angle is greater than the angle threshold, the path is marked as not including the editing feature. The standard angle is a preset angle that can be set according to the characteristics of the straight line.

[0107] For example, the standard angle is 180°. When querying the path of the target touch trajectory, the display device 200 can read the touch information of all touch points of the target touch trajectory, which may include touch time and touch coordinates. Then, the touch points adjacent in time are connected to generate trajectory segments. As shown in Figure 13, every three adjacent touch points on the path form the line shown in Figure 13, and there is a sub-angle α between the trajectory segments. If the difference between the sub-angle α of all trajectory segments in the target touch trajectory and 180° is less than or equal to the angle threshold, the marked path includes the editing feature; if there is a sub-angle α in the target touch trajectory and the difference between it and 180° is greater than the angle threshold, the marked path does not include the editing feature.

[0108] Alternatively, in some embodiments, the display device 200 can also detect whether the path of the trajectory line satisfies the characteristics of an approximate straight line through the distance difference, that is, every two adjacent touch points on the path can constitute a trajectory segment, and the length of the trajectory segment is the distance between the two touch points. Calculate the sum of the lengths of all trajectory segments in the target touch trajectory as the judgment length value. At the same time, calculate the distance between the starting point and the end point in the target touch trajectory as the reference length value. Then calculate the difference between the above-mentioned judgment length value and the reference length value. If the difference between the judgment length value and the reference length value is less than or equal to a fixed ratio of the reference length value, such as 1 / 10, 1 / 15, etc., the marked path includes editing features; if the difference between the judgment length value and the reference length value is greater than the fixed ratio of the reference length value, the marked path does not include editing features.

[0109] For example, as shown in Figure 14, P0 and P2 are the starting and ending points of the target touch trajectory, and P1 is the trajectory point of the target touch trajectory. The sum of the trajectory segments P0-P1 and P1-P2 is d1, and the distance between touch points P0-P2 is d2. Therefore, the judgment length is d1 and the reference length is d2. If d1-d2 ≤ (1 / 10)*d2, the marked path includes editing features; if d1-d2 > (1 / 10)*d2, the marked path does not include editing features.

[0110] It is understandable that the above-mentioned approximate straight line method is only one way of implementing the present application, and the path editing features can also adopt more different features. In this regard, the present application does not impose any limitation.

[0111] S400: If the path includes an editing feature, query the target writing graphic according to the path, and obtain a bounding box of the target writing graphic.

[0112] After the display device 200 finds that the path of the track line includes an editing feature, it can search for a target written shape according to the path and obtain a bounding box of the target written shape. The target written shape can be a written shape to be edited by the target touch track, and the bounding box can be used to represent the boundary range of the target written shape. For example, if the target touch track is a deletion track and the display device 200 needs to delete the written shape, it can search for a written shape that intersects with the path as the target written shape.

[0113] The bounding box of a written shape can be generated based on the data object that forms its trajectory. For example, if the target written shape being edited is non-standard, the display device 200 can query the bounding box of the written shape based on the data object of the original touch data. If the target written shape being edited is standard, the display device 200 can query the bounding box of the writing trajectory based on the standard data object.

[0114] S500: Editing the target written graphic according to the bounding box.

[0115] After the display device 200 finds the target handwritten graphic that needs to be edited, it can edit the target handwritten graphic according to the bounding box of the target handwritten graphic, such as deleting, inserting, or replacing it. The display device 200 provided in the embodiment of the present application can trigger the editing of the target handwritten graphic through the target touch track including the editing feature, which can improve the problem of erasing a large amount of handwritten graphics when editing handwritten graphics, thereby improving the editing efficiency of handwritten graphics.

[0116] To achieve editing of different functions, as shown in FIG. 15, in some embodiments, the display device 200 may also obtain the bounding box of the written graph (S1501), and may detect the relative position between the detection path and the written graph according to the bounding box (S1502). If there are at least two intersections between the path and the first target bounding box, and the intersections are located on the opposite sides of the first target bounding box (S1503), that is, when the path of the trajectory line penetrates through two opposite sides of the first target bounding box, the display device 200 may mark the target touch trajectory as a deletion trajectory (S1504), and may mark the written graph of the first target bounding box as the target written graph. For example, when the editing feature is an approximate straight line feature, if the display device 200 detects that the path of the trajectory line includes an approximate straight line feature and the path penetrates through two opposite sides of a certain bounding box, the display device 200 may mark the target touch trajectory as a deletion trajectory and may mark the written graph of the bounding box as the target written graph.

[0117] Since the written graph is rendered at a fixed position in the whiteboard application interface, when the user has the problem of overwriting, if the redundant written graph is erased, there will be some blank areas in the whiteboard application interface, making the layout of the remaining written graphs irregular and affecting the display effect of the written graphs.

[0118] Therefore, to improve the above layout problem, for the deletion trajectory, in some embodiments, in response to the touch event of the deletion trajectory, the display device 200 may clear the target written graph, and may read the first width and height information of the first target bounding box. Detect the arrangement attribute of the written graph, and may obtain the supplementary written graph of the target written graph according to the arrangement attribute. Then, the supplementary written graph may be moved according to the first width and height information and the arrangement attribute. The first width and height information may be the horizontal and vertical ranges (bounds) of the first target bounding box; the arrangement attribute may be the attribute information when the written graphs are arranged, which may be arranged horizontally or vertically, and the arrangement attribute may also include the position attributes of the front and rear positions.

[0119] For example, as shown in FIG. 16, the written graphs "you", "hao", and "ya" shown in FIG. 16 are arranged horizontally, and the editing feature is an approximate straight line feature. The user inputs a touch trajectory that penetrates through "hao" in the whiteboard application interface. The display device 200 may then respond to the touch trajectory, may render the trajectory line of the touch trajectory in real time, and may mark it as a deletion trajectory according to the path of the trajectory line. Then, in response to the touch event of the deletion trajectory, it may clear the written graph of "hao", and may detect the width and height information of the bounding box of "hao", which may include the width B1 and the height H1. When it is detected that the arrangement attribute is horizontal arrangement, the supplementary written graph "ya" located behind "hao" may be moved forward by the length of B1.

[0120] Arrangement attributes can be determined based on the time a written shape was input. That is, in some embodiments, when the display device 200 detects a touch track used to generate a written shape, it can also mark the horizontal or vertical arrangement attributes according to the time the touch track was input, and can also mark the front and back positions of the horizontal or vertical arrangement according to the time the touch track was input. For example, when the display device 200 detects a second written shape input by the user, it can obtain the position of the second written shape and the first written shape. If the second written shape and the first written shape are on the same horizontal line, the arrangement attribute can be marked as horizontal arrangement, and based on their positions on the horizontal line, the side closer to the first written shape can be marked as the front position.

[0121] As shown in Figure 15, in some embodiments, the display device 200 can obtain the bounding box of the written graphic and detect the relative position of the path and the written graphic based on the bounding box. If there is no intersection between the path and the bounding box of the written graphic, and the path is located between the second target bounding box and the third target bounding box (S1505), the target touch track can be marked as an insertion track (S1506), and the written graphic of the second target bounding box can be marked as the target written graphic. For example, when the editing feature is an approximate straight line feature, if the display device 200 detects that the path of the trajectory line includes an approximate straight line feature and the path is located between the bounding boxes of two written graphics, the display device 200 can mark the target touch track as an insertion track and mark the written graphic located in the latter position of the two written graphics as the target written graphic.

[0122] With respect to the insertion trajectory, in some embodiments, the display device 200 can obtain an inserted writing graphic in response to a touch event of the insertion trajectory. The inserted writing graphic can be generated based on a touch point whose touch time is later than the touch time of the end point in the target touch trajectory, and the end point can be a touch point of a lift event. The second width and height information of the second target bounding box is then read, and the inserted writing graphic can be scaled according to the second width and height information. The arrangement properties of the writing graphic are detected, and the associated writing graphic of the target writing graphic can be obtained according to the arrangement properties. The inserted writing graphic can be moved to the first center position of the target writing graphic, and the target writing graphic can be moved according to the first width and height information and the arrangement properties. The first center position can be the center coordinates of the second target bounding box.

[0123] For example, as shown in FIG. 17, the written characters "Ni", "Hao", and "Ya" shown in FIG. 17 are arranged horizontally, and the editing feature is a feature approximated by a straight line. When the user inputs a touch trajectory between "Ni" and "Hao" in the whiteboard application interface, the display device 200 responds to the touch trajectory, can render the trajectory line of the touch trajectory in real time, and can mark it as an insertion trajectory according to the path of the trajectory line. Then, in response to the touch event of the insertion trajectory, the inserted written character "Men" can be obtained, and the line height information of the bounding box of "Hao" and the width information B1 of the width of "Men" after equal-proportion scaling can be detected. When it is detected that the arrangement attribute is horizontal arrangement, "Men" can be moved to the position of "Hao", and the written character "Hao" located behind "Hao" and the associated written character "Ya" of "Hao" can be moved backward by the length of B1.

[0124] In some embodiments, the display device 200 can also obtain the bounding box of the written character, detect the relative position between the path and the written character according to the bounding box, and then calculate the proportion of the path in the bounding box according to the relative position. If the proportion of the path in the third target bounding box is greater than or equal to the replacement threshold, the target touch trajectory can be marked as a replacement trajectory, and the written character in the third target bounding box can be marked as the target written character. For example, when the replacement threshold is 75%, when the display device 200 detects that the proportion of the path in the bounding box of a certain written character is greater than or equal to 75%, the touch trajectory of the path can be marked as a replacement trajectory.

[0125] For the replacement trajectory, in some embodiments, in response to the touch event of the replacement trajectory, the display device 200 can read the third width and height information of the third target bounding box, and scale the trajectory line of the target touch trajectory according to the third width and height information. Then, move the trajectory line of the target touch trajectory to the second center position of the target written character. The second center position can be the center coordinates of the third target bounding box.

[0126] For example, as shown in FIG. 18, the written characters "Ni", "Hao", and "Ya" shown in FIG. 18. When the user inputs a touch trajectory above "Ya" in the whiteboard application interface, the display device 200 responds to the touch trajectory, can render the trajectory line of the touch trajectory in real time, form the font graphic of "Ma", and can mark the touch trajectory as a replacement trajectory. The display device 200 then responds to the touch event of the replacement trajectory, can detect the width and height information of the bounding box of "Ya", which can include the width B1 and the height H1. Then, scale the trajectory line of the replacement trajectory by the height H1, and move the trajectory line of "Ma" to the center position of the bounding box of "Ya".

[0127] In some embodiments, the replacement trajectory can also be a touch trajectory that includes specific editing features, such as a touch trajectory whose path can be enclosed into a circular area, etc. The user can draw a touch trajectory that can be enclosed into a circular area around the periphery of the writing graphic that needs to be replaced, and draw a new writing graphic within the circular area to replace the new writing graphic with the circular area.

[0128] It is understandable that the replacement track can also be determined based on the time when the user draws the writing graphic, etc. This application does not impose any limitation on this.

[0129] Since the user may also input punctuation marks when inserting or replacing a writing graphic, such as a comma, period, question mark, quotation mark, or exclamation mark, different symbols need to be mapped to different positions. Therefore, in some embodiments, if the target touch track is an insert track or a replace track, the display device 200 may perform pre-conversion on the updated writing graphic through a standard body model. The updated writing graphic may be a newly drawn writing graphic by the user in response to an insert track or a replace track; the pre-conversion may be to identify the insert writing graphic or the replace track in the database through a standard body model, but not draw the converted track line to the whiteboard application interface. If the updated writing graphic is a punctuation mark writing graphic, the position attribute of the punctuation mark may be queried, and the updated writing graphic may be mapped to different positions according to the position attribute. For example, a comma may be mapped to the lower right corner of the target writing graphic boundary box, etc.

[0130] In some embodiments, the insertion track may also be a touch track including specific editing features, such as a touch track including a preset angle range, a touch track including a path that encloses a closed area, etc. After identifying the editing features of the insertion track, the display device 200 may mark the touch track as an insertion track.

[0131] In one embodiment, when the display device 200 detects the insertion trajectory, it also needs to obtain the touch trajectory of the inserted writing graphic. Therefore, after detecting the end point of the insertion trajectory, the display device 200 can detect touch points whose touch time is later than the end point of the insertion trajectory, and can generate a trajectory line of the inserted writing graphic based on the touch points whose touch time is later than the insertion trajectory. For example, after detecting the end point of the insertion trajectory, the display device 200 can collect the touch points of the touch trajectory; if no new touch points are detected within a certain touch cycle, the collection of touch points can be terminated, and a trajectory line can be generated based on the touch trajectory composed of the collected touch points to serve as the inserted writing graphic.

[0132] It is understandable that the above-mentioned editing features can be any features of approximate straight lines, approximate angles or approximate figures, and different strategies can be adopted for querying target written figures according to different editing features. This application does not impose any restrictions on this.

[0133] In addition, since the writing graphics of some non-standard fonts are composed of multiple touch trajectories, there may be no concept of the overall bounding box of the writing graphics. Therefore, in some embodiments, the display device 200 can detect the rendering position of the writing graphics in the whiteboard application interface, and can detect the distance between the writing graphics through the rendering position, and can generate the width and height information of each row and / or each column of writing graphics based on the distance.

[0134] For example, as shown in FIG. 19, the inserted writing graphic is "Li Si", and its height is insertBounds; the height of the row where "Zhang San" is located can be obtained as toBounds by detecting the rendering position of "Zhang San", and "Zhang San" can be cleared. The height of insertBounds can be set to the same height as toBounds, the writing graphic "Li Si" can be scaled proportionally, and it can be moved to the position of "Zhang San".

[0135] Alternatively, in some embodiments, the display device 200 can also respond to an insertion trajectory whose path can enclose a closed area, detect the width and height information of the closed area, and scale the inserted writing graphic based on the width and height information of the closed area. For example, as shown in FIG. 20, the editing feature of the insertion trajectory is a closed rectangle, and the display device 200 can scale the inserted writing graphic "Li Si" according to the rectangle, clear "Zhang San" within the rectangle, and move the scaled "Li Si" to the position of "Zhang San".

[0136] However, since the writing graphics that have not been converted to the standard font may have situations such as connected strokes and small spacing, it is difficult to input an insertion trajectory that does not intersect with the bounding box of the writing graphics. To enable the normal input of the insertion trajectory, in some embodiments, if the path includes an editing feature, the row height can be obtained from the width and height information of each row of writing graphics. When the length of the path is greater than the row height of the writing graphics, the relative position between the path and the writing graphics can be detected. If the path has at least two intersections with a certain bounding box, and the two intersections are located on opposite sides of the bounding box, the target touch trajectory can be marked as a deletion trajectory; if the path has an intersection with a certain bounding box, but the intersection is not located on opposite sides of the bounding box, the target touch trajectory can be marked as an insertion trajectory.

[0137] That is to say, the display device 200 can also determine whether the target touch trajectory includes an editing feature based on the length of the path. Only when the length of the path is greater than the row height of the writing graphics, the display device 200 marks that the path includes an editing feature. In this way, the display device 200 can accurately edit the writing graphics that have not been converted to the standard font, thereby improving the user's interaction experience.

[0138] It can be understood that when the editing features are different, the display device 200 can mark the target touch trajectory through different judgment logics. This application does not limit this.

[0139] Moreover, due to the characteristics of text, the inserted writing graphics may include different width-to-height ratios. For example, the width of writing graphics such as "one" is much greater than its height. At this time, if scaling is performed according to the length of the bounding box, the scaled writing graphics may have inconsistent problems. Therefore, in some embodiments, the display device 200 can detect the bounding box of the inserted writing graphics and obtain the line height from the width and height information of each line of writing graphics. Calculate the height of the inserted writing graphics according to the bounding box of the inserted writing graphics. If the height of the inserted writing graphics is less than the line height, no scaling is performed on the inserted writing graphics. To improve the user interaction experience, if the target touch trajectory includes an editing feature, after the display device 200 performs editing on the target writing graphics, it can release the trajectory line of the target touch trajectory, making it disappear in the whiteboard application interface; if the target touch trajectory does not include an editing feature, the display device 200 can continue to maintain the trajectory line of the target touch trajectory, making it normally displayed in the whiteboard application interface.

[0140] Based on the above display device, a touch trajectory editing method for a display device according to some embodiments of the present application is applied to the display device 200 provided in the above embodiments. The display device 200 may include a display 260. Among them, the display 260 may be configured to display a whiteboard application interface. The display 260 may be connected to a touch interaction module, and the touch interaction module may be used to detect the touch trajectory input by the user. The whiteboard application interface may further include writing graphics, and the writing graphics may be a trajectory line generated based on the touch trajectory. As shown in FIG. 12, the method may include but is not limited to the following program steps S100 to S500, which will not be elaborated here.

[0141] As can be seen from the above solutions, according to the display device and the touch trajectory editing method for the display device according to some embodiments of the present application, the method can detect the touch points of the target touch trajectory in response to a touch event of the user inputting the target touch trajectory. A trajectory line of the target touch trajectory can be generated according to the touch points, and the display of the trajectory line can be controlled on the display, and the path of the trajectory line can be queried. If the path includes an editing feature, the target writing graphics can be queried according to the path, and the bounding box of the target writing graphics can be obtained. The bounding box can be used to represent the boundary range of the target writing graphics. Then, the target writing graphics are edited according to the bounding box. The method can directly edit the writing graphics according to the target touch trajectory including the editing feature and the path of the target touch trajectory, without the user erasing the existing writing graphics, thereby improving the problem that the writing graphics are not convenient to edit.

[0142] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be repeated here.

[0143] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the solution in the embodiments of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and can include a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application or certain parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding solutions from the scope of the embodiments of the present application.

Claims

1. A display device, comprising: A display configured to display a whiteboard application interface, the display being connected to a touch interaction module, the touch interaction module being used to detect a touch track input by a user, the whiteboard application interface comprising a writing graphic, the writing graphic being a track line generated based on the touch track; a memory configured to store computer instructions and / or data associated with the display device; At least one processor is configured to execute the computer instructions to cause the display device to perform: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track; Generate a track line of the target touch track according to the touch point; Controlling the display to display the trajectory line, and querying the path of the trajectory line; If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic; Editing is performed on the target written graphic according to the bounding box.

2. The display device according to claim 1, wherein the at least one processor executes querying the path of the trajectory line, and is specifically configured to execute the computer instruction to enable the display device to execute: Reading touch information of the touch point, wherein the touch information includes position coordinates of the touch point and touch time; Connecting touch points adjacent in time according to the touch time to generate a track segment of the target touch track; Calculating a sub-angle of the trajectory segment; If the difference between the sub-angle and the standard angle is less than or equal to the angle threshold, marking the path includes the editing feature, and the standard angle is a preset angle; If the difference between the sub-angle and the standard angle is greater than the angle threshold, the path is marked as not including the editing feature.

3. The display device according to claim 1, wherein the at least one processor executes the query of the target writing graph according to the path, and is specifically configured to execute the computer instructions to enable the display device to execute: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; If the path has at least two intersections with the first target bounding box, and the intersections are located at opposite sides of the first target bounding box, the target touch track is marked as a deletion track, and the writing graphic of the first target bounding box is marked as the target writing graphic.

4. The display device according to claim 3, wherein the at least one processor performs editing on the target written graphic according to the bounding box, and is specifically configured to execute the computer instructions to enable the display device to perform: In response to the touch event of deleting the track, clearing the target writing graphic, and reading first width and height information of the first target bounding box; detecting an arrangement property of the written pattern; Acquire a replacement writing pattern of the target writing pattern according to the arrangement attribute; The replacement writing graphic is moved according to the first width and height information and the arrangement attribute.

5. The display device according to claim 1, wherein the at least one processor executes the query of the target writing graph according to the path, and is further configured to execute the computer instructions to enable the display device to execute: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; If there is no intersection between the path and the bounding box of the written figure, and the path is located between the second target bounding box and the third target bounding box, the target touch trajectory is marked as an insertion trajectory, and the written figure of the second target bounding box is marked as the target written figure.

6. The display device according to claim 5, wherein the at least one processor performs editing on the target written graphic according to the bounding box, and is specifically configured to execute the computer instructions to enable the display device to perform: In response to the touch event of the insertion track, acquiring an insertion writing graphic, wherein the insertion writing graphic is generated based on a touch point whose touch time is later than the touch time of the termination point in the target touch track, wherein the termination point is the touch point of the lift event; Reading second width and height information of the second target bounding box; scaling the inserted writing graphic according to the second width and height information; detecting an arrangement property of the written pattern; Acquire a related writing graphic of the target writing graphic according to the arrangement attribute; The inserted writing graphic is moved to a first center position of the target writing graphic, and the target writing graphic and the associated writing graphic are moved according to the first width and height information and the arrangement attributes, wherein the first center position is the center coordinate of the second target bounding box.

7. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Obtaining a bounding box of the written graphic; detecting a relative position between the path and the written figure according to the bounding box; Calculate the proportion of the path in the bounding box according to the relative position; If the proportion of the path in the third target bounding box is greater than or equal to the replacement threshold, the target touch track is marked as a replacement track, and the writing graphic of the third target bounding box is marked as a target writing graphic.

8. The display device according to claim 7, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to the touch event of the replacement track, reading third width and height information of the third target bounding box; Scaling the track line of the target touch track according to the third width and height information; The track line of the target touch track is moved to a second center position of the target writing figure, where the second center position is the center coordinate of the third target bounding box.

9. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Check the activation status of the standard body; If the standard body is in the open state, detecting the end point of the touch track, the end point being the touch point of the lift event; Taking the end point as the timing start point, receiving the touch point within the touch time period; If no touch point is received within the touch time period, calling the standard body model, and acquiring the position coordinates of the converted touch point, the converted touch point includes a touch point whose touch time is earlier than the touch time of the end point; Inputting the position coordinates of the converted touch point into the standard volume model to generate a standard volume data object; The trajectory line of the converted touch point is rendered in the whiteboard application interface based on the standard volume data object to obtain the written graphic.

10. A touch track editing method for a display device, the display device comprising a display, the display being configured to display a whiteboard application interface, the display being connected to a touch interaction module, the touch interaction module being used to detect a touch track input by a user, the whiteboard application interface further comprising a writing graphic, the writing graphic being a track line generated based on the touch track; the method comprising: In response to a touch event of a target touch track input by a user, detecting a touch point of the target touch track; Generate a track line of the target touch track according to the touch point; Controlling a display to display the trajectory line, and querying the path of the trajectory line; If the path includes an editing feature, querying a target writing graphic according to the path, and obtaining a bounding box of the target writing graphic, wherein the bounding box is used to represent a boundary range of the target writing graphic; Editing is performed on the target written graphic according to the bounding box.

Images