Display device and display method of written graph

Abstract

The embodiment of the invention provides a display device and a display method of a written graph, and the method can respond to a touch event that a user inputs a target touch track, obtain a bounding box of the target touch track, and recognize a mapping word of the target touch track. When the display area of the bounding box is a blank area, generating a first average width according to the bounding box and the number of the mapping words; if the difference between the first average width and the reference width is greater than a difference threshold, generating a target writing graph according to the first average width; and if the difference value is smaller than or equal to the difference threshold value, generating the target writing graph according to the reference width. Wherein the reference width is the width of the written graph with the latest rendering time. And displaying the target writing graph according to the starting point of the target touch track. According to the method, the position and the size of the to-be-displayed writing graph can be automatically set by comparing the size of the displayed writing graph with the average size of each mapping character in the touch track, and the process of manually adjusting the writing graph by a user is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for displaying written graphics. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] In order to meet the needs of scenarios such as meetings and education, for display devices that support touch interactive operations, you can run a whiteboard application and implement writing or drawing functions based on the touch area of ​​the whiteboard application. The whiteboard application can respond to the touch track input by the user in real time, and display the touch track through the application interface to form a corresponding writing graphic or drawing graphic. In addition, in order to improve the standardization of the writing process, the whiteboard application can also convert the touch track input by the user into a standard body based on text recognition technology, and render the writing graphic in the form of a standard body.

[0004] However, when the display device converts the standard body of the written graphics, the display position of the written graphics is random, and the size of the written graphics is also proportionally scaled according to the range of the touch track. Therefore, written graphics with messy positions and non-uniform sizes may appear in the application interface of the whiteboard application. If you want to adjust the position and size of the written graphics, you need to manually adjust the position and size of each written graphics, which increases the complexity of the touch interaction operation and reduces the user experience. Summary of the invention

[0005] The present application provides a display device and a method for displaying written graphics to solve the problem of high complexity of touch interaction operations when the display device displays written graphics.

[0006] In a first aspect, some embodiments of the present application provide a display device, including a display and a controller. The display is configured to display a whiteboard application interface, the whiteboard application interface includes a writing area, the display is connected to a touch interaction module, the touch interaction module is used to detect a touch track input by a user, the writing area is used to display a writing graphic, and the writing graphic is a track line generated based on the touch track; the controller is configured to execute the following program steps:

[0007] In response to a touch event of a target touch track input by a user, a bounding box of the target touch track is obtained, and a mapping word of the target touch track is identified, wherein the bounding box is used to represent a boundary range of the target touch track, and the mapping word is a text identified based on the target touch track;

[0008] Detecting a target area in the writing area, the target area being a display area of ​​the bounding box;

[0009] When the target area is a blank area, generating a first average width according to the number of the bounding box and the mapping words;

[0010] Calculate the difference between the first average width and a reference width, where the reference width is the width of the writing graphic with the latest rendering time in the writing area;

[0011] If the difference is greater than the difference threshold, a target writing pattern is generated according to the first average width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch trajectory, where the starting point is the touch point of the drop event in the target touch trajectory, and the target writing pattern is a trajectory line generated based on the target touch trajectory;

[0012] If the difference is less than or equal to the difference threshold, the target writing pattern is generated according to the reference width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch track.

[0013] In a second aspect, some embodiments of the present application further provide a method for displaying a written graphic, which is applied to the display device provided in the first aspect, wherein the display device includes a display and a controller, wherein the display is configured to display a whiteboard application interface, wherein the whiteboard application interface includes a writing area, wherein the display is connected to a touch interaction module, wherein the touch interaction module is used to detect a touch track input by a user, wherein the writing area is used to display a written graphic, wherein the written graphic is a track line generated based on the touch track; wherein the method includes:

[0014] In response to a touch event of a target touch track input by a user, a bounding box of the target touch track is obtained, and a mapping word of the target touch track is identified, wherein the bounding box is used to represent a boundary range of the target touch track, and the mapping word is a text identified based on the target touch track;

[0015] Detecting a target area in the writing area, the target area being a display area of ​​the bounding box;

[0016] When the target area is a blank area, generating a first average width according to the number of the bounding box and the mapping words;

[0017] Calculate the difference between the first average width and a reference width, where the reference width is the width of the writing graphic with the latest rendering time in the writing area;

[0018] If the difference is greater than the difference threshold, a target writing pattern is generated according to the first average width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch trajectory, where the starting point is the touch point of the drop event in the target touch trajectory, and the target writing pattern is a trajectory line generated based on the target touch trajectory;

[0019] If the difference is less than or equal to the difference threshold, the target writing pattern is generated according to the reference width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch track.

[0020] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a method for displaying a written graphic, and the method can respond to a touch event of a user inputting a target touch track, obtain a bounding box of the target touch track, and identify the mapping word of the target touch track. Detect the target area of ​​the bounding box in the writing area, and when the target area is a blank area, generate a first average width according to the number of bounding boxes and mapping words. If the difference between the first average width and the reference width is greater than the difference threshold, generate the target writing graphic according to the first average width, and display the target writing graphic in the writing area according to the starting point of the target touch track; if the difference is less than or equal to the difference threshold, generate the target writing graphic according to the reference width, and display the target writing graphic in the writing area according to the starting point of the target touch track. Among them, the reference width is the width of the writing graphic with the latest rendering time. The method can automatically set the position and size of the writing graphic to be displayed by comparing the size of the displayed writing graphic and the average size of each mapping word in the touch track, thereby reducing the process of manual adjustment of the writing graphic by the user and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0023] Figure 2A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0024] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0025] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0026] Figure 5 A schematic diagram of touch tracks provided in some embodiments of the present application;

[0027] Figure 6 A schematic diagram of a whiteboard application interface provided in some embodiments of the present application;

[0028] Figure 7 A schematic diagram of a whiteboard application display system provided in some embodiments of the present application;

[0029] Figure 8 A schematic diagram of an acceleration layer provided for some embodiments of the present application;

[0030] Fig. 9 A schematic diagram of the rendering process of the trajectory line when the standard body provided in some embodiments of the present application is in the open state;

[0031] Fig.10 A schematic diagram of the rendering effect of the trajectory line when the standard body provided in some embodiments of the present application is in the open state;

[0032] Fig.11 This is an example diagram of the effect of a messy layout of written graphics provided in some embodiments of the present application;

[0033] Fig.12 A schematic diagram of a flow chart of a method for displaying a written graphic provided in some embodiments of the present application;

[0034] Fig.13 A schematic diagram of a process for detecting a target area provided in some embodiments of the present application;

[0035] Fig.14 A display effect diagram of a written graphic when the difference value provided in some embodiments of the present application is greater than a difference threshold value;

[0036] Fig.15 A display effect diagram of a written graphic when the difference value provided in some embodiments of the present application is less than or equal to the difference threshold value;

[0037] Fig.16 A schematic diagram of a process of displaying a written graphic when a target area is a non-blank area provided in some embodiments of the present application;

[0038] Fig.17The display effect diagram of the written graph when the proportion is less than the proportion threshold value provided by some embodiments of the present application;

[0039] Fig.18 Another display effect diagram of the written graph when the proportion is less than the proportion threshold value provided by some embodiments of the present application;

[0040] Fig.19 The display effect diagram of the written graph when the proportion is greater than the proportion threshold value provided by some embodiments of the present application. Detailed implementation manners

[0041] The embodiments will be described in detail below, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application detailed in the claims.

[0042] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0043] The terms "first", "second", "third", etc. in the specification, claims and the above drawings of the present application are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0044] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0045] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to the element.

[0046] Figure 1 The schematic diagram of the operation scenario between the display device and the control device provided by some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300 and the control device 100. Among them, the control device 100 can be a remote control, a stylus, etc.

[0047] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0048] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0049] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0050] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0051] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0052] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0053] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0054] In some embodiments, the display 260 is connected to a touch interaction module so that the display device 200 supports touch interaction operations. The touch interaction module can be a built-in module or an external module. For example, in order to implement touch interaction operations, for a display device 200 with a built-in touch interaction module, the display 260 can be a touch screen, which can receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy.

[0055] For the display device 200 with an external touch interaction module, a terminal, device, or component with a touch function can be connected through the device interface of the display device 200. For example, the USB or USB-C interface of the display device 200 is connected to a touch panel, and the touch panel can receive the touch track input by the user in real time and transmit the touch track to the display device 200 through the USB channel. The display device 200 can perform an interactive response based on the received touch track in combination with the touch application, so that the display device 200 supports touch interaction operations.

[0056] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0057] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0058] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0059] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0060] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0061] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0062] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0063] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0064] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0065] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0066] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0067] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0068] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0069] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0070] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0071] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0072] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0073] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0074] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0075] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0076] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0077] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0078] The display device 200 can also deploy other device interfaces, hardware, and drivers based on the kernel layer according to the specific display requirements of the display device 200. For example, in order to enrich the display types of the display device 200, an HDMI interface and a USB interface can be set on the display device 200, wherein the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver. The USB interface can connect devices with signal and data transmission functions and interactive functions, and based on the USB driver, access signals, transmit data, and perform specific interactive actions.

[0079] For a display device 200 that supports touch interaction operations, the display device 200 has a built-in touch interaction module, and can receive touch data input by the user through the touch interaction module. According to the interaction principle of the touch interaction module, the touch interaction module can detect relevant parameters in the touch action. For example, a capacitive touch layer is provided on the display 260 of the display device 200. When the user's finger touches any position of the touch layer, the capacitance of the touch layer at the touch position will change. At this time, the touch interaction module can record the position of the capacitance change, and record the time when the capacitance change occurs in combination with the timer in the controller to generate touch data.

[0080] In some embodiments, the physical signal detected by the touch interaction module needs to be converted before it can be read by the controller 250 of the display device 200. For example, the analog signal of the capacitance change needs to be converted into a digital signal and then transmitted to the controller 250 of the display device 200. In order to adapt to the screen display process of the display device 200, the touch data detected by the touch interaction module also needs to be coordinate mapped so that the position point corresponding to the touch data can have a corresponding relationship with the display 260 of the display device 200 or the pixel point in the user interface. For example, the coordinate position detection accuracy of the touch interaction module is 65536×65536, and the resolution of the display 260 of the display device 200 is 3840×2160. After the touch interaction module detects the touch data, the coordinate mapping of the touch point position in the width and height directions can be performed according to the ratio of 65536 / 3840 and 65536 / 2160, so that the display device 200 can perform related operation responses based on the mapped touch point position coordinates.

[0081] The touch data may include different operation parameters according to the user's input method. Figure 5 As shown, the touch data may include the touch time, touch position, touch event type, number of touch points, etc. of the touch operation. In a touch interaction process, the touch event types that can be detected by the touch interaction module include: down events, move events, and up events of touch actions. Corresponding to the entire touch track formed by the touch data, it can be the starting point [down; x1, y1, t1], the track point [move; x2, y2, t2], and the end point [up; x3, y3, t3]. Each touch data records the corresponding touch event type, touch time t, and touch position (x, y), that is, the touch points of the touch track can include the starting point, track point, and end point.

[0082] It should be noted that, since the user's finger or touch device is in surface contact with the touch interaction module when performing touch interaction operations, that is, the touch interaction module can detect a contact area, and the contact area includes multiple contact points. Therefore, when the display device 200 performs interactive control, it can detect (or specify) a contact point in the contact area as a touch point. For example, when the contact area is an elliptical area formed when the user's finger contacts the touch interaction module, the display device 200 can traverse the contact point coordinates in the elliptical area, determine the coordinate extremes in two mutually perpendicular directions, to delineate the major axis and minor axis of the elliptical area, and use the contact point at the intersection of the major axis and the minor axis as the touch point.

[0083] In the process of executing the touch interaction response, the display device 200 can classify different touch times, touch positions, the number of touch points and the type of touch events, so as to detect the user's touch input gesture, thereby performing different interactive responses for different input gestures. For example, when the number of touch points in the touch trajectory detected by the touch interaction module is 1, the touch event type only includes a down event and an up event, and the time interval between the down event and the up event is less than the single-click event time threshold. In addition, if the touch position change distance of the down event and the up event is less than the jitter distance threshold, it can be determined that the current user's touch input gesture is a click gesture. Based on the click gesture, the display device 200 executes the operation of starting the application according to the icon in the area where the touch position corresponding to the click gesture is located.

[0084] Similarly, the display device 200 can also detect other touch interaction gestures input by the user based on the touch time, touch position, number of touch points and touch event type, such as double-click gesture, slide gesture, long press gesture, multi-finger click gesture, multi-finger slide gesture, etc. Different interaction gestures can derive different touch parameters. For example, a slide gesture can also generate parameters such as slide distance, slide speed, slide direction, slide track shape, etc. Different touch parameters can be used to perform different interactive controls.

[0085] In some embodiments, in order to enrich the touch interaction types supported by the display device 200, the display device 200 can also perform complex gesture detection based on the dynamic change rules of touch time, touch position, number of touch points and touch event type. For example, when the touch interaction module detects that the touch trajectory of the user input includes multiple touch points, and the touch position of each touch point approaches each other as the user inputs, it can be determined that the touch gesture currently input by the user is a grab gesture. Based on complex gesture detection, the display device 200 can perform more interactive responses. For example, when the detected touch gesture is a grab operation for an image view, the display device 200 can respond to the grab gesture and perform a control action to select and move the image display position.

[0086] In some embodiments, the touch gesture can also be combined with the touch position to form a touch interaction operation based on a specific position. For example, when the touch interaction module detects that the down event position during the touch operation is at the top of the display 260, the touch point position corresponding to the move event moves downward. When the up event is detected, the display device 200 can determine that the current touch gesture is to slide down from the top of the screen. Therefore, the display device 200 can respond to the touch gesture and control the display 260 to present a drop-down list.

[0087] Some touch interaction gestures can be used to perform interactive actions based on the operating system of the display device 200, which are called global gestures. For example, the operation of sliding down from the top of the screen to call out a drop-down list in the above example of the display device 200. Some touch interaction gestures can be used to perform interactive actions based on specific touch applications. For example, when the display device 200 runs a whiteboard application, the sliding touch operation input by the user can be used to draw a track line on the whiteboard.

[0088] The touch application can be a system application or a third-party application developed based on a specific display device type and user needs. The touch application can be installed in the display device 200 and started to run under user control. For example, the user can control the display device 200 to run the whiteboard application by clicking the "whiteboard" application icon in the application interface of the display device 200.

[0089] It should be noted that the whiteboard application can be an independent application, an associated application, or a functional module in a touch application. Different forms of whiteboard applications can have different startup or calling methods. For example, a user can start an independent whiteboard application through the application interface of the display device 200; a user can also start a whiteboard application by executing a "whiteboard" operation on a conference application; a user can also control the conference application to start the whiteboard functional module by clicking the "whiteboard" icon on the conference application.

[0090] Different forms of whiteboard applications may present different forms of whiteboard application interfaces after startup. For example, when a user starts a whiteboard application in an associated application form, the display device 200 may automatically enter a dual-screen (or split-screen) mode. In the dual-screen (or split-screen) mode, the display device 200 may display a user interface through two display areas, i.e., the main screen displays the original application (such as a conference application) interface, and the secondary screen displays the whiteboard application interface.

[0091] After the whiteboard application is running, the display device 200 can be switched to display the whiteboard application interface. Figure 6 As shown, the whiteboard interface may include drawing tool controls for drawing operations, such as pen shape, graphics, line shape, color, erase, clear screen, selection, etc. The user can click any control to draw graphics according to specific drawing needs.

[0092] In some embodiments, the display device 200 classifies the touch track input by the user based on the whiteboard application according to the selected state of the drawing tool control. For example, when the drawing tool control such as pen shape, graphic shape, line shape, etc. is in the selected state, the touch track input by the user is marked as a drawing event; when the erasing drawing tool is in the selected state, the touch track input by the user is marked as an erasing event. The display device 200 then responds to the touch track of the drawing event or the erasing event and executes different programs according to the preset interaction strategy.

[0093] In order to distinguish the touch track, the whiteboard application includes a writing area and a menu area. The writing area is used to display the track line formed by the touch track and execute the drawing or erasing event of the touch track; the menu area is used to display the above-mentioned drawing tool control and execute the click event of the drawing tool control. When the user inputs the touch track in the writing area, the display device 200 draws the track line of the touch track and renders the track line to the whiteboard application interface; when the user inputs the touch track in the menu area, the display device 200 modifies the selection state of the drawing tool control according to the position of the touch track.

[0094] like Figure 7As shown, the display device 200 can render and display the drawing picture in real time through an image generation (Image Producer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module and a display (screen).

[0095] In the process of displaying the drawing picture, the graphic interface (openGLES) of the image generation module can be connected to the whiteboard application to generate the drawing image content and pass the image content to the image rendering module. The image rendering module is used to control the display 260 to display the content, that is, to render the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the picture frames can be stored in the synthesis module through the frame buffer (framebuffer) and the video buffer (video buffer), respectively, and the picture content is formed in the corresponding layer through the video process module. That is, the image frames generated by the video playback process form the picture content through the video layer (video plane), and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the primary OSD layer (primary OSD plane). Finally, the display device 200 forms the final picture through the blending unit (blend) of the video process module and displays it on the display 260.

[0096] Since the whiteboard application will present the user's touch trajectory in real time according to the user's touch operation, in order to present a better real-time response effect, the display device 200 should shorten the touch action input time and the delay time of the drawing result display. To this end, in some embodiments, the display device 200 can present a real-time painting effect by setting an acceleration layer. That is, in the video process module of the display device 200, an additional extended display layer (external OSD plane) can be set on the basis of the main OSD layer and the video layer. Among them, the extended display layer can be configured with different layer characteristics according to the specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0097] Taking the display device 200 running a whiteboard application as an example, in order to improve the real-time response speed of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). Figure 8 As shown, the display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the track line drawn by the acceleration layer is updated to other layers, thereby reducing the input delay and improving the real-time display effect of the whiteboard application drawing process.

[0098] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, that is, when the display device 200 detects the end point of the lift event, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawn graphics, thereby eliminating the need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0099] Based on the above display device 200, in some embodiments, the user can input the touch track of various characters in the display device 200 through touch interaction operations, so that the display device 200 presents the corresponding writing graphics. That is, the user can write the corresponding touch track in the display device 200 according to the strokes of each character, and the display device 200 maps the touch track to a track line according to the preset mapping relationship, and renders the track line to the OSD layer, so that the display 260 displays the track line. Among them, the track lines of each stroke in the text can constitute a complete writing graphic.

[0100] like Fig. 9 As shown, in order to improve the standardization of the written graphics, the display device 200 can also convert the trajectory line of the written graphics into a standard body, that is, in some embodiments, the display device 200 also detects the enabled state of the standard body. If the standard body is in the enabled state, the display device 200 detects the end point of the touch trajectory. Among them, the end point is the touch point of the lift (up) event. The end point is used as the timing starting point, and the touch point is received within the touch event cycle. If the touch point is not received within the touch time period, the standard body model is called, and the position coordinates of the converted touch point are obtained. The converted touch point includes a touch point whose touch time is earlier than the touch time of the end point. Then the position coordinates of the converted touch point are input into the standard body model to generate a standard body data object. Then, based on the standard body data object, the trajectory line of the converted touch point is rendered in the whiteboard application interface to obtain the written graphics.

[0101] That is to say, the whiteboard application of the display device 200 can be set with a conversion function for standard fonts. Among them, the standard fonts can include a variety of different fonts. When the standard font function is in the enabled state, the display device 200 can detect the termination point of the touch trajectory, and when detecting the termination point of a certain touch trajectory and no new touch points are detected within a preset time period, input the original touch data of the touch points that have not been processed before the termination point into the standard font model. The standard font model will convert these touch points into corresponding standard font coordinate points, and output the converted standard font coordinate points to generate a new standard font data object. Among them, the standard font data object and the touch data of the touch points are two different data objects, which can be stored separately in the display device 200, for example, stored in the memory or file of the display device 200. After generating the standard font data object, the display device 200 renders the trajectory line of the touch trajectory based on the standard font data object, so that the whiteboard application interface can present the writing graphics in standard fonts. On the contrary, if the standard font function is in the disabled state, the trajectory line is directly rendered according to the touch data of the touch points.

[0102] For example, the preset touch time period is 3 seconds. When the display device 200 detects the termination point a of a certain touch trajectory and no new touch points are detected within 3 seconds, input all the touch points that have not been processed and whose touch time is earlier than the touch time of point a into the standard font model, so that the standard font model outputs a standard font data object. The display device 200 then renders a more regular writing graphic in the whiteboard application interface based on the standard font object, as Fig.10 shown.

[0103] In the embodiment of the present application, taking the writing graphic converted into the standard font as an example, that is, after the user inputs a touch trajectory in the writing area of the whiteboard application interface, the display device 200 will convert the touch trajectory into a standard font data object and render the trajectory line according to the standard font data object, so as to display the writing graphic in the standard font in the writing area of the whiteboard application interface.

[0104] However, different from the keyboard input method, there is no control such as a cursor for characterizing the input position in the touch interaction operation. Therefore, when the display device 200 converts the writing graphic in the standard font, the display position of the writing graphic is randomly set, resulting in the display device 200 being unable to display the writing graphic at the position expected by the user. Moreover, since the size of the writing graphic is set according to the range of the touch trajectory, the size of the writing graphic cannot be kept consistent when the ranges of the touch trajectories are inconsistent. Therefore, when the display device 200 converts the writing graphic in the standard font, there is also a problem that it is difficult to keep the sizes of the writing graphics the same. In this way, when the display device 200 converts the writing graphic in the standard font, it is easy to appear as Fig.11If the position and size of each writing graphic are modified, the user needs to manually edit the size and display position of the writing graphic again, which increases the complexity of touch interaction operations and reduces user experience.

[0105] Based on the above application scenarios, in order to improve the problem of high complexity of touch interaction operation when the display device 200 displays written graphics, some embodiments of the present application provide a display device 200, including a display 260 and a controller 250. The display 260 is configured to display a whiteboard application interface, which includes a writing area; the display 260 is connected to a touch interaction template, the touch interaction module is used to detect the touch track input by the user, and the writing area is used to display the writing graphics, which are track lines generated based on the touch track.

[0106] like Fig.12 As shown, the controller 250 is configured to perform the following program steps:

[0107] S100: In response to a touch event of a target touch track input by a user, a boundary box of the target touch track is acquired, and a mapping word of the target touch track is identified.

[0108] After detecting a touch event of a target touch track input by a user, the display device 200 detects a touch point of the target touch track in response to the touch event. The target touch track includes at least one touch point, which may include the starting point, track point, and end point described in the above embodiment.

[0109] In some embodiments, when the display device 200 detects the end point of the touch track, it also generates a bounding box of the touch track based on all the touch points of the touch track. The bounding box is used to characterize the boundary range of the touch track, that is, the closed area formed by the bounding box can cover all the touch points of the touch track. For example, the bounding box can be a rectangular box, etc. The display device 200 can determine the area where the touch track is located through the bounding box of the touch track.

[0110] It should be noted that the boundary box of the touch track is the attribute data stored when the display device 200 stores the touch track, and will not be displayed in the whiteboard application interface.

[0111] In some embodiments, the display device 200 may recognize the touch tracks based on the OCR (Optical Character Recognition) technology, mark the touch tracks constituting a font as a group, and use the union of the bounding boxes corresponding to the touch tracks in the same group as the bounding box of a written graphic.

[0112] It should be noted that the target touch track in the embodiment of the present application is the touch track input by the user based on the strokes of each character, and the target touch track may correspond to one touch track or multiple touch tracks. When the target touch track corresponds to multiple touch tracks, it may include a touch track corresponding to one character or multiple touch tracks corresponding to multiple characters.

[0113] Therefore, in order to facilitate the display device 200 to obtain the target touch track, in some embodiments, when the display device 200 detects the end point of the touch track, the end point is used as the timing starting point to receive the touch point within the touch time period. If no touch point is received within the touch time period, the target touch point whose touch time is earlier than the touch time of the end point is obtained to obtain the target touch track.

[0114] After acquiring the target touch track, the display device 200 acquires the bounding box of the target touch track according to the touch data corresponding to the target touch track, and the bounding box is used to characterize the boundary range of the target touch track. At the same time, the display device 200 can also recognize the target touch track based on algorithms such as feature recognition, so as to obtain the mapping word of the target touch track. The mapping word is the text recognized based on the target touch track.

[0115] In order to obtain the bounding box of the target touch track, in some embodiments, the display device 200 also reads the position coordinates of the target touch point, which is the touch point of the target touch track. Then, the boundary coordinates are generated according to the position coordinates of the target touch track. Among them, the boundary coordinates include the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the position coordinates. Then, a rectangular object (rect) is created according to the boundary coordinates, and the bounding box of the target touch track is generated based on the rectangular object.

[0116] For example, among the position coordinates of each target touch point, the minimum horizontal coordinate is x min , the maximum horizontal coordinate is x max , the minimum vertical coordinate is y min , the maximum vertical coordinate is y max When the display device 200 creates a rectangular object, the upper left corner coordinates of the rectangular object are defined as (x min ,y max ), the coordinates of the lower right corner of the rectangle object are (x max ,y min ), the width of the resulting rectangular object is |x max -x min |, height is |y max -y min |. After completing the creation of the above rectangular object, use the rectangular object as the bounding box of the target touch track.

[0117] S200: Detecting a target area in the writing area.

[0118] After obtaining the bounding box of the target touch track, the display device 200 can detect the target area in the writing area of ​​the whiteboard application interface according to the bounding box of the target touch track. The target area is the display area of ​​the bounding box. The target area can represent the range of the target touch track in the writing area. The display device 200 can determine whether the target touch track overlaps with other graphics by detecting whether the target area is a blank area.

[0119] like Fig.13 As shown, in some embodiments, the display device 200 also queries the target pixel set. The target pixel set includes the pixels of the target area. Then the pixel values ​​of each pixel in the target pixel set are detected. If the pixel values ​​of the pixels in the target pixel set are all initial pixel values, the target area is marked as a blank area; if the target pixel set includes pixels whose pixel values ​​are not the initial pixel values, the target area is marked as a non-blank area. The initial pixel value is used to characterize that the current pixel does not include any graphics.

[0120] For example, the pixel value of each pixel is represented by the RGB component, and the initial pixel value is (0, 0, 0). After obtaining the bounding box of the target touch track, the display device 200 determines the target area according to the display area of ​​the bounding box in the whiteboard application interface, and uses the pixel points of the target area as the target pixel point set. The pixel value of each pixel point is detected row by row in the target pixel point set. When all pixel values ​​in the target pixel point set are (0, 0, 0), the target area is marked as a blank area; otherwise, the target area is marked as a non-blank area.

[0121] S300: When the target area is a blank area, generate a first average width according to the number of bounding boxes and mapped characters.

[0122] When the display device 200 detects that the target area is a blank area, it represents that the target touch track does not overlap with other graphics, so the display device 200 can directly display the writing graphics of the target touch track in the target area, that is, the display device 200 can display the writing graphics according to the position of the current target touch track. Before generating the writing graphics, the display device 200 also needs to determine the size of the target writing graphics. Therefore, the display device 200 also generates the first average width of each mapped word based on the total width of the target touch track and the number of mapped words. Since the bounding box is used to represent the boundary range of the target touch track, the display device 200 can calculate the first average width based on the width of the bounding box and the number of mapped words.

[0123] Therefore, in some embodiments, when the display device 200 generates the first average width according to the number of bounding boxes and mapping characters, it reads the rectangular width of the bounding box and the target number of mapping characters. Then it calculates the ratio of the rectangular width to the target number as the first average width.

[0124] For example, as Fig.14 shown, the target touch trajectory is the touch trajectory within the dashed box. The minimum abscissa in the bounding box of the target touch trajectory is x min and the maximum abscissa is x max . The mapping characters recognized by the display device 200 are "early", "morning", and "good", and the number of mapping characters is 3. The display device 200 then calculates the first average width according to the following formula

[0125]

[0126] S400: Calculate the difference between the first average width and the reference width.

[0127] After obtaining the first average width, the display device 200 also calculates the difference between the first average width and the reference width. Here, the reference width is the width of the writing graphic with the latest rendering time in the writing graphics, that is, the width of the writing graphic rendered by the display device 200 in the previous time. To enable the display device 200 to automatically keep the sizes of the writing graphics consistent, a difference threshold is preset in the display device 200. When the difference between the first average width and the reference width is less than or equal to this difference threshold, the display device 200 generates the writing graphic of the target touch trajectory according to the reference width. In this way, when the size of the writing graphic of the target touch trajectory input by the user can have a small difference from the size of the writing graphic rendered in the previous time, the display device 200 generates the writing graphic of the target touch trajectory according to the size of the already displayed writing graphic, so that the size of the writing graphic of the target touch trajectory can be kept consistent with the size of the writing graphic rendered in the previous time.

[0128] S500: If the difference is greater than the difference threshold, generate the target writing graphic according to the first average width, and control the display to display the target writing graphic at the starting point of the target touch trajectory in the writing area.

[0129] When the difference between the first average width and the reference width is greater than the difference threshold, it indicates that there is a significant difference between the size of the writing pattern of the target touch trajectory and the size of the writing pattern rendered previously. At this time, the user may not want to keep the size of the writing pattern of the target touch trajectory consistent with the size of the displayed writing pattern. Therefore, when the target area is a blank area and the difference between the first average width and the reference width is greater than the difference threshold, the display device 200 generates a target writing pattern according to the first average width, and controls the display 260 to display the target writing pattern at the starting point of the target touch trajectory in the writing area. The starting point is the touch point of the down event in the target touch trajectory, and the target writing pattern is the trajectory line generated based on the target touch trajectory.

[0130] In some embodiments, when the display device 200 generates a target writing pattern according to the first average width and controls the display 260 to display the target writing pattern at the starting point of the target touch trajectory in the writing area, it detects the end point of the target touch trajectory. The end point is the touch point of the lift event in the target touch trajectory. Taking the end point as the starting point of timing, touch points are received within the touch time period. If no touch point is received within the touch time period, the standard body model is called, and the target touch point of the target touch trajectory is obtained. The position coordinates of the target touch point, the position coordinates of the starting point, and the first average width are input into the standard body model to generate a standard body data object. Then, based on the standard body data object, the trajectory line of the target touch point is rendered in the writing area to obtain the target writing pattern.

[0131] That is to say, in order for the display device 200 to generate a target writing pattern according to the first average width and display the target writing pattern at the starting point of the target touch trajectory, the position coordinates of the starting point and the first average width are used as input features and input into the standard body model, so that the standard body model performs coordinate transformation on the position coordinates of the target touch point according to the first average width with the starting point of the target touch trajectory as the starting point, and outputs a standard body data object.

[0132] For example, as Fig.14 shown, the target area is a blank area, and the displayed writing patterns in the writing area are "big" and "home"; the writing pattern with the latest rendering time is "home", and its corresponding width is X last ; the difference threshold is m; the average width of the target touch trajectory "good morning" is The starting point of the target touch trajectory is α. After the display device 200 calculates , it compares the magnitudes of k and m. When k > m, as Fig.14 shown, a writing pattern is generated according to , and the writing pattern is displayed according to the position coordinates of α. Here, α is the rendering starting point of the writing pattern.

[0133] S600: If the difference value is less than or equal to the difference threshold, generate a target writing graphic according to the reference width, and control the display to display the target writing graphic at the starting point of the target touch trajectory in the writing area.

[0134] When the difference between the first average width and the reference width is less than or equal to the difference threshold, the display device 200 can keep the size of the target writing graphic consistent with the size of the previously rendered writing graphic. Therefore, when the target area is a blank area and the difference between the first average width and the reference width is less than or equal to the difference threshold, the display device 200 generates a target writing graphic according to the reference width, and controls the display 260 to display the target writing graphic at the starting point of the target touch trajectory in the writing area.

[0135] For example, as Fig.15 shown, the target area is a blank area, and the writing graphics already displayed in the writing area are "big" and "home"; the writing graphic with the latest rendering time is "home", and its corresponding width is X last ; the difference threshold is m; the average width of the target touch trajectory "good morning" is The starting point of the target touch trajectory is α. After the display device 200 calculates , compare the magnitudes of k and m. When k ≤ m, as Fig.15 shown, generate a writing graphic according to X last and display the writing graphic according to the position coordinates of α. Among them, α is the rendering starting point of the writing graphic.

[0136] Based on the above embodiments, the display device 200 provided by the present application can, when the target area is a blank area, automatically set the size of the writing graphic based on the reference width of the previously rendered writing graphic and the first average width of each mapped character in the target touch trajectory, and display the writing graphic at the corresponding position in the writing area according to the starting point of the target touch trajectory, realizing the adaptive setting of the position and size of the writing graphic, reducing the manual operation process of the user, and thus reducing the complexity of the touch interaction operation when the display device 200 displays the writing graphic.

[0137] However, since the target touch trajectory may also overlap with other graphics, displaying the target writing graphic at the starting point of the target touch trajectory at this time will cause the target writing graphic to overlap with other graphics, thereby affecting the display effect of the target writing graphic.

[0138] Therefore, as Fig.16As shown, in some embodiments, when the target area is a non-blank area, the display device 200 also queries the overlapping area in the target area. The overlapping area is composed of pixel points whose pixel values ​​in the target pixel point set are not the initial pixel values. Then calculate the proportion of the overlapping area in the target area. If the proportion is less than the ratio threshold, detect the relative position of the overlapping area and the bounding box, and obtain the first edge point of the overlapping area and the second edge point of the bounding box according to the relative position. The horizontal coordinate or vertical coordinate of the first edge point is the same as that of the second edge point. Calculate the distance between the first edge point and the second edge point to generate a rendering width. Then generate a second average width based on the rendering width and the number of mapped words. After generating the second average width, generate a target writing graphic according to the second average width, and control the display 260 to display the target writing graphic in the writing area according to the center coordinates of the bounding box.

[0139] That is, when the target touch track overlaps with other images, the display device 200 can query the blank area of ​​the target area according to the relative position of the target touch track and the overlapping area, and then reduce the size of the target writing graphic according to the blank area in the target area, and display the target writing graphic according to the center coordinates of the bounding box, so that the target writing graphic can avoid the graphics displayed in the writing area and be displayed normally in the whiteboard application interface.

[0140] For example, Fig.17 As shown, the ratio threshold is L%, the relative position of the bounding box of the target touch track and the overlapping area is: the overlapping area is located on the left side of the bounding box, the mapping words of the target touch track are "early", "up", "good", and the center coordinate of the bounding box is β. When the display device 200 detects that the target area is a non-blank area, the overlapping area of ​​the target area is queried. Then calculate the proportion Q% of the overlapping area in the target area, and compare Q% with the ratio threshold L%. When Q% < L%, the display device 200 obtains the first edge point (x 1 ,y 1 ) and the second edge point of the bounding box on the left (x 2 ,y 2 ), y 1 =y 2 . Calculate the distance d between the first edge point and the second edge point, d = |x 2 -x 1 |. Then calculate the second average width according to d and the number of mapped words according to Generate a writing graphic and display it according to the position coordinates of β, where β is the rendering center point of the writing graphic.

[0141] Alternatively, in some embodiments, if the proportion is less than the ratio threshold, such as Fig.16As shown, the display device 200 also obtains the third edge point of the overlapping area. Then, the moving area is detected in the writing area based on the third edge point. Among them, the edge points of the moving area include the third edge point, the area of ​​the moving area is equal to the area of ​​the bounding box, and the moving area is a blank area. If the writing area includes the moving area, a first average width is generated according to the number of bounding boxes and mapped characters. Then, the target writing graphic is generated according to the first average width, and the display 260 is controlled to display the target writing graphic in the moving area according to the center coordinates of the moving area. On the contrary, if the writing area does not include the moving area, the target writing graphic is generated according to the second average width, and the display 260 is controlled to display the target writing graphic in the writing area according to the center coordinates of the bounding box.

[0142] That is to say, different from the method of reducing the size of the target writing graphic in the above-mentioned embodiment, when the target touch track overlaps with other images, the display device 200 may also check whether there are other blank areas equal to the bounding box in the writing area. And, in order to improve the user experience, the blank area currently queried by the display device 200 needs to be adjacent to the overlapping area. Therefore, the display device 200 detects the blank area that meets the above conditions as the moving area based on the edge points of the overlapping area. When there is a moving area in the writing area, the display device 200 can display the target writing graphic in the moving area without reducing the size of the target writing graphic. When there is no moving area in the writing area, the display device 200 displays the target writing graphic of the target touch track in accordance with the method of reducing the size of the target writing graphic provided in the above-mentioned embodiment.

[0143] In order to facilitate the detection of whether there is a moving area, in some embodiments, after obtaining the third edge point, the display device 200 establishes a two-dimensional coordinate system with the third edge point as the origin, and queries whether there is a moving area in the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant based on the two-dimensional coordinate system. Among them, the edge point of the moving area at least includes the origin of the two-dimensional coordinate system, the area of ​​the moving area is equal to the area of ​​the bounding box, and the moving area is a blank area, that is, the display device 200 can use the third edge point as the starting point to query whether there is a blank area with an area equal to the area of ​​the bounding box above, below, above, and below the third edge point.

[0144] For example, Fig.18 As shown, the ratio threshold is L%, the relative positions of the bounding box of the target touch track and the overlapping area are: the overlapping area is located on the left side of the bounding box, the mapping words of the target touch track are "early", "up", "good", and the width of the bounding box is |x max -x min |, height is |y max -y min|. When the display device 200 detects that the target area is a non-blank area, the overlapping area of ​​the target area is queried. Then the proportion of the overlapping area in the target area, Q%, is calculated, and Q% is compared with the ratio threshold value, L%. When Q% < L%, the display device 200 randomly obtains the third edge point (x3, y3) on the right side of the overlapping area, and takes the third edge point (x3, y3) as the starting point to query whether there is a width of |x in the upper left, lower left, upper right, and lower right directions of the third edge point in the writing area. max -x min |, height is |y max -y min | blank area. The existence degree of the third edge point in the writing area in the lower left direction is |x max -x min |, height is |y max -y min |The blank area is calculated according to the first average width Generate the target writing graph. The center point of the blank area found is γ, such as Fig.18 As shown, the display device 200 displays the writing graphic according to the position coordinate of γ, where γ is the rendering center point of the writing graphic.

[0145] When the proportion of the overlapping area is greater than or equal to the ratio threshold, it means that the overlap between the target touch track and other graphics is not a user's mistake. Fig.16 As shown, if the proportion is greater than or equal to the proportion threshold, the step of generating a first average width according to the number of bounding boxes and mapped words is performed.

[0146] That is, when the proportion is greater than or equal to the ratio threshold, the display device 200 generates a first average width according to the number of bounding boxes and mapped characters. Then calculate the difference between the first average width and the reference width. If the difference is greater than the difference threshold, the target writing graphic is generated according to the first average width, and the display 260 is controlled to display the target writing graphic in the writing area according to the starting point of the target touch track; if the difference is less than or equal to the difference threshold, the target writing graphic is generated according to the reference width, and the display 260 is controlled to display the target writing graphic in the writing area according to the starting point of the target touch track.

[0147] For example, Fig.19 As shown, the ratio threshold is L%, the relative positions of the bounding box and the overlapping area of ​​the target touch track are: the overlapping area is located on the left side of the bounding box, the mapping words of the target touch track are "early", "up", and "good", and the average width of the target touch track "early", "up", and "good" is The starting point of the target touch track is α. When the display device 200 detects that the target area is a non-blank area, the overlapping area of ​​the target area is queried. Then the proportion of the overlapping area in the target area Q% is calculated, and Q% is compared with the proportion threshold L%. When Q% ≥ L%, if Fig.19 As shown, according to Generate a target writing graphic and display the writing graphic according to the position coordinates of α, where α is the rendering starting point of the writing graphic.

[0148] In order to meet the personalized needs of users, in some embodiments, the display device 200 is provided with an original graphics mode. The display device 200 also detects the enabled state of the original graphics mode. For example, when the whiteboard application is started, the application layer initializes the Flag of the original graphics mode to 0, indicating that the default original graphics mode is in the closed state. The user can modify the original graphics mode to the open state in the menu area of ​​the whiteboard application interface, and the display device 200 sets the Flag of the original graphics mode to 1. The display device 200 can determine the enabled state of the original graphics mode by detecting the Flag value of the original graphics mode.

[0149] After the display device 200 detects the enabled state of the original graphics mode, when the enabled state of the original graphics mode is on, a first average width is generated according to the number of bounding boxes and mapped characters. Then a target writing graphic is generated according to the first average width, and the display 260 is controlled to display the target writing graphic in the writing area according to the starting point of the target touch track. That is, when the enabled state of the original graphics mode is on, the display device 200 does not need to detect whether the target area is a blank area, and directly calculates the first average width according to the number of bounding boxes and mapped characters, and generates a target writing graphic of the first average width, and controls the display 260 to display the target writing graphic according to the starting point of the target touch track, thereby maintaining the original state of the writing graphic.

[0150] Based on the above-mentioned display device 200, some embodiments of the present application also provide a method for displaying a written graphic, which is applied to the display device 200 provided in the above-mentioned embodiment, and the display device 200 includes a display 260 and a controller 250. The display 260 is configured to display a whiteboard application interface, and the whiteboard application interface includes a writing area; the display 260 is connected to a touch interaction template, and the touch interaction module is used to detect the touch track of the user input, and the writing area is used to display the written graphic, which is a track line generated based on the touch track. Fig.12 As shown, the method comprises the following procedural steps:

[0151] S100: In response to a touch event of a target touch track input by a user, a bounding box of the target touch track is obtained, and a mapping word of the target touch track is identified, wherein the bounding box is used to represent a boundary range of the target touch track, and the mapping word is a text identified based on the target touch track;

[0152] S200: Detecting a target area in the writing area, where the target area is a display area of ​​the bounding box;

[0153] S300: When the target area is a blank area, generating a first average width according to the number of the bounding box and the mapping words;

[0154] S400: Calculating the difference between the first average width and a reference width, where the reference width is the width of the writing graphic with the latest rendering time in the writing area;

[0155] S500: if the difference is greater than the difference threshold, generate a target writing graphic according to the first average width, and control the display to display the target writing graphic in the writing area according to the starting point of the target touch trajectory, where the starting point is the touch point of the drop event in the target touch trajectory, and the target writing graphic is a trajectory line generated based on the target touch trajectory;

[0156] S600: If the difference is less than or equal to the difference threshold, generate the target writing graphic according to the reference width, and control the display to display the target writing graphic in the writing area according to the starting point of the target touch track.

[0157] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a method for displaying a written graphic, and the method can respond to a touch event of a user inputting a target touch track, obtain a bounding box of the target touch track, and identify the mapping word of the target touch track. Detect the target area of ​​the bounding box in the writing area, and when the target area is a blank area, generate a first average width according to the number of bounding boxes and mapping words. If the difference between the first average width and the reference width is greater than the difference threshold, generate the target writing graphic according to the first average width, and display the target writing graphic in the writing area according to the starting point of the target touch track; if the difference is less than or equal to the difference threshold, generate the target writing graphic according to the reference width, and display the target writing graphic in the writing area according to the starting point of the target touch track. Among them, the reference width is the width of the writing graphic with the latest rendering time. The method can automatically set the position and size of the writing graphic to be displayed by comparing the size of the displayed writing graphic and the average size of each mapping word in the touch track, thereby reducing the process of manual adjustment of the writing graphic by the user and improving the user experience.

[0158] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0159] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display device, It is characterized in that include: A display configured to display a whiteboard application interface, the whiteboard application interface including a writing area, the display connected to a touch interaction module, the touch interaction module used to detect a touch track input by a user, the writing area used to display a writing graphic, the writing graphic being a track line generated based on the touch track; The controller is configured as: In response to a touch event of a target touch track input by a user, a bounding box of the target touch track is obtained, and a mapping word of the target touch track is identified, wherein the bounding box is used to represent a boundary range of the target touch track, and the mapping word is a text identified based on the target touch track; Detecting a target area in the writing area, the target area being a display area of ​​the bounding box; When the target area is a blank area, generating a first average width according to the number of the bounding box and the mapping words; Calculate the difference between the first average width and a reference width, where the reference width is the width of the writing graphic with the latest rendering time in the writing area; If the difference is greater than the difference threshold, a target writing pattern is generated according to the first average width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch trajectory, where the starting point is the touch point of the drop event in the target touch trajectory, and the target writing pattern is a trajectory line generated based on the target touch trajectory; If the difference is less than or equal to the difference threshold, the target writing pattern is generated according to the reference width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch track.

2. The display device according to claim 1, It is characterized in that The controller performs detection of a target area in the writing area, and is further configured to: Querying a target pixel point set, where the target pixel point set includes pixel points of the target area; Detecting pixel values ​​of pixel points in the target pixel point set; If the pixel values ​​of the pixels in the target pixel set are all initial pixel values, marking the target area as a blank area; If the target pixel set includes pixel points whose pixel values ​​are not the initial pixel values, the target area is marked as a non-blank area.

3. The display device according to claim 2, It is characterized in that When the target area is a non-blank area, the controller is configured to: Querying an overlapping area in the target area, where the overlapping area is composed of pixel points in the target pixel point set whose pixel values ​​are not the initial pixel values; Calculate the proportion of the overlapping area in the target area; If the proportion is less than the ratio threshold, detecting the relative position of the overlapping area and the bounding box; Acquire a first edge point of the overlapping area and a second edge point of the boundary box according to the relative positions, wherein the first edge point and the second edge point have the same horizontal coordinate or vertical coordinate; Calculating the distance between the first edge point and the second edge point to generate a rendering width; generating a second average width according to the rendering width and the number of the mapping words; The target writing pattern is generated according to the second average width, and the display is controlled to display the target writing pattern in the writing area according to the center coordinates of the bounding box.

4. The display device according to claim 3, It is characterized in that The controller is also configured to: If the proportion is greater than or equal to the ratio threshold, a step of generating a first average width according to the number of the bounding box and the mapping word is performed.

5. The display device according to claim 3, It is characterized in that If the proportion is less than the proportion threshold, the controller is further configured to: Obtaining a third edge point of the overlapping area; Detecting a moving area in the writing area according to the third edge point, the edge points of the moving area include the third edge point, the area of ​​the moving area is equal to the area of ​​the boundary box, and the moving area is a blank area; If the writing area includes the moving area, generating a first average width according to the bounding box and the number of the mapped characters; generating the target writing pattern according to the first average width, and controlling the display to display the target writing pattern in the moving area according to the center coordinates of the moving area; If the writing area does not include the moving area, the target writing pattern is generated according to the second average width, and the display is controlled to display the target writing pattern in the writing area according to the center coordinates of the bounding box.

6. The display device according to claim 1, It is characterized in that The controller executes obtaining the bounding box of the target touch track, and is configured to: Reading the position coordinates of a target touch point, wherein the target touch point is a touch point of the target touch track; Generate boundary coordinates according to the position coordinates of the target touch point, wherein the boundary coordinates include a minimum horizontal coordinate, a maximum horizontal coordinate, a minimum vertical coordinate, and a maximum vertical coordinate of the position coordinates; Create a rectangle object according to the boundary coordinates; The bounding box is generated based on the rectangular object.

7. The display device according to claim 6, It is characterized in that The controller generates a first average width according to the number of the bounding box and the mapping words, and is configured to: Reading the rectangular width of the bounding box and the target number of the mapping words; The ratio of the rectangle width to the target number is calculated as the first average width.

8. The display device according to claim 1, It is characterized in that The controller is also configured to: Detects whether raw graphics mode is enabled; When the enabling state of the original graphics mode is on, generating a first average width according to the number of the bounding box and the mapping words; A target writing pattern is generated according to the first average width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch track.

9. The display device according to claim 1, It is characterized in that The controller generates a target writing pattern according to the first average width, and controls the display to display the target writing pattern in the writing area according to the starting point of the target touch track, and is configured as follows: Detecting an end point of the target touch track, where the end point is a touch point of a lift event in the target touch track; Taking the end point as the timing start point, receiving the touch point within the touch time period; If no touch point is received within the touch time period, calling a standard body model, and acquiring a target touch point of the target touch trajectory; Inputting the position coordinates of the target touch point, the position coordinates of the starting point and the first average width into the standard volume model to generate a standard volume data object; The trajectory line of the target touch point is rendered in the writing area based on the standard volume data object to obtain the target writing figure.

10. A method for displaying a written graphic, It is characterized in that The method is applied to a display device, the display device includes a display and a controller, the display is configured to display a whiteboard application interface, the whiteboard application interface includes a writing area, the display is connected to a touch interaction module, the touch interaction module is used to detect a touch track input by a user, the writing area is used to display a writing graphic, and the writing graphic is a track line generated based on the touch track; the method includes: In response to a touch event of a target touch track input by a user, a bounding box of the target touch track is obtained, and a mapping word of the target touch track is identified, wherein the bounding box is used to represent a boundary range of the target touch track, and the mapping word is a text identified based on the target touch track; Detecting a target area in the writing area, the target area being a display area of ​​the bounding box; When the target area is a blank area, generating a first average width according to the number of the bounding box and the mapping words; Calculate the difference between the first average width and a reference width, where the reference width is the width of the writing graphic with the latest rendering time in the writing area; If the difference is greater than the difference threshold, a target writing pattern is generated according to the first average width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch trajectory, where the starting point is the touch point of the drop event in the target touch trajectory, and the target writing pattern is a trajectory line generated based on the target touch trajectory; If the difference is less than or equal to the difference threshold, the target writing pattern is generated according to the reference width, and the display is controlled to display the target writing pattern in the writing area according to the starting point of the target touch track.

Images