Display device and line drawing rendering method

Abstract

The application discloses a display device and a line drawing rendering method. The display device comprises a display, a touch component for detecting a touch track, and a controller configured to generate a line drawing bitmap for receiving the touch track in response to an input instruction of starting a whiteboard application, superimpose the line drawing bitmap onto a first layer for refreshing display when the touch track does not extend to a menu area, superimpose the line drawing bitmap and a pre-generated menu bitmap onto the first layer for refreshing display in sequence when the touch track extends to the menu area, and refresh drawing content in the first layer to a second layer after current touch track input is completed. In the application, the display device can accelerate refreshing display of the superimposed menu bitmap and the line drawing bitmap through the first layer, ensure that a layer where a menu bar is located is higher than a layer where line drawing content is located in a line drawing process on the basis of maintaining line drawing speed, does not block the menu bar, and improves user experience.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display device and a line drawing rendering method. Background Technology

[0002] A smart TV is a television product that enables two-way human-computer interaction, integrating multiple functions such as audio-visual entertainment and drawing. The display of a smart TV serves as the medium for interaction and information exchange with the user. To meet diverse user needs, touch components can be integrated into the display, creating a touch-screen TV for user touch interaction. In educational settings, smart TVs can be equipped with whiteboard applications, allowing users to conduct whiteboard presentations.

[0003] During whiteboard applications, users touch the display, and the smart TV renders the user's touch trajectory to generate corresponding content. When rendering, the smart TV typically copies the acquired user touch trajectory to the GOP (Group of Pictures) layer for refresh and display. After the user finishes drawing, the GOP layer is deactivated, and the touch trajectory is then refreshed to the OSD (On-Screen Display) layer for display. The GOP layer is higher than the OSD layer, which also displays the menu bar. Therefore, during the drawing process, the content appears above the menu bar, obscuring it and resulting in a poor user experience. Summary of the Invention

[0004] This application provides a display device and a line drawing rendering method to solve the technical problem in the prior art where the drawn content obscures the menu bar, resulting in a poor user experience.

[0005] To address the aforementioned technical problems, the embodiments of this application disclose the following technical solutions:

[0006] In a first aspect, embodiments of this application disclose a display device, the display device comprising:

[0007] monitor;

[0008] The touch component is configured to detect the touch trajectory of user input;

[0009] The controller is configured to:

[0010] In response to an input command to launch the whiteboard application, a plotting bitmap for receiving the touch trajectory is generated;

[0011] When the touch trajectory does not extend to the menu area, the drawing bitmap is overlaid on the first layer for refresh display, wherein the first layer is a layer used to display the currently drawn content, and the menu area is an area containing the menu bar;

[0012] When the touch trajectory extends to the menu area, the drawn line bitmap and the pre-generated menu bitmap are sequentially superimposed on the first layer for refresh display, wherein the menu bitmap is a bitmap containing the menu bar generated based on the menu area;

[0013] After the current touch trajectory input is completed, the drawing content in the first layer is refreshed to the second layer, so that the display shows the user's touch trajectory according to the content in the second layer, wherein the second layer is the layer located below the first layer.

[0014] Secondly, embodiments of this application disclose a line drawing rendering method, the method comprising:

[0015] In response to the input command to launch the whiteboard application, a line drawing bitmap is generated to receive touch trajectories;

[0016] When the touch trajectory does not extend to the menu area, the drawing bitmap is overlaid on the first layer for refresh display, wherein the first layer is a layer used to display the currently drawn content, and the menu area is an area containing the menu bar;

[0017] When the touch trajectory extends to the menu area, the drawn line bitmap and the pre-generated menu bitmap are sequentially superimposed on the first layer for refresh display, wherein the menu bitmap is a bitmap containing the menu bar generated based on the menu area;

[0018] After the current touch trajectory input is completed, the drawing content in the first layer is refreshed to the second layer, so that the display shows the user's touch trajectory according to the content in the second layer, wherein the second layer is the layer located below the first layer.

[0019] Compared with the prior art, the beneficial effects of this application are as follows:

[0020] This application provides a display device and a line drawing rendering method. A user selects a whiteboard application on the display device and controls the device to launch the whiteboard application. The user can generate touch traces via a touch display. After the whiteboard application is launched, the display device generates a line drawing bitmap to receive the user-input touch traces. Based on the received user-input touch trace, the display device determines whether the touch trace extends to the area containing the menu bar. If the touch trace has not yet extended to the menu area, it means the drawn content will not conflict with the menu bar, and the display device can directly overlay the line drawing bitmap onto the first layer for refresh display. Conversely, if the touch trace extends to the menu area, it means the drawn content conflicts with the menu bar, and the display device needs to sequentially overlay the line drawing bitmap and a pre-generated menu bitmap onto the first layer for refresh display, so that the drawn content will not obscure the menu bar during final display. In this application, if the touch trajectory does not coincide with the menu area during the drawing process, the display device accelerates the refresh display of the line drawing bitmap through the first layer. If the touch trajectory coincides with the menu area, the display device accelerates the refresh display of the superimposed menu bitmap and line drawing bitmap through the first layer. This maintains the line drawing speed and ensures that the layer containing the menu bar is higher than the layer containing the line drawing content during the line drawing process, so as not to obscure the menu bar and improve the user experience.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 The diagram illustrates an operational scenario between a display device and a control device according to some embodiments;

[0024] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of a display device 200 according to some embodiments;

[0025] Figure 3 The diagram illustrates, by way of example, a software configuration schematic of a display device 200 according to some embodiments;

[0026] Figure 4 The diagram illustrates the display effect of a whiteboard application according to some embodiments;

[0027] Figure 5 The diagram illustrates a multi-layered display device 200 according to some embodiments;

[0028] Figure 6 The diagram above exemplifies a hierarchical display effect of a whiteboard application according to some embodiments;

[0029] Figure 7 The diagram above illustrates another layer of display effect for a whiteboard application according to some embodiments;

[0030] Figure 8 The diagram above illustrates another layer of display effect for a whiteboard application according to some embodiments;

[0031] Figure 9 The diagram illustrates a flow chart of a line drawing rendering method according to some embodiments;

[0032] Figure 10 The diagram above exemplarily illustrates the display effect of the area to be detected according to some embodiments;

[0033] Figure 11 The image above exemplifies a schematic diagram showing the effect of multiple bitmap overlays according to some embodiments;

[0034] Figure 12 The diagram illustrates another multi-bitmap overlay display effect according to some embodiments;

[0035] Figure 13 The diagram illustrates, for example, the display effect of a rendered drawing according to some embodiments. Detailed Implementation

[0036] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0037] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0038] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0039] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0040] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0041] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0042] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0043] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0044] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0045] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0046] Figure 2 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0047] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0048] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0049] In some embodiments, the display 260 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0050] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0051] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0052] In some embodiments, the user interface can be used to receive control signals from the control device 100 (e.g., an infrared remote control).

[0053] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0054] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0055] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0056] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0057] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0058] In some embodiments, the object can be any of the optional objects, such as a hyperlink, an icon, or other operable controls. Operations related to the selected object include: displaying links to hyperlinked pages, documents, images, etc., or performing operations corresponding to the program associated with the icon.

[0059] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0060] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0061] In some embodiments, a graphics processor is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, and the rendered objects are used to display on a monitor.

[0062] In some embodiments, the video processor is configured to receive external video signals and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signals, so as to obtain a signal that can be directly displayed or played on the display device 200.

[0063] In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a GUI signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0064] In some embodiments, the audio processor is configured to receive external audio signals, and according to the standard codec protocol of the input signals, perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing, to obtain a sound signal that can be played in a speaker.

[0065] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0066] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, converting information between its internal form and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0067] In some embodiments, the display device's system may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0068] See Figure 3 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0069] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0070] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0071] like Figure 3As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0072] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0073] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0074] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 3 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0075] In some embodiments, the hardware or software architecture can be based on the description in the above embodiments, and in some embodiments it can be based on other similar hardware or software architectures, as long as the technical solution of this application can be implemented.

[0076] In some embodiments, the display 260 in the display device 200 is configured to display a user interface, images, text, video, etc., and the controller 250 in the display device 200 is used to provide the user interface, images, text, video, etc. to the display 260. The controller 250 or the control device 100 can control the fixing component, thereby realizing the rotation of the display device 200 through the fixing component, so that the display device 200 can switch between landscape and portrait modes.

[0077] In some embodiments, the display device 200 can support touch interaction functions by adding a touch component. Typically, the touch component can be combined with the display 260 to form a touch screen. On the touch screen, users can input different control commands through touch operations. For example, users can input touch commands such as click, swipe, long press, double click, etc., and different touch commands can represent different control functions.

[0078] To achieve the different touch actions described above, the touch component can generate different electrical signals when the user inputs different touch actions, and send the generated electrical signals to the controller 250. The controller 250 can extract features from the received electrical signals, and then determine the control function to be executed by the user based on the extracted features.

[0079] For example, when a user taps any application icon in the application interface, the touch component senses the touch and generates an electrical signal. Upon receiving the signal, the controller 250 first determines the duration of the touch signal's corresponding voltage level. If the duration is less than a preset time threshold, it identifies the user's input as a tap command. The controller 250 then extracts the positional features of the generated electrical signal to determine the touch location. When the touch location is within the application icon's display area, it is determined that the user has input a tap command at the application icon location. Accordingly, the tap command is used to execute the function of running the corresponding application in the current scenario, thus the controller 250 can launch and run the corresponding application.

[0080] For example, when a user inputs a swipe gesture on the media asset display page, the touch component also sends the sensed electrical signal to the controller 250. The controller 250 first determines the duration of the signal corresponding to the touch gesture in the electrical signal. If the duration is greater than a preset time threshold, it then determines the positional change of the signal. Obviously, for interactive touch gestures, the position of the signal will change, thus determining that the user has input a swipe touch command. The controller 250 then determines the swiping direction of the swipe touch command based on the change in the position of the signal, and controls the page to be turned on the media asset display page to display more media asset options. Furthermore, the controller 250 can also extract features such as the swiping speed and swiping distance of the swipe touch command, and control the page turning according to the extracted features to achieve a responsive effect.

[0081] Similarly, for touch commands such as double-tap and long-press, the controller 250 can extract different features and determine the type of touch command based on these features, then execute the corresponding control function according to preset interaction rules. In some embodiments, the touch component also supports multi-touch, allowing users to input touch actions on the touchscreen using multiple fingers, such as multi-finger taps, multi-finger long presses, and multi-finger swipes.

[0082] The aforementioned touch actions can also be combined with specific applications to achieve specific functions. For example, when a user opens the "Whiteboard" application, the display 260 can display a drawing area. The user can draw specific touch action trajectories in the drawing area by sliding touch commands. The controller 250 then determines the touch action pattern based on the touch action detected by the touch component and controls the display 260 to display it in real time to meet the demonstration effect.

[0083] Figure 4 The diagram illustrates, for example, the display effect of a whiteboard application according to some embodiments. Figure 4 As shown, users can install drawing applications such as whiteboards through display device 200. When a user selects a whiteboard application on display device 200, the user controls display device 200 to launch the whiteboard application, which is then displayed on monitor 260. Figure 4 The application's touch interface allows users to write, draw lines, and perform other operations. The display device generates touch trajectories based on the user's touch actions, enabling users to conduct whiteboard presentations on the display device using the whiteboard application. The touch interface can also include toolbars and other menus. For example, the toolbar can include various tool controls, such as multiple pen controls for providing different pen styles, and an eraser control for erasing touch trajectories. It should be noted that the display device in this application refers not only to smart TVs but also to computers, tablets, etc.

[0084] Figure 5 The diagram illustrates a multi-layered display device 200 according to some embodiments. To achieve real-time display, the display device 200 can display the drawing process by overlaying multiple layers. Typically, the display device 200 can use one layer to display the user's touch trajectory in real time, and another layer to display the whiteboard interface; the final image displayed on the monitor 260 is composed of these two layers superimposed. For ease of distinction, as... Figure 5 As shown in this embodiment, the layer used to display the touch trajectory pattern in real time is called the first layer, which is a Group of Pictures (GOP) layer. The layer used to display the whiteboard interface is called the second layer, which is an on-screen display (OSD) layer. The GOP layer, also known as the native layer, GOP2 layer, or acceleration layer, can be used to display temporarily drawn content displayed on top of the menu. The OSD layer, also known as the display layer, intermediate layer, or menu layer, is used to display the application interface, application menu, toolbar, and other content.

[0085] Obviously, in order to present the final image, the display device 200 can display not only the two mentioned above, but also other layers to display different image content.

[0086] For example, the display device 200 may also include three layers: a first layer, a second layer, and a third layer, wherein the third layer is a video layer. The video layer, also known as the bottom layer, is generally used to display the image content corresponding to the external signal connected to the television.

[0087] Different layers can be hierarchically configured to achieve specific display effects. For example, the hierarchy of the GOP layer, OSD layer, and Video layer can be as follows: GOP layer - OSD layer - Video layer. That is, the Video layer is displayed at the bottom, showing the external signal screen content; the OSD layer is displayed above the Video layer, allowing the application menu to float above the external signal screen; and the GOP layer is displayed above the OSD layer so that it can be highlighted when the user inputs or draws graphics.

[0088] In some embodiments, the GOP layer, being used to display temporarily drawn content, causes the content displayed in the GOP layer to change depending on the user's drawing input. Therefore, in practical applications, to meet drawing requirements, after a swipe touch input is completed, the display device 200 can update the drawn pattern to the OSD layer for display and continue to display other touch trajectory content through the GOP layer.

[0089] In some embodiments, the GOP layer (first layer) is at a higher level than the OSD layer (second layer). The OSD layer is also used to display the menu bar, so during the drawing process, the drawn content is above the menu bar, and the drawn content occludes the menu bar. It should be noted that the menu bar displayed by the OSD layer here includes not only the menu bar in the current whiteboard application, but also other floating menus in the system, such as... Figure 5 The prompt message pop-up, clock control, etc. shown are examples of this. Therefore, during the drawing process, the drawn content can not only cover the toolbar and other menu bars in the whiteboard application, but also cover other high-level menu bars in the system, such as the floating menu bar. Figure 6 The diagram illustrates, exemplarily, the hierarchical display effect of a whiteboard application according to some embodiments. See also... Figure 6 The first line obscures the first menu bar, the second line obscures the second menu bar, and the third line obscures the third menu bar, resulting in a poor user experience.

[0090] like Figure 6 As shown, the menu bar in the whiteboard application has a first menu bar and a second menu bar with regular shapes, and a third menu bar with an irregular shape. The transparency of the menu bars can also be set, such as setting the first menu bar to have transparency. It should also be noted that for menu bars with regular shapes, the area containing the menu bar is the menu area; for menu bars with irregular shapes, the smallest regular area that can cover the menu bar is set as the menu area, such as... Figure 4 In the clock control, the area enclosed by the dashed line is the menu area of ​​the clock control. Here, "having a regular shape" can refer to a quadrilateral with opposite sides parallel to each other.

[0091] In related technologies, in order to prevent the drawn lines from obscuring the menu bar, the display device 200 controls the menu area to disable line drawing refresh during the line drawing process. That is, if the user's touch trajectory extends to the menu area, the touch trajectory of the menu area is prohibited from being displayed, so as to prevent the drawn lines formed by the touch trajectory from obscuring the menu area. Figure 7 The diagram illustrates another layer of display effect for a whiteboard application according to some embodiments. Figure 7 In the middle, the menu area of ​​the third menu bar is disabled to refresh the drawing line, which causes the touch trajectory extending to the menu area to no longer be displayed. However, the actual touch trajectory does not intersect with the third menu bar, so the user experience is still poor.

[0092] in addition, Figure 8 The diagram illustrates, exemplarily, another layer of display effect for a whiteboard application according to some embodiments. Figure 8In the process of drawing lines, the menu area adopts a layered refresh method. The non-menu area is refreshed and displayed with the help of the GOP layer, while the menu area switches back to the OSD layer for refresh and display. This method will cause a difference in the line drawing speed between the menu area and the non-menu area, which is particularly noticeable on 4K displays. There is also a noticeable gap difference at the junction of the menu area of ​​the irregular third menu bar, which also affects the user experience.

[0093] Here, it is possible to Figure 8 It can be seen that the first menu bar has its transparency set. If the menu bar has transparency, the line drawn below it will be visible. Figure 8 As shown, the first line located below the first menu bar is visible.

[0094] To address the aforementioned problems, this application provides a display device and a line drawing rendering method in some embodiments. This line drawing rendering method can be applied to a display device 200. To implement the line drawing rendering method, the display device 200 includes a display 260, a touch component, and a controller 250, wherein the touch component is configured to detect touch trajectories input by the user. The line drawing rendering process provided by the embodiments of this application will be described below with reference to the accompanying drawings.

[0095] Figure 9 The diagram illustrates a flow chart of a line drawing rendering method according to some embodiments. Combined with... Figure 9 The process of rendering the line drawing is as follows:

[0096] S901: In response to the input command to launch the whiteboard application, generate a line drawing bitmap for receiving the touch trajectory.

[0097] In some embodiments, a whiteboard application may be installed on the display device 200. The user can select the whiteboard application as needed to launch it. The user can also launch the desired application by browsing all applications on the display device 200 and clicking on its controls when the desired application is found.

[0098] In some embodiments, users can also launch the whiteboard application via voice control of the display device 200. For example, after the user enters the wake word "Hi! ××", they can input the voice command "Open whiteboard" into the display device 200, and the display device 200 will then launch the whiteboard application.

[0099] In some embodiments, after the whiteboard application is enabled, the display device 200 generates a line drawing bitmap for receiving the touch trajectory, and receives the touch trajectory drawn by the user in real time through the line drawing bitmap. In order to ensure that the display device 200 can respond to the user's touch trajectory as quickly as possible, the display device 200 can directly copy the touch trajectory on the line drawing bitmap to the first layer for refresh display.

[0100] S902: Determine whether the touch trajectory extends to the menu area.

[0101] In some embodiments, the display device 200 displays a menu bar on a second layer. Therefore, the display device 200 can obtain a view of the menu bar through the second layer to draw the menu area based on the view. In some embodiments, when a user touches the display, the touch component can detect the user's touch trajectory and feed back the current touch coordinates to the display device 200 at preset time intervals. The display device 200 then generates the user's touch trajectory based on the touch coordinates. Alternatively, the display device 200 can directly obtain the detected touch trajectory from the touch component in real time.

[0102] In some embodiments, after obtaining the touch coordinates, the display device 200 generates a touch trajectory based on the touch coordinates, and the display device 200 can further obtain multiple areas to be detected based on the touch trajectory. Figure 10 The diagram illustrates, exemplarily, the display effect of a region to be detected according to some embodiments. Combined with... Figure 10 Taking points A and B as an example, the first part of the touch trajectory is generated based on points A and B. Here, points A and B can be two consecutively reported touch coordinates, or coordinates optimized from two consecutively reported touch coordinates. Of course, points A and B can be consecutive adjacent points, or there can be several other touch coordinates between points A and B; this is not limited here. The display device 200 obtains a segment of touch trajectory based on points A and B, and further obtains a rectangular area with the smallest area that can cover this segment of touch trajectory as the detection area, i.e., the first part. Similarly, the second part, the third part, and the fourth part can be obtained.

[0103] The display device 200 can detect whether the vertices of each detection area fall into the menu area. If a vertex of the detection area falls into the menu area, the detection area and the menu area are considered to intersect. If a vertex of the detection area does not fall into the menu area, the detection area and the menu area are considered not to intersect. When the detection area and the menu area do not intersect, it is determined that the touch trajectory has not extended to the menu area. Figure 10 The first and second parts. When the area to be detected intersects with the menu area, it will be determined that the touch trajectory extends to the menu area, such as... Figure 10 Parts three and four.

[0104] When the display device 200 determines that the touch trace does not extend into the menu area, that is, when the display device 200 is processing... Figure 10 When performing the first and second parts of the process, the following step S903 is executed.

[0105] S903: Overlay the line drawing bitmap onto the first layer and refresh the display.

[0106] In some embodiments, the display device 200 can directly copy touch traces that do not extend to the menu area, i.e., touch traces located in non-menu areas, to the first layer, i.e., the GOP layer, for refresh and display. Through the GOP layer, the display device 200 can display content temporarily drawn on the bitmap and displayed above the menu, improving the display speed of touch traces and ensuring responsiveness.

[0107] When the display device 200 determines that the touch trajectory extends into the menu area, that is, when the display device 200 is processing... Figure 10 When performing the third and fourth parts of the process, the following steps S904 are executed.

[0108] S904: Overlay the line drawing bitmap and the pre-generated menu bitmap onto the first layer and refresh the display.

[0109] In some embodiments, the display device 200 may pre-create a blank bitmap to create a menu bitmap. The display device 200 first obtains a view of the menu bar in a second layer. After obtaining the menu bar view, the display device 200 can obtain data such as the specific position, width, and height of each menu bar on the display interface, so that the menu bar can be mapped to the blank bitmap according to the menu bar view. That is, the same menu bar as in the second layer is drawn on the blank bitmap to obtain the menu bitmap. After the menu bitmap is created, it can be directly retrieved and used when a menu bar needs to be overlaid and displayed subsequently.

[0110] In some embodiments, the menu bar may change due to user actions. For example, under user control, the tool menu bar containing tools may be hidden; or when the user finishes drawing, directly closing the window may trigger a message prompting whether to save. When the menu bar changes, a message indicating the change can be sent to the controller. Upon receiving the message, the controller can re-obtain the view of the menu bar from the second layer to update the menu bitmap.

[0111] In some embodiments, to prevent the touch trajectory from obscuring the menu bar, the display device 200 overlays a menu bitmap onto the touch trajectory extending to the menu area, ensuring that the menu bar's display layer is above the touch trajectory. Additionally, by overlaying the line drawing bitmap and the menu bitmap... Figure 1 Copying it to the first layer ensures the display speed of touch traces in the menu area.

[0112] Here, if there is an erasure trajectory drawn by the eraser control, in order to prevent the erasure trajectory from obscuring the menu bar, the display device 200 can draw an eraser bitmap based on the user's erasure trajectory and overlay a menu bitmap on the eraser bitmap.

[0113] Since the line drawing bitmap that needs to be copied to the first layer may or may not have a background, the following section, with reference to the attached diagram, introduces the overlay methods of different line drawing bitmaps.

[0114] Figure 11 The image illustrates, exemplarily, a schematic diagram of a multi-bitmap overlay display effect according to some embodiments. For example... Figure 11 As shown, the line drawing bitmap created by the display device 200 is a transparent bitmap, meaning it does not include a background. The background can be understood as the background style set by the user for the touch interface, such as a checkered pattern, a diagonal line pattern, or a red background, etc. The display device 200 can generate a background bitmap based on the background style set by the user for the line drawing bitmap. When the line drawing bitmap is transparent, the display device 200 first overlays the background bitmap onto the first layer, making it the bottom layer and covering the menu bar in the second layer, then overlays the line drawing bitmap, and finally the menu bitmap. If the menu bar has a transparency setting, its transparency is maintained during the overlay process.

[0115] Figure 12 The image above exemplifies another schematic diagram of a multi-bitmap overlay display effect according to some embodiments. For example... Figure 12 As shown, the line drawing bitmap created by display device 200 is a non-transparent bitmap, meaning it includes the background. Display device 200 can directly overlay the line drawing bitmap onto the first layer, and then overlay the menu bitmap. If the menu bar has a transparency setting, its transparency is maintained during the overlay process.

[0116] In some embodiments, after the user's touch trajectory is drawn onto the line drawing bitmap, the line drawing bitmap is directly copied to the first layer. Then, after the user raises their hand, the first layer can be undone, and the drawing content refreshed and displayed in the first layer is submitted to the second layer. When submitting the drawing content in the first layer to the second layer, it can be submitted in the form of a refreshed bitmap, that is, the refreshed bitmap contains the drawing content in the first layer, and the refreshed bitmap is submitted to the second layer.

[0117] In some embodiments, when there is already drawn content in the second layer, the display device 200 needs to obtain the refresh bitmap before overlaying the line drawing bitmap. When the touch trajectory extends to the menu area, the display device 200 sequentially overlays the line drawing bitmap, the refresh bitmap, and the menu bitmap onto the first layer to ensure that the touch trajectory being drawn by the user, the already drawn content, and the menu bitmap are overlaid, thus ensuring the integrity of the drawn content.

[0118] S905: After the current touch trajectory input is completed, refresh the drawing content in the first layer to the second layer.

[0119] S906: Controls the display to show the user's touch trajectory based on the content in the second layer.

[0120] In some embodiments, after the current touch trajectory is completed, i.e. after the user lifts their hand after a swipe touch, the display device 200 can refresh the drawing content in the first layer to the second layer, so that the display can make the final display based on the content in the second layer. Figure 13 The diagram illustrates, for example, the display effect of a rendered drawing according to some embodiments. Figure 13 As shown, the first, second, and third lines no longer obscure the corresponding menu bars, and for the first menu bar with transparency, the first line below it can be seen through.

[0121] In this application, if the touch trajectory does not coincide with the menu area during the drawing process, the display device accelerates the refresh display of the line drawing bitmap through the first layer. If the touch trajectory coincides with the menu area, the display device accelerates the refresh display of the superimposed menu bitmap and line drawing bitmap through the first layer. This maintains the line drawing speed and ensures that the layer containing the menu bar is higher than the layer containing the line drawing content during the line drawing process, thus preventing obstruction of the menu bar and improving the user experience.

[0122] Based on the same inventive concept as the aforementioned display device, this application also provides a line drawing rendering method in some embodiments. The method includes: a display device 200, in response to an input command to launch a whiteboard application, generating a line drawing bitmap for receiving touch traces. When the touch trace does not extend to the menu area, the display device 200 overlays the line drawing bitmap onto a first layer for refresh display, wherein the first layer is a layer for displaying the currently drawn content, and the menu area is an area containing a menu bar. When the touch trace extends to the menu area, the display device 200 sequentially overlays the line drawing bitmap and a pre-generated menu bitmap onto the first layer for refresh display, wherein the menu bitmap is a bitmap containing the menu bar generated based on the menu area. After the current touch trace input is completed, the display device 200 refreshes the drawn content in the first layer to a second layer, so that the display shows the user's touch trace based on the content in the second layer, wherein the second layer is a layer located below the first layer.

[0123] In some embodiments, after generating a plotted bitmap for receiving the touch trajectory, the method further includes: the display device 200 acquiring a view of the menu bar to draw the menu area based on the view. The display device 200 controls the real-time acquisition of the touch trajectory fed back by the touch component and detects the touch trajectory in units of a preset length. When the touch trajectory of the preset length does not intersect with the menu area, the display device 200 determines that the touch trajectory does not extend to the menu area. When the touch trajectory of the preset length intersects with the menu area, the display device 200 determines that the touch trajectory extends to the menu area.

[0124] In some embodiments, when the touch trajectory extends to the menu area, the step of sequentially overlaying the drawn line bitmap and the pre-generated menu bitmap onto the first layer for refresh display includes: when the drawn line bitmap is a transparent bitmap, the display device 200 sequentially overlays a background bitmap, the drawn line bitmap, and the menu bitmap onto the first layer, wherein the background bitmap is formed according to a background style set by the user for the drawn line bitmap. When the drawn line bitmap is a non-transparent bitmap, the display device 200 sequentially overlays the drawn line bitmap and the menu bitmap onto the first layer.

[0125] In some embodiments, when the touch trajectory extends to the menu area, the step of sequentially overlaying the drawn bitmap and the pre-generated menu bitmap onto the first layer for refresh display includes: when there is already drawn content in the second layer, the display device 200 obtains a refresh bitmap, wherein the refresh bitmap is used to submit the drawn content in the first layer to the second layer. When the touch trajectory extends to the menu area, the display device 200 sequentially overlays the drawn bitmap, the refresh bitmap, and the menu bitmap onto the first layer.

[0126] Since the above embodiments are all described in conjunction with other methods, and different embodiments have the same parts, the same or similar parts between the various embodiments in this specification can be referred to mutually. They will not be described in detail here.

[0127] It should be noted that in this specification, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a circuit structure, article, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a circuit structure, article, or device. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the circuit structure, article, or device that includes the element.

[0128] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the content of the claims.

[0129] The above embodiments of this application do not constitute a limitation on the scope of protection of this application.

Claims

1. A display device, characterized in that, The display device includes: monitor; The touch component is configured to detect the touch trajectory of user input; The controller is configured to: In response to an input command to launch the whiteboard application, a plotting bitmap for receiving the touch trajectory is generated; When the touch trajectory does not extend to the menu area, the drawing bitmap is overlaid on the first layer for refresh display, wherein the first layer is a layer used to display the currently drawn content, and the menu area is an area containing the menu bar; When the touch trajectory extends to the menu area, the drawn line bitmap and the pre-generated menu bitmap are sequentially superimposed on the first layer for refresh display, wherein the menu bitmap is a bitmap containing the menu bar generated based on the menu area; After the current touch trajectory input is completed, the drawing content in the first layer is refreshed to the second layer, so that the display shows the user's touch trajectory according to the content in the second layer, wherein the second layer is the layer located below the first layer.

2. The display device according to claim 1, characterized in that, After generating the plotted bitmap for receiving the touch trajectory, the controller is further configured to: Obtain a view of the menu bar, and draw the menu area based on the view; Receive touch coordinates reported by the touch component at preset intervals, generate touch trajectory based on the touch coordinates, and obtain the area to be detected; When the area to be detected does not intersect with the menu area, it will be determined that the touch trajectory does not extend to the menu area; When the area to be detected intersects with the menu area, it will be determined that the touch trajectory extends to the menu area.

3. The display device according to claim 1, characterized in that, In the step of refreshing the display by sequentially overlaying the drawn bitmap and the pre-generated menu bitmap onto the first layer when the touch trajectory extends to the menu area, the controller is further configured to: When the line drawing bitmap is a transparent bitmap, the background bitmap, the line drawing bitmap, and the menu bitmap are sequentially overlaid on the first layer, wherein the background bitmap is formed according to the background style set by the user for the line drawing bitmap; When the line drawing bitmap is a non-transparent bitmap, the line drawing bitmap and the menu bitmap are sequentially superimposed on the first layer.

4. The display device according to claim 1, characterized in that, In the step of refreshing the display by sequentially overlaying the drawn bitmap and the pre-generated menu bitmap onto the first layer when the touch trajectory extends to the menu area, the controller is further configured to: When there is already drawn content in the second layer, a refresh bitmap is obtained, wherein the refresh bitmap is used to submit the drawn content in the first layer to the second layer; When the touch trajectory extends to the menu area, the line drawing bitmap, the refresh bitmap, and the menu bitmap are sequentially superimposed onto the first layer.

5. The display device according to claim 1, characterized in that, The controller is also configured to: Create a blank bitmap and obtain the view of the menu bar; The menu bar is mapped to the blank bitmap according to the view of the menu bar to obtain the menu bitmap.

6. The display device according to claim 5, characterized in that, The controller is also configured to: In response to a received message indicating a change in the menu bar, the view of the menu bar is reacquired to update the menu bitmap.

7. A line drawing and rendering method, characterized in that, The method includes: In response to the input command to launch the whiteboard application, a line drawing bitmap is generated to receive touch trajectories; When the touch trajectory does not extend to the menu area, the drawing bitmap is overlaid on the first layer for refresh display, wherein the first layer is a layer used to display the currently drawn content, and the menu area is an area containing the menu bar; When the touch trajectory extends to the menu area, the drawn line bitmap and the pre-generated menu bitmap are sequentially superimposed on the first layer for refresh display, wherein the menu bitmap is a bitmap containing the menu bar generated based on the menu area; After the current touch trajectory input is completed, the drawing content in the first layer is refreshed to the second layer so that the display shows the user's touch trajectory based on the content in the second layer, wherein the second layer is the layer located below the first layer.

8. The line drawing and rendering method according to claim 7, characterized in that, After generating the plotted bitmap for receiving the touch trajectory, the method further includes: Obtain a view of the menu bar, and draw the menu area based on the view; Receive touch coordinates reported by the touch component at preset intervals, generate a touch trajectory based on the touch coordinates, and obtain the area to be detected; When the area to be detected does not intersect with the menu area, it will be determined that the touch trajectory does not extend to the menu area; When the area to be detected intersects with the menu area, it will be determined that the touch trajectory extends to the menu area.

9. The line drawing and rendering method according to claim 7, characterized in that, In the step of sequentially overlaying the drawn bitmap and the pre-generated menu bitmap onto the first layer for refresh display when the touch trajectory extends to the menu area, the method includes: When the line drawing bitmap is a transparent bitmap, the background bitmap, the line drawing bitmap, and the menu bitmap are sequentially overlaid on the first layer, wherein the background bitmap is formed according to the background style set by the user for the line drawing bitmap; When the line drawing bitmap is a non-transparent bitmap, the line drawing bitmap and the menu bitmap are sequentially superimposed on the first layer.

10. The line drawing and rendering method according to claim 7, characterized in that, In the step of sequentially overlaying the drawn bitmap and the pre-generated menu bitmap onto the first layer for refresh display when the touch trajectory extends to the menu area, the method includes: When there is already drawn content in the second layer, a refresh bitmap is obtained, wherein the refresh bitmap is used to submit the drawn content in the first layer to the second layer; When the touch trajectory extends to the menu area, the line drawing bitmap, the refresh bitmap, and the menu bitmap are sequentially superimposed onto the first layer.

Images