Display device, annotation method, electronic device, storage medium, and program product

Abstract

The application discloses a display device, a comment method, an electronic device, a storage medium and a program product, relates to the display technical field, and is used for covering the display content which does not need to be commented to a certain extent, focusing on the comment content, and improving the comment experience of a user. The display device comprises a display screen and a control circuit, wherein the display screen is configured to display content; the control circuit comprises a processor and a memory, the memory is used for storing a program executable by the processor, and the processor is used for reading the program in the memory and performing the following steps: in response to an operation instruction of a user in a comment mode, adding a mask layer to at least part of display content of a display interface; and in response to handwriting operation of the user on the mask layer, displaying handwriting corresponding to the handwriting operation.

Description

Technical Field

[0001] This invention relates to the field of display technology, and in particular to a display device, annotation method, electronic device, storage medium, and program product. Background Technology

[0002] Meetings are one of the most common forms of work, but with the continuous development of the times and the continuous advancement of technology, traditional meeting models are no longer suitable for the needs of modern offices. Meeting efficiency and interactive experience are becoming increasingly important in corporate meetings. This has led to the development of smart meetings, which are office products that integrate multiple devices such as electronic whiteboards, computers, and remote conferencing terminals, and possess various functions including interactive smart whiteboards.

[0003] As a large-screen display device, conference all-in-one machines often require annotation in meeting scenarios. When users use the handwriting function to annotate on the display device, the readability of the handwritten annotations is low due to the cluttered display content. When users want to focus on a specific display content for annotation, the annotation content is scattered, resulting in a poor user experience. Summary of the Invention

[0004] This invention provides a display device, annotation method, electronic device, storage medium, and program product for partially obscuring display content that does not require annotation, focusing on the annotation content, and improving the user's annotation experience.

[0005] In a first aspect, an embodiment of the present invention provides a display device, comprising a display screen and a control circuit, wherein:

[0006] The display screen is configured to display content;

[0007] The control circuit includes a processor and a memory. The memory stores programs executable by the processor, and the processor reads the programs from the memory and performs the following steps:

[0008] In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface;

[0009] In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

[0010] As an optional implementation, the operation instructions include a user's touch operation on a designated button on the display interface; the processor is specifically configured to execute:

[0011] In response to user touch operations on the display interface, a masking layer is added to all displayed content on the display interface.

[0012] As an optional implementation, the operation instructions include user marking operations on the displayed content; the processor is specifically configured to execute:

[0013] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation;

[0014] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

[0015] As an optional implementation, the operation instructions include handwriting operations performed by the user on the display interface; the processor is further configured to execute:

[0016] In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation.

[0017] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

[0018] As an optional implementation, before adding a masking layer to at least a portion of the displayed content of the display interface, the processor is further configured to perform:

[0019] A comment menu is displayed on the display interface, and the comment menu includes at least one menu item;

[0020] In response to the user's touch command on the menu item, determine the third display content selected by the user.

[0021] As an optional implementation, the annotation menu includes a first menu item; the processor is specifically configured to execute:

[0022] In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas;

[0023] In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

[0024] As an optional implementation, the annotation menu further includes a second menu item; the processor is specifically configured to execute:

[0025] In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0026] As an optional implementation, the annotation menu includes a third menu item; the processor is specifically configured to execute:

[0027] In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content;

[0028] The third display content is determined based on the position information of the selection operation.

[0029] As an optional implementation, the processor is further configured to execute:

[0030] Receive multiple selections of the displayed content from the user;

[0031] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0032] As an optional implementation, the selection operation includes multiple selections; the processor is further configured to execute:

[0033] If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

[0034] As an optional implementation, the selection operation includes multiple selections; the processor is further configured to execute:

[0035] Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations;

[0036] In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

[0037] As an optional implementation, the processor is specifically configured to execute:

[0038] The closed region is determined based on the position information of the selection operation;

[0039] When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0040] As an optional implementation, the processor is further configured to execute:

[0041] In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

[0042] As an optional implementation, the processor is further configured to execute:

[0043] A mask layer will be added to at least part of the displayed content except for the third displayed content.

[0044] As an optional implementation, the content displayed under the mask layer is visible; the processor is further configured to execute:

[0045] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized.

[0046] When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

[0047] As an optional implementation, the processor is further configured to execute:

[0048] In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display.

[0049] A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

[0050] As an optional implementation, the display interface also includes historical annotations, and the processor is specifically configured to execute:

[0051] Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or,

[0052] Add a masking layer to at least part of the displayed content, excluding the content bound to historical annotations.

[0053] As an optional implementation, the display interface further includes annotation content, which is bound to the display content of the display interface, and the processor is specifically configured to execute:

[0054] Determine the fourth display content in the display interface;

[0055] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0056] As an optional implementation, in response to user operation commands in annotation mode, the processor is further configured to execute:

[0057] Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content;

[0058] In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

[0059] As an optional implementation, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0060] Secondly, an annotation method is provided in an embodiment of the present invention, the method comprising:

[0061] In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface;

[0062] In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

[0063] As an optional implementation, the operation instructions include user touch operations on designated buttons on the display interface; in response to user operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content on the display interface, including:

[0064] In response to user touch operations on the display interface, a masking layer is added to all displayed content on the display interface.

[0065] As an optional implementation, the operation instructions include user marking operations on the displayed content; in response to the user's operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content of the display interface, including:

[0066] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation;

[0067] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

[0068] As an optional implementation, the operation instructions include handwriting operations by the user on the display interface; in response to the user's operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content on the display interface, further comprising:

[0069] In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation.

[0070] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

[0071] As an optional implementation, before adding a masking layer to at least a portion of the displayed content of the display interface, the method further includes:

[0072] A comment menu is displayed on the display interface, and the comment menu includes at least one menu item;

[0073] In response to the user's touch command on the menu item, determine the third display content selected by the user.

[0074] As an optional implementation, the annotation menu includes a first menu item; in response to a user's touch command on the menu item, determining the third display content selected by the user includes:

[0075] In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas;

[0076] In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

[0077] As an optional implementation, the annotation menu further includes a second menu item; the method further includes:

[0078] In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0079] As an optional implementation, the annotation menu includes a third menu item; in response to a user's touch command on the menu item, determining the third display content selected by the user includes:

[0080] In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content;

[0081] The third display content is determined based on the position information of the selection operation.

[0082] As an optional implementation, determining the third display content based on the position information of the selection operation further includes:

[0083] Receive multiple selections of the displayed content from the user;

[0084] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0085] As an optional implementation, the selection operation includes multiple selections; determining the third display content based on the position information of the selection operation further includes:

[0086] If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

[0087] As an optional implementation, the selection operation includes multiple selections; determining the third display content based on the position information of the selection operation further includes:

[0088] Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations;

[0089] In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

[0090] As an optional implementation, determining the third display content based on the position information of the selection operation includes:

[0091] The closed region is determined based on the position information of the selection operation;

[0092] When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0093] As an optional implementation, the method further includes:

[0094] In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

[0095] As an optional implementation, the method further includes:

[0096] A mask layer will be added to at least part of the displayed content except for the third displayed content.

[0097] As an optional implementation, the content displayed under the mask layer is visible; the method further includes:

[0098] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized.

[0099] When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

[0100] As an optional implementation, the method further includes:

[0101] In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display.

[0102] A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

[0103] As an optional implementation, the display interface further includes historical annotation content, and a masking layer is added to at least a portion of the displayed content of the display interface, including:

[0104] Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or,

[0105] Add a masking layer to at least part of the displayed content, excluding the content bound to historical annotations.

[0106] As an optional implementation, the display interface further includes annotation content, which is bound to the display content of the display interface. The method further includes:

[0107] Determine the fourth display content in the display interface;

[0108] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0109] As an optional implementation, in response to user operation commands in annotation mode, the method further includes:

[0110] Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content;

[0111] In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

[0112] As an optional implementation, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0113] Thirdly, embodiments of the present invention also provide an electronic device, which includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0114] In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface;

[0115] In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

[0116] Fourthly, embodiments of the present invention also provide an annotation device, the device comprising:

[0117] The overlay module is used to add an overlay layer to at least a portion of the displayed content of the display interface in response to user operation commands in annotation mode.

[0118] The display module is used to respond to the user's handwriting operation on the mask layer and display the handwriting corresponding to the handwriting operation.

[0119] Fifthly, embodiments of the present invention also provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the method described in any of the second aspects above.

[0120] In a sixth aspect, this application provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of the second aspects.

[0121] These or other aspects of this application will become more apparent in the following description of embodiments. Attached Figure Description

[0122] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0123] Figure 1 This is a schematic diagram of the structure of a display device provided in an embodiment of the present invention;

[0124] Figures 2A-2E This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0125] Figures 3A-3D This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0126] Figures 4A-4C This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0127] Figure 5A-5G This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0128] Figures 6A-6E This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0129] Figures 7A-7B This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0130] Figures 8A-8B This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0131] Figures 9A-9E This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0132] Figures 10A-10C This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0133] Figure 11A-11F This is a schematic diagram of a display interface in annotation mode provided by an embodiment of the present invention;

[0134] Figure 12 A flowchart illustrating the implementation of an annotation method provided in an embodiment of the present invention;

[0135] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0136] Figure 14 This is a schematic diagram of an annotation device provided in an embodiment of the present invention. Detailed Implementation

[0137] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0138] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0139] The application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems. In the description of this invention, unless otherwise stated, "multiple" means two or more.

[0140] Before introducing the display device and annotation method provided in the embodiments of this application, the technical background of the embodiments of this application will be described in detail below for ease of understanding.

[0141] As a large-screen display device, conference all-in-one machines often require annotation in meeting scenarios. When users use the handwriting function to annotate on the display device, the readability of the handwritten annotations is low due to the cluttered display content. When users want to focus on a specific display content for annotation, the annotation content is scattered, resulting in a poor user experience.

[0142] To address the aforementioned technical problems, this application provides a display device and annotation method. The core idea is to focus on the content the user wants to annotate by adding a masking layer to at least part of the displayed content when the user is making annotations. This makes the content annotated by the user on the masking layer clearer, improves visibility, and effectively enhances the user's meeting experience.

[0143] It should be noted that the display device in this embodiment refers to a large-size (generally 50 inches or more) intelligent interactive display device with touch handwriting function; the display device is equipped with writing programs such as electronic whiteboard, and users can write on the display device using their fingers, active pen or passive pen, etc. The display device may include display modules such as liquid crystal display (LCD), organic light-emitting semiconductor (OLED), electronic ink, etc., and can display the handwriting in real time.

[0144] Reference Figure 1 The display device in this embodiment will be described.

[0145] The display device 100 in this embodiment includes a display screen 1040, a processor 1080, and a memory 1020. The display screen 1040 includes a display panel 1041, which is used to display information input by the user or information provided to the user, as well as various operation interfaces of the application. In this embodiment, it is mainly used to display the interface, shortcut window, three-dimensional menu model, menu information of the menu item of the client installed in the display device 100.

[0146] Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0147] The processor 1080 is used to read a computer program and then execute the methods defined by the computer program, such as the processor 1080 reading an application, thereby running the application on the display device 100 and displaying the application's interface on the display screen 1040. The processor 1080 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this disclosure.

[0148] The memory 1020 generally includes main memory and secondary storage. The main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. The secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 1020 is used to store computer programs and other data. The computer programs include client-side applications, etc. Other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, program instructions are stored in the memory 1020, and the processor 1080 executes the program instructions in the memory 1020 to implement any of the three-dimensional menu display methods provided in this disclosure.

[0149] In addition, the display device 100 may also include a touch unit 1100 for receiving input digital information, word information, or contact touch operations or non-contact gestures, as well as generating signal inputs related to user settings and function control of the display device 100. The touch unit 1100 includes, but is not limited to, infrared touch units, capacitive touch units, electromagnetic touch units, and camera acquisition units, wherein the camera acquisition unit is used to capture gestures of the user that are not touching the display screen. When the touch unit 1100 includes an infrared touch unit or an electromagnetic touch unit, the touch unit 1100 and the display screen 1040 can be stacked. For example, when a user performs a touch operation on the touch screen, the touch unit 1100 can collect the user's touch operation on or near it (such as the user's operation on the display panel 1041 using a finger, stylus, or any suitable object or accessory), and drive the corresponding connection device according to a pre-set program.

[0150] Optionally, the touch unit 1100 may include two parts: a touch detection device and a touch processor. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch processor. The touch processor receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080. It can also receive and execute commands from the processor 1080. In this embodiment, if the user clicks on an application, the touch detection device in the touch unit 1100 detects a touch operation and sends the signal corresponding to the detected touch operation to the touch processor. The touch processor converts the signal into touch point coordinates and sends it to the processor 1080. The processor 1080 determines the operation the user needs to perform based on the received touch point coordinates.

[0151] The display panel 1041 can be implemented using various types such as LCD and OLED. In addition to the display screen 1040 and the touch unit 1100, the display device 100 may also include an input unit 1030. The input unit 1030 may include an image input device 1031 and other input devices 1032. The other input devices 1032 may be, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0152] In addition to the above, the display device 100 may also include a power supply 1090 for powering other modules, an audio circuit 1060, a near-field communication module 1070, and an RF circuit 1010. The display device 100 may also include one or more sensors 1050, such as an accelerometer, a light sensor, a pressure sensor, etc. The audio circuit 1060 specifically includes a speaker 1061 and a microphone 1062, for example, the display device 100 can use the microphone 1062 to collect the user's voice and perform corresponding operations.

[0153] As one embodiment, the number of processors 1080 can be one or more, and the processors 1080 and memory 1020 can be coupled together or relatively independent.

[0154] As one embodiment, the processor 1080 is used to perform the following steps:

[0155] In response to user operation commands in annotation mode, a mask layer is added to at least part of the displayed content of the display interface; in response to user handwriting operation on the mask layer, the handwriting corresponding to the handwriting operation is displayed.

[0156] Optionally, the masking layer in this embodiment includes, but is not limited to, overlays or canvases. The masking layer is used to cover areas of the display screen that are not of primary interest to the user, thereby focusing attention on the areas of primary interest. Masking layers can be implemented in different ways, such as using overlays or canvases. The transparency of the masking layer in this embodiment is adjustable. An overlay is a common interface design element, typically referring to a semi-transparent (or multi-effect) overlay placed on top of a page or component to achieve specific interactive or visual effects. By reducing the visibility of background content (such as a semi-transparent overlay), overlays guide the user's attention to the core area (such as the content the user wants to annotate). When an overlay is placed on the page, it can also prevent the user from interacting with the underlying content, forcing the completion of the current handwriting operation (handwriting annotation). The canvas is the core interactive area for handwriting input. It simulates the writing experience of real paper, allowing users to write directly on the screen with their fingers or styluses. The system captures the strokes in real time and renders them. It supports an infinitely expandable writing space, breaking through the limitations of the physical screen. It allows switching between opaque display modes such as black backgrounds. The Android system uses Canvas objects to encapsulate the graphics buffer, combined with Surface to achieve layer rendering. Layer compositing is managed through SurfaceFlinger, converting handwriting strokes into graphics buffer data to achieve low-latency handwriting rendering. Both the mask and the canvas are independent units in the graphics system's hierarchical structure. The canvas is the core drawing area for handwriting, and content rendering is achieved through the graphics buffer. The mask is a temporary layer overlaid on the canvas or other UI elements for interactive isolation or visual assistance.

[0157] Optionally, the transparency of the masking layer is greater than 0 and less than or equal to 1. A transparency of 0 indicates complete transparency, and a transparency of 1 indicates complete opacity. In this embodiment, the masking layer is not completely transparent; it can be a masking layer with some transparency or a completely opaque masking layer. In this embodiment, the masking layer is used to cover or obscure content that the user is not annotating, so that the user is not disturbed by other content during the annotation process. This allows the annotation process to focus on the currently desired content, and the handwriting information of the annotation can be displayed more clearly on the display interface without interference from other content.

[0158] In implementation, in annotation mode, users can select some or all of the displayed content, add a mask layer to at least part of the displayed content, focus on displaying the content that the user wants to annotate, and perform handwriting operations on the mask layer. By displaying the user's handwriting on the mask layer, the user's annotation content and the displayed content are isolated to a certain extent, which can effectively improve the user experience and thus improve meeting efficiency.

[0159] In some embodiments, the operation instructions in this embodiment include touch instructions from the user on a designated button on the display interface; in response to the user's touch operation on the display interface, a masking layer is added to all display content of the display interface.

[0160] Optionally, the touch operation can be a touch operation on a designated button, a gesture operation, or a click operation, etc. Optionally, in response to the user's touch operation on a designated button on the display interface, a mask layer is added to the entire display content of the display interface. The designated button can be a virtual button or a real button on the display device. This embodiment does not impose too many limitations on this.

[0161] In some embodiments, if the display interface includes historical annotations, a masking layer may be added to at least a portion of the display content other than the historical annotations.

[0162] In practice, when there are already annotations (i.e., historical annotations) on the display interface, when the user wants to annotate again, the overlay will not cover the historical annotations, but will display the historical annotations, and the new annotations will continue to be displayed on the overlay.

[0163] like Figures 2A-2E As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 2A In annotation mode, the initial state displays all content by default. Users can click the "Annotate" button in the menu to jump to... Figure 2B The display interface adds a black, opaque overlay layer to the displayed content. The default transparency can be 80%. See [link / reference]. Figure 2C Users can perform handwriting operations on the overlay layer, and the corresponding handwriting will be displayed as the user's annotations. See also Figure 2D If the user's handwritten handwriting is already displayed on the screen, the user will be redirected to [the appropriate page / section] after clicking the "Annotate" button in the menu. Figure 2E On the displayed interface, the handwritten handwriting is still displayed normally; that is, the overlay layer does not cover the original handwritten handwriting. When the user clicks the "Annotate" button again, the overlay layer is turned off, and the display returns to the original screen mode.

[0164] Optionally, the transparency of the mask layer in this embodiment can be 100%, meaning the mask layer can be completely opaque, or it can be within a preset transparency range, such as 30% to 80%. Users can set this, or it can be automatically adjusted according to the complexity of the currently displayed content. This embodiment does not impose excessive limitations on this.

[0165] In some embodiments, the operation instructions in this embodiment include user marking operations on the displayed content; the processor is specifically configured to execute:

[0166] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation; a masking layer is added to at least a portion of the displayed content in the display interface other than the first displayed content.

[0167] Optionally, the marking operation in this embodiment is used to mark the displayed content, indicating the content that the user wants to annotate from the displayed content of the display interface. The marking operation includes, but is not limited to, at least one of the following: drawing a graphic, circling. The drawn graphic includes, but is not limited to, at least one of the following: a straight line, a wavy line, a dashed line, a triangle, a circle, a rectangle, and a polygon. For example, when the user draws a horizontal line on the display interface, the displayed content above or below the horizontal line is determined as the displayed content that the user wants to annotate, i.e., the first displayed content. When the user draws a vertical line on the display interface, the displayed content to the left or right of the vertical line is determined as the displayed content that the user wants to annotate, i.e., the first displayed content. If there is no displayed content within the area of ​​the graphic drawn by the user, for example, if the user draws a triangle or circle below some displayed content, the displayed content closest to the graphic (e.g., when the distance between the graphic and the graphic is within a set pixel range) can be determined as the displayed content that the user wants to annotate, i.e., the first displayed content selected by the user. If the area within the user-drawn graphic contains display content, it indicates that the user is performing a selection operation. By drawing a closed path, the user selects the content they want to annotate, and the content within that graphic is then designated as the first content selected by the user. Optionally, the size of the area of ​​the closed path drawn by the user determines whether the current operation is a selection or marking of display content using a graphic.

[0168] In some embodiments, the first display content is determined by the following steps:

[0169] Step a: Determine the content areas to be displayed;

[0170] In implementation, the distance between pixels within the same content area is less than a pixel threshold, while the distance between pixels in different content areas exceeds this threshold. For example, if different content areas are blank, the distance between the outermost pixels of different content areas will be larger, while the distance between pixels within the same content area will be smaller. The distance between pixels can be used to determine which pixels belong to the same content area and which belong to different content areas. Optionally, AI-based recognition can be used to partition the displayed content by judging the distance between pixels against the pixel threshold, resulting in one or more content areas.

[0171] Step b: Based on the position information of the first handwritten stroke corresponding to the marking operation, select the content area that is closest to the first handwritten stroke from each content area of ​​the displayed content;

[0172] In practice, by calculating the distance between pixels, the content area closest to the position of the first handwritten stroke is determined and used as the first displayed content.

[0173] Optionally, based on the position information of the first handwritten stroke, the content area closest to the position of the stroke can be determined and used as the first displayed content.

[0174] Step c: Select the filtered content area as the first content to be displayed.

[0175] In some embodiments, the first display content can also be determined by performing graphic recognition on the first handwritten handwriting corresponding to the marking operation and based on the position information of the graphic recognition result.

[0176] In implementation, the position of the first displayed content can be set differently for different graphic positions. For example, when the recognition result is a horizontal line, the content area above the horizontal line and closest to it is determined as the first displayed content; when the recognition result is a vertical line, the content area to the left of the vertical line and closest to it is determined as the first displayed content.

[0177] like Figures 3A-3D As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figures 3A-3B The user draws a horizontal line on the display interface, and the area of ​​content closest to the line is designated as the primary display content. Other displayed content is then overlaid with a mask layer. See also Figure 3C-3D Users draw vertical lines on the display interface, and the content area closest to the left of the vertical line is determined as the first displayed content, while other displayed content is covered by a mask layer.

[0178] In some embodiments, after determining the first display content, the first display content is highlighted.

[0179] In some embodiments, the operation instructions of this embodiment include a user's handwriting operation on the display interface; in response to the user's handwriting operation on the display interface, determining the second display content based on the position information of the second handwriting corresponding to the handwriting operation; and adding a masking layer to at least a portion of the display content on the display interface other than the second display content and the second handwriting.

[0180] In practice, users can directly perform handwriting operations on the display interface. When the second handwriting corresponding to the user's handwriting operation is recognized as text, at least one character can be recognized, such as the first character. When the user handwrites the first character of the annotation on the display interface, the position information of the second handwriting is determined, that is, the position information of the first character. The display content closest to the position information is determined as the second display content. Then, a mask layer is added to the display interface in addition to the second display content and the first character handwritten by the user. The user can continue to handwrite on the mask layer to complete the annotation content for the second display content.

[0181] In some embodiments, the second display content is determined by the following steps:

[0182] Step a: Determine the content areas to be displayed;

[0183] In implementation, the distance between pixels within the same content area is less than a pixel threshold, while the distance between pixels in different content areas exceeds this threshold. For example, if different content areas are empty, the distance between the outermost pixels of different content areas will be larger, while the distance between pixels within the same content area will be smaller. The distance between pixels can be used to determine which pixels belong to the same content area and which belong to different content areas. Optionally, by determining the relationship between the pixel distance and the pixel threshold, the displayed content can be partitioned to obtain one or more content areas.

[0184] Step b: Based on the position information of the second handwritten stroke corresponding to the handwriting operation, select the content area that is closest to the second handwritten stroke from each content area of ​​the displayed content;

[0185] In practice, by calculating the distance between pixels, the content area closest to the position of the second handwritten stroke is determined and used as the second display content.

[0186] Optionally, based on the position information of the pen stroke in the second handwriting, the content area closest to the pen stroke position is determined and used as the second display content.

[0187] Step c: Select the filtered content area as the second display content.

[0188] Optionally, the second display content can be determined by performing character recognition on the second handwriting corresponding to the handwriting operation and determining the position of the character recognition result.

[0189] like Figures 4A-4C As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 4AThe user handwrites at least one item from the annotation on the display interface. The handwriting is then recognized using methods such as OCR. When the recognition result is text, the closest matching content area is selected from the displayed content based on the location of the handwriting. This selected content area is then used as the second displayed content. (See [link to relevant documentation]). Figure 4B Add a masking layer to at least part of the displayed content other than the second displayed content and the second handwritten handwriting, see [link to documentation]. Figure 4C Users can continue to perform handwriting operations on the overlay layer and continue to annotate the second displayed content.

[0190] In some embodiments, after the second display content is determined, the second display content is highlighted.

[0191] In this embodiment, after enabling annotation mode on the display device, all current content is displayed by default. The position of each user's pen stroke is determined. If the user's handwriting is a line or circle, an overlay layer is automatically added to the display area outside the line or circle. If the user's handwriting is text, the area to the left of the pen stroke closest to the handwriting is automatically highlighted, and the rest of the page display is automatically overlaid. It should be noted that if the user's handwriting is text, only the position of the first handwriting stroke can be determined, and the area to the left of the pen stroke closest to the handwriting is automatically highlighted, while the rest of the page display is automatically overlaid. The initial default state can also be restored after erasing all handwriting or clearing the screen.

[0192] In some embodiments, before adding a masking layer to at least a portion of the displayed content of the display interface, this embodiment may also display an annotation menu on the display interface, the annotation menu including at least one menu item; in response to the user's touch command on the menu item, a third display content selected by the user is determined.

[0193] In practice, after the annotation mode is activated, an annotation menu is displayed on the screen. Users can select the third content to annotate by clicking on the menu items in the annotation menu.

[0194] In some embodiments, the annotation menu includes a first menu item; in response to a user's touch command on the first menu item, the display content of the display interface can be divided into multiple content areas; in response to a user's second selection operation on the multiple content areas, a third display content corresponding to the second selection operation can be determined.

[0195] In practice, users can click on menu items to first divide the entire display content of the interface into sections, grouping content belonging to the same section together. Users can then select the content area they want to annotate as the third display content from each of the divided content areas.

[0196] Optionally, after dividing the display content into multiple content areas, a first icon can be displayed in each content area. When the user clicks the first icon, the content area containing the first icon is designated as the third display content. Optionally, the third display content can also be highlighted.

[0197] In some embodiments, the annotation menu further includes a second menu item; in response to a user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0198] In implementation, when a user clicks the first menu item, the displayed content is divided according to the default pixel threshold. If the user feels that the currently divided content areas are not suitable, they can redivide the content. At this time, when the user clicks the second menu item, a pixel threshold is randomly generated from the preset pixel range. Based on this pixel threshold, the displayed content of the interface is divided into areas. The principle of division is based on the distance between pixels. By comparing the distance between pixels with the pixel threshold, two pixels smaller than the pixel threshold are divided into the same area, and two pixels larger than the pixel threshold are divided into different areas.

[0199] Optionally, this embodiment can automatically partition the displayed content using a layout analysis algorithm. As a key technology for digital content processing, the layout analysis algorithm is mainly used to analyze the layout structure in documents, web pages, or images, identify the spatial relationships and logical levels of different content modules (such as text, images, tables, etc.), and thus achieve automated typesetting, content reorganization, or interactive optimization. Among them, the layout analysis algorithm is mainly implemented in the following ways: (1) Visual feature extraction: using computer vision technology (such as edge detection, region segmentation) to identify the boundaries of visual elements in the layout and distinguish the physical positions of modules such as titles, body text, and illustrations. For example, the image processing tools in MATLAB can adjust the auxiliary element segmentation through the Alpha channel. (2) Semantic association analysis: combining natural language processing (NLP) technology to analyze the semantic association of text content and optimize content recommendation and arrangement order. For example, the algorithm recommendation service needs to balance user preferences and content compliance to avoid the information cocoon effect. (3) Dynamic layout optimization: adopting the responsive design principle, dynamically adjusting the element layout according to the device screen size. Data visualization tools (such as D3.js) use SVG (Scalable Vector Graphics) rendering technology to achieve adaptive arrangement of complex charts. Optionally, layout analysis algorithms include, but are not limited to, any one or more of the following: a) segmenting layout elements using image processing techniques (such as black-and-white connected component partitioning). Classic algorithms such as the Docstrum algorithm divide images into text, text lines, and text blocks from the bottom up, suitable for simple document layouts; b) Detect table lines using operations such as erosion and dilation, and reconstruct the table structure by combining row and column region division; c) Classification models encode the text coordinates and content of the OCR output, and predict the module category using classifiers such as XGBoost; d) Sequence generation models treat layout analysis as a translation task, and generate structured output through Seq2Seq (sequence to sequence) models to solve complex layout logic relationships; e) Graph Neural Networks (GNNs) improve the parsing accuracy of complex layouts such as tables and multi-column text by modeling the spatial relationships between elements (such as relative position and semantic association); f) Attention mechanism fusion combines visual features and text semantics, and uses attention mechanisms to optimize cross-modal information alignment, enhancing the parsing capability of mixed text and image scenarios.

[0200] Optionally, after selecting one or more content areas, the user can annotate each content area individually. When annotating individually, the currently selected content area (i.e., the third display content) is highlighted, and at least part of the display content on the display interface other than the currently selected content area (i.e., the third display content) is covered with a mask layer.

[0201] Optionally, after a user selects a content area, that area is displayed as the third display content. The user can then directly perform handwriting operations outside this third display content, and the handwritten annotations will be displayed. In other words, the user can directly annotate the selected third display content without an overlay. When the user deems it unnecessary to add an overlay for annotation, they can directly write on the display interface after selecting the content area, and the corresponding handwritten annotations will be displayed in real time.

[0202] like Figure 5A-5G As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 5A The user clicks the first menu item in the annotation menu, and then see... Figure 5B The display interface is divided into multiple content areas. Each content area has a first icon in its lower right corner. Users can select the first icon to confirm the content they want to annotate. The content area containing the selected first icon will then be designated as the third display area. See [link to relevant documentation]. Figure 5C When a user selects a primary icon, they enter the annotation interface for that content area. The content area containing that primary icon is then highlighted, and a mask layer is added over all displayed content except for the third displayed content. The mask layer has 100% transparency. (See also...) Figure 5D It can also display a zoomed-out view of all content on the overlay, allowing users to write annotations on the overlay. When the user clicks on the zoomed-out view, it returns to the original default image (i.e., returns to the previous view). Figure 5B (Show all content), and users can then select another content area as the third display area for annotation. See also Figure 5E When the user clicks the second menu item, they are redirected to... Figure 5F The display interface is randomly grouped according to the distance between pixels, and the displayed content is re-divided into multiple content areas. A first icon is displayed in each content area. The user selects the first icon to enter the annotation interface for that content area. (See [link]). Figure 5G At this point, the content area selected by the user is highlighted, while the remaining areas are covered by a mask layer, allowing the user to write annotations on the mask layer.

[0203] In some embodiments, the annotation menu includes a third menu item; in response to a user's touch command on the third menu item, the system receives a selection operation of the user on the displayed content; and determines the third displayed content based on the position information of the selection operation.

[0204] In implementation, after clicking the third menu item, the displayed content is manually divided into sections, and the third display content to be annotated is selected by circling. Optionally, the circling operation includes, but is not limited to, drawing a closed path. The user uses the handwriting function to draw a closed path on the displayed content, and the displayed content circled within the closed path is determined as the third display content.

[0205] In some embodiments, this embodiment can also display multiple display contents through multiple selection operations, thereby simultaneously annotating multiple third display contents. The specific implementation steps are as follows:

[0206] Receive multiple selections of the displayed content from the user;

[0207] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0208] In implementation, users can select multiple third-party display items by making multiple selections on the displayed content. Then, a mask layer is added to at least a portion of the displayed content other than the selected third-party items. Users can then annotate the selected third-party items on this mask layer. When users annotate multiple third-party items, the annotation content bound to each third-party item is determined. If there is overlap between annotation content bound to different third-party items, the position of the annotation content can be automatically adjusted. Similarly, if annotation content bound to different third-party items overlaps with any other third-party item, the position of the annotation content can be automatically adjusted.

[0209] In some embodiments, a closed region is determined based on the position information of the selection operation; when the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0210] In practice, after receiving the user's selection operation, the selection operation is tracked. If the identification result is a closed path and the area of ​​the closed region determined by the closed path is greater than a threshold, the content displayed within the closed region is determined as the third display content.

[0211] Optionally, after determining the third display content based on the position information of the selection operation, the trajectory information corresponding to the selection operation is displayed, and a second icon is displayed within the closed area determined by the trajectory information. In response to the user's touch operation on the second icon, a masking layer is added to the display area other than the closed area.

[0212] Optionally, after determining the content of the closed area as the third display content, the system can also respond to user handwriting operations in areas other than the third display content by displaying the corresponding handwriting. In practice, users can directly annotate and display their handwriting on the display interface after determining the third display content, without adding a masking layer.

[0213] like Figures 6A-6E As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 6A When the user clicks the third menu item, they are redirected to... Figure 6BThe display interface allows users to select content and draw a closed path. A second icon is displayed within the closed area. Clicking the second icon redirects the user to... Figure 6C The interface shown highlights the closed area containing the second icon, while an overlay is added to the area outside this closed area, allowing the user to write annotations on the overlay. See also Figure 6D It can also display a zoomed-out view of all content on the overlay, and when the user clicks the zoom view, it returns to the previous state. Figure 6B The interface shown is the one displaying all content. See also... Figure 6E Users can also Figure 6B The interface shown allows for direct handwritten annotations and displays the handwriting information of the annotations.

[0214] In some embodiments, this embodiment can perform multiple selection operations, the selection operation including multiple selections; determine the third display content corresponding to each of the multiple selection operations based on the position information of each selection operation; in response to the user's selection operation of any third display content, determine to enter the individual annotation mode, for annotating the third display content selected by the user individually.

[0215] In practice, users can continuously select different areas of the displayed content for individual annotation. Simultaneously, a second icon appears within each selected closed area. When the user selects the second icon, they are taken to the annotation interface for that closed area, where they can add their own annotations. This effectively isolates annotations for different displayed content, improving user annotation efficiency and the meeting experience.

[0216] like Figures 7A-7B As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 7A The user continuously circles a closed path on the display interface. A second icon appears within each closed area. Clicking the second icon takes the user to the annotation interface of the corresponding closed area, and then redirects to... Figure 7B The displayed interface highlights the closed area and adds a mask layer to other areas, allowing users to write annotations on the mask layer. The mask layer has 100% opacity.

[0217] In some embodiments, this embodiment may perform multiple selection operations, the selection operation including multiple selections; if the area of ​​one selection operation contains at least a portion of the area of ​​the areas of other selection operations, the third display content is determined based on the area location information of the one selection operation.

[0218] In practice, if the area selected in one operation overlaps with the area selected in another, the size of the overlap area can be used to determine whether the two selection operations need to be merged. For example... Figures 8A-8B As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 8A The user selects multiple times, and a second icon is displayed within each selected closed area for the user to choose from. When one of the selected closed areas contains more than half the area of ​​the other two selected closed areas, the user is redirected to... Figure 8B The display interface hides the second icons and closed paths of the other two closed areas, displaying only the largest selected closed path. A second icon is then displayed within the closed area of ​​this largest closed path for user selection. When the user selects the closed area containing this second icon as the third displayed content, this third displayed content is highlighted, and other displayed content is overlaid with a mask layer. Handwritten annotations are made on this mask layer. If the user clears the screen or erases all handwriting at this point, the interface returns to the original display (i.e., all content is displayed). If some handwriting is erased, the handwriting is erased on the current mask layer.

[0219] In some embodiments, in response to a user's handwriting operation in an area other than the third display content, the handwriting corresponding to the handwriting operation is displayed.

[0220] In some embodiments, a mask layer is added to at least a portion of the displayed content in the display interface, excluding the third displayed content. Optionally, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0221] In some embodiments, the content displayed under the mask layer can be set to be visible; that is, the transparency of the mask layer can be set to be greater than 0 and less than 1. In this case, the following steps can also be performed:

[0222] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting is recognized; when the recognition result is a closed path and the area of ​​the closed path is greater than a threshold, the content within the closed path area is determined as the third display content, and the mask layer covering the third display content is cleared.

[0223] In practice, after the user selects the third display content on the display interface, the third display content is highlighted. At least part of the display content other than the third display content is covered by a mask layer, and the transparency of the mask layer is greater than 0 and less than 1. Since the display content under the mask layer is still visible, the user can continue to perform a selection operation on the mask layer, select the display content under the mask layer as the currently selected third display content, and highlight it.

[0224] Optionally, the trajectory of the third handwritten stroke is recognized. If the recognition result is a closed path, a smooth trajectory is automatically generated based on the third handwritten stroke as the closed path corresponding to the third handwritten stroke. When the area of ​​the closed region of the closed path is greater than 100×100dp, it is assumed that the user has selected the display content of the closed region as the third display content to be annotated. The third display content (i.e., the closed region) can be highlighted, and the mask layer on the closed region can be cleared.

[0225] like Figures 9A-9E As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 9A The user draws a closed path on the display interface and is redirected to... Figure 9B In the display interface, the closed area of ​​the closed path is highlighted as the third display content, while a mask layer is added to other areas with an 80% transparency. See [link / reference]. Figure 9C Users can perform handwriting operations on the overlay layer, and the corresponding handwriting will be displayed. See also Figure 9D Users can perform selection operations on the mask layer, that is, the user continues to draw a closed path. A smooth closed path is generated based on the closed path drawn by the user. When the image recognizes a closed path and the area of ​​the closed path is greater than 100×100dp, it jumps to Figure 9E The display interface clears the mask layer of the closed area of ​​the currently drawn closed path and highlights it, while the mask layer is retained in the remaining areas.

[0226] In some embodiments, after selecting the third display content, the third display content can also be dragged, as shown in the following specific execution method:

[0227] Method 1) In response to the user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation and displayed; a masking layer is added to at least part of the display content other than the screenshot in the display interface.

[0228] In practice, after selecting the third display content to be annotated, the user can also take a screenshot of the third display content and drag the screenshot to any position according to the dragging operation trajectory. This will add a mask layer to other areas besides the third display content, and the user can then annotate the third display content.

[0229] Method 2) Move the screenshot of the third display content to a designated location for display; add a mask layer to at least part of the display content other than the screenshot in the display interface.

[0230] In practice, after the user selects the third display content they want to annotate, the system automatically takes a screenshot of the third display content and moves the screenshot to the designated location for display.

[0231] like Figures 10A-10C As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. See also... Figure 10A Users can select multiple third-party content items by circling them to be annotated. The two selected third-party content items are then highlighted, while the remaining areas are masked. See also... Figure 10B Users can drag and drop any one or more third-party display items to capture a screenshot and move it to the desired position. This adds a mask layer to the display screen excluding the screenshot, allowing users to write annotations on this mask layer. Users can also drag a closed area containing the third-party display content to change the position of the highlighted area, effectively rearranging the layout. After dragging the highlighted closed area to a suitable position, the closed area (the user-selected first, second, or third display item) remains highlighted, and the remaining screen can be covered with an 80% opaque mask layer; users can continue writing annotations on this mask layer. See also... Figure 10C Users can click the "Share" menu item in the annotation menu to bring up a sharing pop-up window. Clicking the close button in the upper right corner of the sharing pop-up window will close it, allowing the current interface and the user's annotations to be shared with the user's terminal. Furthermore, users can click "File Flash Send," allowing the user's terminal to receive the current interface and annotations via specified software. Alternatively, users can click "Share" to upload the current interface and corresponding annotations to the cloud, and then download them from the cloud via a specified program. Users can also click the "Save Local" button; upon successful saving, a prompt will appear, allowing the user to view the file at the specified path in the system file manager. Clicking "Clear Screen" or "Erase Writing" will remove the writing on the black overlay, while retaining the highlighted closed areas. Clicking "Regular Annotation" will return to the regular annotation display interface.

[0232] In some embodiments, this embodiment allows for saving, sharing, and sending of a user's handwritten handwriting. This includes saving annotations on user-selected display content, sharing these annotations with other user terminals, or sending the annotations to a designated user terminal via file flash. Specific implementation steps are as follows:

[0233] In response to a user's operation command in annotation mode, determine the display content corresponding to the operation command and the handwritten handwriting associated with the display content; in response to the user's first operation, perform at least one of the following functions: save, share, or send the display content and / or the associated handwritten handwriting.

[0234] Optionally, the determined display content includes a first display content, a second display content, or a third display content. In implementation, after determining the display content that the user wants to annotate, the handwriting associated with that display content is determined. After the user annotates different display contents on the display interface, they can select the annotation content of one of the display contents to share, save, or send.

[0235] like Figure 11A-11F As shown in the diagram, this embodiment provides a display interface diagram in annotation mode. After the user determines the content to be annotated and makes an annotation, the user can click the share menu item in the annotation menu to jump to... Figure 11A The display interface includes a sharing pop-up window that shows one or more user annotations and their content. Users can select one or more to share with other users, and can also close the sharing pop-up by clicking the close button in the upper right corner. The default selection pop-up displays all annotations and content; users can click the thumbnail to deselect or reselect the page. See also... Figure 11B-11C When a user clicks "File Flash Send," a pop-up window for file flash sending appears. Once file flash sending is enabled, the user can receive the file by opening designated software, along with the content and annotations selected by the user. See also... Figure 11D-11E After the user clicks "Software Share," the selected display content and its annotations are uploaded to the cloud or server. A download link (such as a QR code) is then displayed. The user-accessible device opens the specified software and retrieves the displayed content and its annotations via the download link. See also... Figure 11F When a user clicks "Save Local," a message pops up saying "File saved." The user can then click "View" to navigate to the specified path in the system file manager to view the file; clicking the close button will close the pop-up window.

[0236] In some embodiments, after determining the displayed content and the handwritten handwriting associated with the displayed content, a screenshot can be taken of the displayed content and the handwritten handwriting associated with the displayed content, and at least one of the following functions can be performed on the screenshot: saving, sharing, and sending.

[0237] When a user selects to annotate multiple displayed contents on the current display interface, a screenshot of each displayed content and the associated handwritten handwriting can be saved separately. The user can select one or more screenshots to save, share, or send at least one of the following functions:

[0238] In some embodiments, the display interface further includes historical annotation content, and a masking layer is added to at least a portion of the display content other than the historical annotation content; and / or, a masking layer is added to at least a portion of the display content other than the display content bound to the historical annotation content.

[0239] In implementation, if the current interface already displays historical annotations, and the user wants to annotate other displayed content, after determining the content to be annotated (such as the first or second displayed content), the content to be annotated is highlighted. At least a portion of the displayed content other than the content to be annotated and the historical annotations is covered by a mask layer to prevent obscuring the historical annotations. Optionally, a historical annotation is bound to displayed content. The bound historical annotation is the content annotated by the user on that displayed content. The bound displayed content can be determined based on the positions of the historical annotation and the displayed content, or the time difference between the time the displayed content is defined and the time the historical annotation is received. The bound displayed content can be all or part of the displayed content on the interface.

[0240] In some embodiments, the display interface of this embodiment includes annotation content, which is bound to the display content of the display interface. This embodiment can also adjust the annotation content in the following ways:

[0241] Determine the fourth display content in the display interface;

[0242] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0243] Optionally, the fourth display content is selected by the user from the display content on the display interface; or it may be the system's default display content. The fourth display content can also be a specified type of display content obtained by classifying the content of the display interface according to a user-defined algorithm. This embodiment does not impose excessive limitations on this.

[0244] In implementation, users can select display content on the interface through marking operations, handwriting operations, and touch commands on menu items. For example, in response to a user's marking operation on display content, the fourth display content is determined based on the position information of the fourth handwritten stroke corresponding to the marking operation. Similarly, in response to a user's handwriting operation on the display interface, the fourth display content is determined based on the position information of the fourth handwritten stroke corresponding to the handwriting operation. Alternatively, in response to a user's touch command on a first menu item, the display content on the interface is divided into multiple content areas; in response to a user's fourth selection operation on multiple content areas, the fourth display content corresponding to the fourth selection operation is determined. Or, in response to a user's touch command on a third menu item, a user's selection operation on display content is received; the fourth display content is determined based on the position information of the selection operation. The principle behind determining the fourth display content is the same as the principle behind determining the first, second, or third display content described above, and the fourth display content can be determined by referring to the methods described above for determining the first, second, or third display content.

[0245] In implementation, in one scenario, when historical annotations exist on the display interface, and the user selects content to display and wants to annotate it, if the selected content overlaps with historical annotations, the position of the historical annotations can be automatically adjusted to ensure they do not obscure each other. In another scenario, when the user selects multiple pieces of content to display and annotates them, with each piece of content bound to one annotation, if an annotation at least partially overlaps with other annotations, or if an annotation at least partially overlaps with at least one piece of content displayed, the position and size of the annotation are automatically adjusted to prevent overlap between the displayed content and the annotations, or between the annotations themselves.

[0246] This embodiment addresses various annotation needs in electronic annotation. For example, it allows for focused content annotation, with unnecessary content masked. AI can also automatically identify annotation content and intelligently annotate by partitioning the displayed content. This embodiment can add an adjustable transparency mask to the original annotation content; determine the content to be annotated based on the pen stroke and mask other content; automatically partition the displayed content, allowing users to select the desired area for annotation, supporting multi-page annotation; and automatically rearrange partitions, allowing users to select the desired area for annotation after rearrangement. This solves the problem of poor readability of annotation content and improves the user experience.

[0247] Based on the same inventive concept, this application also provides an annotation method. Since the principle of this method in solving the problem is similar to that of the display device, the implementation of this method can be referred to the implementation of the display device, and the repeated parts will not be described again.

[0248] like Figure 12 As shown, the implementation flow of an annotation method provided in this embodiment is as follows:

[0249] Step 1200: In response to the user's operation command in annotation mode, add a mask layer to at least part of the displayed content of the display interface;

[0250] Step 1201: In response to the user's handwriting operation on the mask layer, display the handwriting corresponding to the handwriting operation.

[0251] As an optional implementation, the operation instructions include user touch operations on designated buttons on the display interface; in response to user operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content on the display interface, including:

[0252] In response to user touch operations on the display interface, a masking layer is added to all displayed content on the display interface.

[0253] As an optional implementation, the operation instructions include user marking operations on the displayed content; in response to the user's operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content of the display interface, including:

[0254] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation;

[0255] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

[0256] As an optional implementation, the operation instructions include handwriting operations by the user on the display interface; in response to the user's operation instructions in annotation mode, adding a masking layer to at least a portion of the displayed content on the display interface, further comprising:

[0257] In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation.

[0258] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

[0259] As an optional implementation, before adding a masking layer to at least a portion of the displayed content of the display interface, the method further includes:

[0260] A comment menu is displayed on the display interface, and the comment menu includes at least one menu item;

[0261] In response to the user's touch command on the menu item, determine the third display content selected by the user.

[0262] As an optional implementation, the annotation menu includes a first menu item; in response to a user's touch command on the menu item, determining the third display content selected by the user includes:

[0263] In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas;

[0264] In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

[0265] As an optional implementation, the annotation menu further includes a second menu item; the method further includes:

[0266] In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0267] As an optional implementation, the annotation menu includes a third menu item; in response to a user's touch command on the menu item, determining the third display content selected by the user includes:

[0268] In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content;

[0269] The third display content is determined based on the position information of the selection operation.

[0270] As an optional implementation, determining the third display content based on the position information of the selection operation further includes:

[0271] Receive multiple selections of the displayed content from the user;

[0272] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0273] As an optional implementation, the selection operation includes multiple selections; determining the third display content based on the position information of the selection operation further includes:

[0274] If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

[0275] As an optional implementation, the selection operation includes multiple selections; determining the third display content based on the position information of the selection operation further includes:

[0276] Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations;

[0277] In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

[0278] As an optional implementation, determining the third display content based on the position information of the selection operation includes:

[0279] The closed region is determined based on the position information of the selection operation;

[0280] When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0281] As an optional implementation, the method further includes:

[0282] In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

[0283] As an optional implementation, the method further includes:

[0284] A mask layer will be added to at least part of the displayed content except for the third displayed content.

[0285] As an optional implementation, the content displayed under the mask layer is visible; the method further includes:

[0286] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized.

[0287] When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

[0288] As an optional implementation, the method further includes:

[0289] In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display.

[0290] A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

[0291] As an optional implementation, the display interface further includes historical annotation content, and a masking layer is added to at least a portion of the displayed content of the display interface, including:

[0292] Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or,

[0293] Add a masking layer to at least part of the displayed content, excluding the content bound to historical annotations.

[0294] As an optional implementation, the display interface further includes annotation content, which is bound to the display content of the display interface. The method further includes:

[0295] Determine the fourth display content in the display interface;

[0296] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0297] As an optional implementation, in response to user operation commands in annotation mode, the method further includes:

[0298] Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content;

[0299] In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

[0300] As an optional implementation, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0301] Based on the same inventive concept, this application also provides an electronic device. Since the principle of solving the problem by this electronic device is similar to that of the display device, the implementation of this electronic device can refer to the implementation of the display device, and the repeated parts will not be described again.

[0302] like Figure 13 As shown, the electronic device includes a processor 1300 and a memory 1301. The memory 1301 stores programs executable by the processor 1300. The processor 1300 reads the programs from the memory 1301 and performs the following steps:

[0303] In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface;

[0304] In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

[0305] As an optional implementation, the operation instructions include a user's touch operation on a designated button on the display interface; the processor is specifically configured to execute:

[0306] In response to user touch operations on the display interface, a masking layer is added to all displayed content on the display interface.

[0307] As an optional implementation, the operation instructions include user marking operations on the displayed content; the processor is specifically configured to execute:

[0308] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation;

[0309] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

[0310] As an optional implementation, the operation instructions include handwriting operations performed by the user on the display interface; the processor is further configured to execute:

[0311] In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation.

[0312] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

[0313] As an optional implementation, before adding a masking layer to at least a portion of the displayed content of the display interface, the processor is further configured to perform:

[0314] A comment menu is displayed on the display interface, and the comment menu includes at least one menu item;

[0315] In response to the user's touch command on the menu item, determine the third display content selected by the user.

[0316] As an optional implementation, the annotation menu includes a first menu item; the processor is specifically configured to execute:

[0317] In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas;

[0318] In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

[0319] As an optional implementation, the annotation menu further includes a second menu item; the processor is specifically configured to execute:

[0320] In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0321] As an optional implementation, the annotation menu includes a third menu item; the processor is specifically configured to execute:

[0322] In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content;

[0323] The third display content is determined based on the position information of the selection operation.

[0324] As an optional implementation, the processor is further configured to execute:

[0325] Receive multiple selections of the displayed content from the user;

[0326] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0327] As an optional implementation, the selection operation includes multiple selections; the processor is further configured to execute:

[0328] If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

[0329] As an optional implementation, the selection operation includes multiple selections; the processor is further configured to execute:

[0330] Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations;

[0331] In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

[0332] As an optional implementation, the processor is specifically configured to execute:

[0333] The closed region is determined based on the position information of the selection operation;

[0334] When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0335] As an optional implementation, the processor is further configured to execute:

[0336] In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

[0337] As an optional implementation, the processor is further configured to execute:

[0338] A mask layer will be added to at least part of the displayed content except for the third displayed content.

[0339] As an optional implementation, the content displayed under the mask layer is visible; the processor is further configured to execute:

[0340] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized.

[0341] When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

[0342] As an optional implementation, the processor is further configured to execute:

[0343] In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display.

[0344] A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

[0345] As an optional implementation, the display interface also includes historical annotations, and the processor is specifically configured to execute:

[0346] Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or,

[0347] Add a masking layer to at least part of the displayed content, excluding the content bound to historical annotations.

[0348] As an optional implementation, the display interface further includes annotation content, which is bound to the display content of the display interface, and the processor is specifically configured to execute:

[0349] Determine the fourth display content in the display interface;

[0350] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0351] As an optional implementation, in response to user operation commands in annotation mode, the processor is further configured to execute:

[0352] Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content;

[0353] In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

[0354] As an optional implementation, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0355] Based on the same inventive concept, this application also provides an annotation device. Since the principle of the device in solving the problem is similar to that of the display device, the implementation of the device can refer to the implementation of the display device, and the repeated parts will not be described again.

[0356] like Figure 14 As shown, this embodiment provides an annotation device, including:

[0357] The masking module 1400 is used to add a masking layer to at least part of the displayed content of the display interface in response to the user's operation command in annotation mode.

[0358] Display module 1401 is used to respond to the user's handwriting operation on the mask layer and display the handwriting corresponding to the handwriting operation.

[0359] As an optional implementation, the operation instructions include a user's touch operation on a designated button on the display interface; the covering module 1400 is specifically used for:

[0360] In response to user touch operations on the display interface, a masking layer is added to all displayed content on the display interface.

[0361] As an optional implementation, the operation instructions include user marking operations on the displayed content; the overlay module 1400 is specifically used for:

[0362] In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation;

[0363] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

[0364] As an optional implementation, the operation instructions include handwriting operations by the user on the display interface; the covering module 1400 is further used for:

[0365] In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation.

[0366] Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

[0367] As an optional implementation, before adding a masking layer to at least a portion of the displayed content of the display interface, the masking module 1400 is further configured to:

[0368] A comment menu is displayed on the display interface, and the comment menu includes at least one menu item;

[0369] In response to the user's touch command on the menu item, determine the third display content selected by the user.

[0370] As an optional implementation, the annotation menu includes a first menu item; the overlay module 1400 is specifically used for:

[0371] In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas;

[0372] In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

[0373] As an optional implementation, the annotation menu further includes a second menu item; the overlay module 1400 is specifically used for:

[0374] In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

[0375] As an optional implementation, the annotation menu includes a third menu item; the overlay module 1400 is specifically used for:

[0376] In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content;

[0377] The third display content is determined based on the position information of the selection operation.

[0378] As an optional implementation, the covering module 1400 is further used for:

[0379] Receive multiple selections of the displayed content from the user;

[0380] Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

[0381] As an optional implementation, the selection operation includes multiple selections; the masking module 1400 is further configured to:

[0382] If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

[0383] As an optional implementation, the selection operation includes multiple selections; the masking module 1400 is further configured to:

[0384] Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations;

[0385] In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

[0386] As an optional implementation, the covering module 1400 is specifically used for:

[0387] The closed region is determined based on the position information of the selection operation;

[0388] When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

[0389] As an optional implementation, the covering module 1400 is further used for:

[0390] In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

[0391] As an optional implementation, the covering module 1400 is further used for:

[0392] A mask layer will be added to at least part of the displayed content except for the third displayed content.

[0393] As an optional implementation, the content displayed under the mask layer is visible; the masking module 1400 is further configured to:

[0394] In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized.

[0395] When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

[0396] As an optional implementation, the covering module 1400 is further used for:

[0397] In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display.

[0398] A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

[0399] As an optional implementation, the display interface also includes historical annotation content, and the overlay module 1400 is specifically used for:

[0400] Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or,

[0401] Add a masking layer to at least part of the displayed content, excluding the content bound to historical annotations.

[0402] As an optional implementation, the display interface further includes annotation content, which is bound to the display content of the display interface, and also includes an adjustment module specifically used for:

[0403] Determine the fourth display content in the display interface;

[0404] In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

[0405] As an optional implementation, in response to user operation commands in annotation mode, a response module is further included, specifically for:

[0406] Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content;

[0407] In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

[0408] As an optional implementation, the transparency of the mask layer is greater than 0 and less than or equal to 1.

[0409] Based on the same inventive concept, this disclosure provides a computer storage medium comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the access control methods discussed above. Since the principle by which the computer storage medium solves the problem is similar to that of the access control methods, the implementation of the computer storage medium can be referred to the implementation of the methods, and repeated details will not be elaborated further.

[0410] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0411] Based on the same inventive concept, this disclosure also provides a computer program product, which includes computer program code that, when executed on a computer, causes the computer to perform any of the access control methods discussed above. Since the principle by which the above-described computer program product solves the problem is similar to that of the access control methods, the implementation of the above-described computer program product can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0412] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0413] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0414] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0415] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0416] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0417] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A display device, characterized in that, Includes a display screen and control circuitry, wherein: The display screen is configured to display content; The control circuit includes a processor and a memory. The memory stores programs executable by the processor, and the processor reads the programs from the memory and performs the following steps: In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface; In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

2. The display device according to claim 1, characterized in that, The operation instructions include user touch operations on designated buttons on the display interface; the processor is specifically configured to execute: In response to user touch operations on the display interface, a mask layer is added to all displayed content on the display interface.

3. The display device according to claim 1, characterized in that, The operation instructions include user marking operations on the displayed content; the processor is specifically configured to execute: In response to a user's marking operation on the displayed content, the first displayed content is determined based on the position information of the first handwritten stroke corresponding to the marking operation; Add a masking layer to at least a portion of the displayed content in the display interface, excluding the first displayed content.

4. The display device according to claim 1, characterized in that, The operation instructions include handwriting operations performed by the user on the display interface; the processor is further configured to execute: In response to the user's handwriting operation on the display interface, the second display content is determined based on the position information of the second handwriting stroke corresponding to the handwriting operation. Add a masking layer to at least a portion of the displayed content in the display interface, excluding the second displayed content and the second handwritten handwriting.

5. The display device according to claim 1, characterized in that, Before adding a masking layer to at least a portion of the displayed content of the display interface, the processor is specifically configured to perform: A comment menu is displayed on the display interface, and the comment menu includes at least one menu item; In response to the user's touch command on the menu item, determine the third display content selected by the user.

6. The display device according to claim 5, characterized in that, The annotation menu includes a first menu item; the processor is specifically configured to execute: In response to the user's touch command on the first menu item, the display content of the display interface will be divided into multiple content areas; In response to the user's second selection operation on multiple content areas, determine the third display content corresponding to the second selection operation.

7. The display device according to claim 6, characterized in that, The annotation menu also includes a second menu item; the processor is further configured to execute: In response to the user's touch command on the second menu item, the displayed content is re-divided into multiple content areas.

8. The display device according to claim 5, characterized in that, The annotation menu includes a third menu item; the processor is specifically configured to execute: In response to the user's touch command on the third menu item, it receives the user's selection operation of the displayed content; The third display content is determined based on the position information of the selection operation.

9. The display device according to claim 8, characterized in that, The processor is further configured to execute: Receive multiple selections of the displayed content from the user; Based on the position information of each of the multiple selection operations, multiple third-party display contents are determined.

10. The display device according to claim 8, characterized in that, The selection operation includes multiple selections; the processor is further configured to execute: If the area selected in one of the selection operations contains at least a portion of the area selected in other selection operations, then the third display content is determined based on the location information of the area selected in one of the selection operations.

11. The display device according to claim 8, characterized in that, The selection operation includes multiple selections; the processor is further configured to execute: Based on the position information of each selection operation, determine the third display content corresponding to each of the multiple selection operations; In response to the user's selection of any third display content, the system enters a separate annotation mode, which is used to annotate the user-selected third display content individually.

12. The display device according to claim 8, characterized in that, The processor is specifically configured to execute: The closed region is determined based on the position information of the selection operation; When the area of ​​the closed region is greater than a threshold, the content of the closed region is determined as the third display content.

13. The display device according to any one of claims 5 to 12, characterized in that, The processor is further configured to execute: In response to a user's handwriting operation in an area other than the third display content, the corresponding handwriting is displayed.

14. The display device according to any one of claims 5 to 12, characterized in that, The processor is further configured to execute: A mask layer will be added to at least part of the displayed content except for the third displayed content.

15. The display device according to claim 14, characterized in that, The content displayed under the mask layer is visible; the processor is further configured to execute: In response to a user's handwriting operation on the mask layer, the trajectory of the third handwriting stroke of the handwriting operation is recognized. When the recognition result is a closed path, and the area of ​​the closed path is greater than a threshold, the content within the area of ​​the closed path is determined as the third display content, and the masking layer covering the third display content is removed.

16. The display device according to any one of claims 5 to 12, 15, characterized in that, The processor is further configured to execute: In response to a user's drag operation on the third display content, the screenshot of the third display content is moved to the position corresponding to the drag operation for display; or, the screenshot of the third display content is moved to a specified position for display. A masking layer will be added to at least a portion of the displayed content in the display interface, excluding the screenshot.

17. The display device according to any one of claims 1 to 12, 15, characterized in that, The display interface also includes historical annotations, and the processor is specifically configured to execute: Add a masking layer to at least part of the displayed content, excluding historical annotations; and / or, Add a masking layer to at least part of the displayed content other than the content bound to historical annotations.

18. The display device according to any one of claims 1 to 12, 15, characterized in that, The display interface also includes annotation content, which is bound to the display content of the display interface. The processor is further configured to execute: Determine the fourth display content in the display interface; In response to the fourth display content and the annotation content at least partially overlapping, the position and / or size of the annotation content are adjusted.

19. The display device according to any one of claims 1 to 12, 15, characterized in that, In response to user commands in annotation mode, the processor is further configured to execute: Determine the display content corresponding to the operation instruction, and the handwritten handwriting associated with the display content; In response to the user's first action, perform at least one of the following functions: save, share, or send the displayed content and / or associated handwritten text.

20. The display device according to any one of claims 1 to 4, characterized in that, The transparency of the mask layer is greater than 0 and less than or equal to 1.

21. An annotation method, characterized in that, The method includes: In response to user operation commands in annotation mode, a masking layer is added to at least a portion of the displayed content of the display interface; In response to the user's handwriting operation on the overlay, the corresponding handwriting is displayed.

22. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing a program executable by the processor, and the processor for reading the program in the memory and performing the steps of the method of claim 21.

23. A computer storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in claim 21.

24. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method as described in claim 21.

Images