Annotation method and device, equipment and storage medium

Abstract

The invention discloses an annotation method and device, equipment and a storage medium. According to the technical scheme, the method comprises the steps that a document body of a document file is displayed, first annotation content of the document body is generated in response to first annotation operation, the first annotation content is displayed in a blackboard-writing drawing layer, and the blackboard-writing drawing layer is located on the upper layer of a display layer of the document file; in response to the attachment display operation, storing a first annotation content corresponding to the document main body and emptying the first annotation content displayed on the blackboard-writing drawing layer, confirming an attachment element selected by the attachment display operation in the document file, and performing full-screen display on the attachment element; generating a second annotation content of the attachment element in response to the second annotation operation, and displaying the second annotation content in the emptied blackboard writing drawing layer; and in response to the attachment closing operation, storing second annotation contents corresponding to the attachment elements, emptying the second annotation contents on the blackboard-writing drawing layer, displaying the document main body, and displaying the stored first annotation contents on the blackboard-writing drawing layer.

Description

Annotation method, device, equipment and storage medium Technical Field

[0001] The present application relates to the technical field of interactive devices, and in particular to an annotation method, apparatus, device and storage medium. Background Art

[0002] When a user wants to display a document on an interactive device, they can select a document from a local or cloud document list and open it. The document will then be displayed in the document display app installed on the interactive device. Users can also use the brush tool provided by the document display app to annotate the document. As the document content slides, the blackboard content also slides with it, allowing the document content and corresponding annotations to be displayed together, improving the user's writing and reading experience.

[0003] In the prior art, document files include not only text content but also various attachment elements, such as images, audio and video, text, presentation documents, tables, and other resources, as well as specially formatted elements such as code blocks and references. When an attachment element in a document is opened, the interactive device displays the attachment element full screen. When the interactive device displays the attachment element full screen, the annotation function of the brush tool is disabled, preventing users from annotating the attachment element. This indicates that the annotation function of document display applications is not comprehensive.

[0004] Summary of the Invention

[0005] The present application provides an annotation method, apparatus, device and storage medium to solve the problem in the prior art that it is impossible to annotate attachment elements displayed in full screen, improve the annotation function of document display applications, and optimize the user's annotation and reading experience of documents.

[0006] In a first aspect, the present application provides an annotation method, comprising:

[0007] Display the document body of the document file, generate the first batch of annotation content of the document body in response to the first batch of annotation operations, and display the first batch of annotation content in a blackboard drawing layer, which is located above the display layer of the document file;

[0008] In response to the attachment display operation, saving the first batch of annotations corresponding to the document body and clearing the first batch of annotations displayed on the blackboard drawing layer, confirming the attachment element selected in the attachment display operation in the document file, and displaying the attachment element in full screen;

[0009] generating second annotation content of the attachment element in response to the second annotation operation, and displaying the second annotation content in the cleared blackboard drawing layer;

[0010] In response to the attachment closing operation, the second annotation content corresponding to the attachment element is saved and the second annotation content on the blackboard drawing layer is cleared, the document body is displayed and the saved first annotation content is displayed on the blackboard drawing layer.

[0011] In a second aspect, the present application provides an annotation device, comprising:

[0012] a first batch annotation module configured to display the document body of the document file, generate the first batch annotation content of the document body in response to the first batch annotation operation, and display the first batch annotation content in the blackboard drawing layer, which is located above the display layer of the document file;

[0013] The attachment display module is configured to save the first batch of annotations corresponding to the document body and clear the first batch of annotations displayed on the blackboard drawing layer in response to the attachment display operation, confirm the attachment element selected in the attachment display operation in the document file, and display the attachment element in full screen;

[0014] A second annotation module is configured to generate second annotation content of the attachment element in response to a second annotation operation, and display the second annotation content in the cleared blackboard drawing layer;

[0015] The main body display module is configured to save the second annotation content corresponding to the attachment element and clear the second annotation content on the blackboard drawing layer in response to the attachment closing operation, display the document main body and display the saved first annotation content on the blackboard drawing layer.

[0016] In a third aspect, the present application provides an annotation device, comprising:

[0017] One or more processors; a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the annotation method as described in the first aspect.

[0018] In a fourth aspect, the present application provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the annotation method as described in the first aspect.

[0019] In the present application, when the attachment element in the document file is displayed in full screen, the first batch of annotation contents in the blackboard drawing layer can be cleared after the document body is saved in association with the corresponding first batch of annotation contents, that is, the first batch of annotation contents and the attachment element expanded in full screen will not be displayed at the same time, avoiding confusion of the annotation contents. Since the blackboard drawing layer is always displayed in the upper layer of the document file display layer, even when the attachment element is displayed in full screen, the blackboard drawing layer is still located in the upper layer of the attachment element, and the second attachment content corresponding to the attachment element can be displayed through the blackboard drawing layer, realizing the function of annotating the attachment element displayed in full screen. When exiting the full-screen display state of the attachment element, the second annotation content in the blackboard drawing layer can be cleared after the attachment element is saved in association with the corresponding second annotation, that is, the second annotation content and the document body will not be displayed at the same time, avoiding confusion of the annotation content. Since the document file is saved in association with the corresponding first batch of annotation contents, the document file can be displayed together with the corresponding first batch of annotation contents, retaining the annotation traces of the document file, avoiding users from repeatedly annotating the document file. Similarly, the attachment element is saved in association with the corresponding second annotation content. When the attachment element is displayed in full screen, the attachment element and the corresponding second annotation content can be displayed together, retaining the annotation traces of the attachment element and avoiding users from repeatedly annotating the attachment element. This application implements the annotation function of the document body and attachment elements by saving the annotation content corresponding to the document file and the attachment element, and promptly clearing the annotation content on the blackboard drawing layer. It solves the problem in the prior art that it is impossible to annotate attachment elements displayed in full screen, improves the annotation function of the document display application, improves the richness of annotation interaction and the reusability of annotation content, and optimizes the user's annotation and reading experience of documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a flow chart of an annotation method provided in an embodiment of the present application;

[0021] FIG2 is a flow chart showing the first batch of annotation contents provided by an embodiment of the present application;

[0022] FIG3 is a first schematic diagram of a document body provided in an embodiment of the present application;

[0023] FIG4 is a schematic diagram of a visible area provided in an embodiment of the present application;

[0024] FIG5 is a second schematic diagram of a document body provided in an embodiment of the present application;

[0025] FIG6 is a flowchart showing the second annotation content provided by an embodiment of the present application;

[0026] FIG7 is a first schematic diagram of a second attachment element provided in an embodiment of the present application;

[0027] FIG8 is a second schematic diagram of a second attachment element provided in an embodiment of the present application;

[0028] FIG9 is a schematic structural diagram of an annotation device provided in an embodiment of the present application;

[0029] FIG10 is a schematic structural diagram of an annotation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0031] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0032] In a more common existing implementation, a document file includes a document body and attachment elements embedded in the document body, and a document display application for displaying the document file provides a brush tool, which can be used to annotate the document body. When the document display application displays the attachment element in full screen, the annotation function of the brush tool becomes invalid, making it impossible for users to annotate on the attachment element, and the annotation function of the document display application is not perfect. To solve this technical problem, a related technology is to display the annotation content of the document file through an annotation layer, and the annotation layer is displayed on the upper layer of the display layer of the document file. If the user annotates the document body and then displays the attachment element in full screen, the annotation content corresponding to the document body will be displayed on the attachment element expanded in full screen, resulting in confusion of the annotation content. The user has to use the eraser function of the brush tool to erase the annotation content on the attachment element before reading and annotating the attachment element. The erased annotations are annotations to the document body. After exiting the full-screen display of the attachment element, the document body does not retain the corresponding annotation traces, forcing the user to re-annotate the document body. At this time, the annotations to the attachment element may be displayed on the document body, and the annotations to the attachment element must be erased before reading and annotating the document body. Therefore, when using related technologies to annotate the document body and attachment elements, users have to repeatedly erase annotations that do not fit the displayed content and repeatedly add annotations to the displayed content. The annotation interaction process is cumbersome and difficult to operate, affecting the user's annotation and reading experience of the document.

[0033] To solve the above problems, this embodiment provides an annotation method, which saves the annotation content corresponding to the document file and attachment elements, and promptly clears the annotation content on the blackboard drawing layer, thereby realizing the annotation function of the document body and attachment elements while simplifying the complexity of annotation interaction.

[0034] The annotation method provided in this embodiment can be executed by an annotation device, which can be implemented through software and / or hardware. The annotation device can be composed of two or more physical entities, or a single physical entity. For example, the annotation device can be an intelligent interactive device installed with a document display application, such as a computer or interactive tablet, or it can be a processor of the intelligent interactive device.

[0035] The annotation device is installed with at least one type of operating system, including but not limited to Android, Linux, and Windows. The annotation device can install at least one application based on the operating system. The application can be an application that comes with the operating system or an application downloaded from a third-party device or server. In this embodiment, the annotation device has at least one application that can execute the annotation method, and the application can be a document display application.

[0036] For ease of understanding, this embodiment is described by taking an interactive tablet as an example of the main body for executing the annotation method.

[0037] FIG1 is a flow chart of an annotation method provided in an embodiment of the present application. Referring to FIG1 , the annotation method specifically includes:

[0038] S110. Display the document body of the document file, generate the first batch of annotation contents of the document body in response to the first batch of annotation operations, and display the first batch of annotation contents in the blackboard drawing layer, which is located above the display layer of the document file.

[0039] For example, the interactive tablet is installed with a document display application. When a user inputs an operation to open a locally stored document file into the interactive tablet, the interactive tablet confirms the document file selected by the operation, opens the document file through the document display application, and displays the document body of the document file in the application interface of the document display application. The document display application has three corresponding operating modes: selection mode, brush mode, and eraser mode. When the document display application is in selection mode, the content of the document body or attachment elements in the document file can be swiped; when the document display application is in brush mode, the document file can be annotated; when the document display application is in eraser mode, the document file annotations can be erased.

[0040] In one embodiment, controls corresponding to these three modes are configured in the toolbar of the document display application. When the user clicks the control corresponding to the selection mode, the document display application switches to the selection mode, and the same applies to other modes. In another embodiment, the interactive tablet is a touch-type interactive tablet, and the touch-type interactive tablet includes an infrared touch frame and an elastic wave module. The infrared touch frame collects touch data input by the user, and the elastic wave module identifies the material of the touch object. Among them, when the user uses the tip of a stylus pen to write on the touch-type interactive tablet, the elastic wave module identifies that the current touch object is the tip of the stylus pen, then the document display application is switched to the brush mode, and the touch data collected by the infrared touch frame generates the writing handwriting corresponding to the input of the stylus pen tip, and the writing handwriting is added as annotation content in the document file. When a user slides the end of a stylus across a touch-sensitive interactive tablet, the elastic wave module identifies the touch object as the end of the stylus. The document display application switches to eraser mode, and touch data collected by the infrared touch frame generates an erasure trajectory corresponding to the stylus's input. The annotation content at the corresponding position is erased based on the erasure trajectory. When a user slides a finger across a touch-sensitive interactive tablet, the elastic wave module identifies the touch object as a finger. The document display application switches to selection mode, and touch data collected by the infrared touch frame generates a sliding trajectory corresponding to the finger's input. The document file content is slid based on the sliding trajectory.

[0041] In this embodiment, the first batch of annotation operations can be understood as the operations input by the user when annotating the document body. This embodiment is described by taking the interactive tablet as a touch-type interactive tablet as an example. After the document display application displays the document body of the document file in the application interface, the user uses the stylus tip to write in the application interface, and the elastic wave module determines that the current touch object is the stylus tip, and switches the document display application to brush mode. When the document display application is in brush mode and the application interface displays the document body, it is confirmed that the operation currently input by the user is the first batch of annotation operations, and the touch data collected by the infrared touch frame generates the first batch of annotation content corresponding to the first batch of annotation operations, and the first batch of annotation content is rendered in the blackboard drawing layer located above the document file display layer. Among them, the blackboard drawing layer is a transparent layer. From a visual effect point of view, the document file and the first batch of annotation content are displayed correspondingly in the application interface.

[0042] In order to ensure that the first batch of annotations can be displayed at the relative position of the corresponding document file after exiting the full-screen display state of the attachment element, the coordinates corresponding to the first batch of annotations in the document body will be determined when generating and rendering the first batch of annotations, so that the display of the first batch of annotations can be restored based on the coordinates. Figure 2 is a flowchart of generating and rendering the first batch of annotations provided by an embodiment of the present application. As shown in Figure 2, the steps of displaying the first batch of annotations specifically include S1101-S1102:

[0043] S1101. Determine, based on the first coordinates of the document body within the visible area and the second coordinates of the first batch of annotation operations within the visible area, the third coordinates of the first batch of annotation contents generated by the first batch of annotation operations within the document body.

[0044] Exemplarily, the visible area is the application interface of the document display application. Figure 3 is a first schematic diagram of the document body provided by an embodiment of the present application. As shown in Figure 3, the horizontal coordinate range of the document body 11 is 0 to N, and the vertical coordinate range is 0 to M. The first attachment element 12 and the second attachment element 13 are embedded in the document body 11. The coordinate range of the document body includes the coordinate range of the corresponding attachment element when it is embedded and displayed in the document body. Wherein "X" and "x" represent the text content in the document body 11 and the second attachment element 14, respectively. The content in the rectangular area of ​​(A, 0), (A, N), (B, 0) and (B, N) of the document body 11 is displayed in the visible area 12, and (A, 0), (A, N), (B, 0) and (B, N) can be determined as the first coordinates of the document body in the visible area 12. It should be noted that, from a visual perspective, the user can only see the document body 11 within the visible area 12, that is, the content within the rectangular area (A, 0), (A, N), (B, 0) and (B, N) of the document body 11. The content in other areas is not visible. Figure 3 shows the content of other areas only for the convenience of describing the embodiment. The coordinates marked in Figure 3 are generated based on the coordinate system of the document body 11, and the coordinate system of the document body 11 takes the upper left corner of the document body 11 as the origin.

[0045] FIG4 is a schematic diagram of the visible area provided by an embodiment of the present application. As shown in FIG4 , when a user uses the tip of a stylus pen to annotate the document body 11 within the visible area 12 and triggers the first batch of annotation operations, the touch data collected by the infrared touch frame generates the first batch of annotation content, and the coordinates of the first batch of annotation operations in the interactive flat panel display screen are determined based on the touch data. Based on the coordinates of the first batch of annotation operations in the interactive flat panel display screen and the coordinates of the visible area in the interactive flat panel display screen, the second coordinates of the first batch of annotation operations in the visible area 12 are determined. The second coordinates are coordinates generated based on the coordinate system of the visible area 12, and the coordinate system of the visible area 12 takes the upper left corner of the visible area 12 as the origin. Afterwards, based on the first coordinates of the document body 11 within the visible area 12 and the second coordinates of the first batch of annotation operations in the visible area 12, the second coordinates are converted from the coordinate system of the visible area 12 to the coordinate system of the document body 11 to obtain the third coordinates of the first batch of annotation content in the document body 11.

[0046] S1102. Rendering the first batch of annotation contents generated corresponding to the first batch of annotation operations in a blackboard drawing layer according to the second coordinates of the first batch of annotation operations in the visible area. The blackboard drawing layer is configured in the visible area.

[0047] Exemplarily, the blackboard drawing layer is configured within the visible area, and the coordinate system of the blackboard drawing layer is the same as the coordinate system of the visible area. Based on the second coordinates of the first batch of annotation operations within the visible area, the pixel coordinates of the first batch of annotation content generated correspondingly in the blackboard drawing layer can be determined. Based on the pixel coordinates of the first batch of annotation content in the blackboard drawing layer, the first batch of annotation content is rendered in the blackboard drawing layer. Referring to FIG4 , if the first batch of annotation operations is to underline a line of text in the document body 11 within the visible area 12, then after the first batch of annotation content generated corresponding to the first batch of annotation operations is rendered in the blackboard drawing layer, an underline is displayed at the corresponding text.

[0048] When the length of the document body is greater than the length of the visible area, the visible area cannot fully display all the contents of the document body. At this time, the user can input a first sliding operation to the interactive tablet to adjust the contents of the document body displayed in the visible area. In one embodiment, for a touch-sensitive interactive tablet equipped with an elastic wave module, the elastic wave module can identify the material of the touch object, so that the first sliding operation can be triggered when the user slides the document body in the visible area with a finger, the sliding distance is determined based on the touch data collected by the infrared touch frame, and the document body and the first batch of annotations are slid based on the sliding distance. Exemplarily, when the elastic wave module identifies that the current touching object is a finger, determines the finger sliding based on the touch data collected by the infrared touch frame, and determines that the visible area displays the document body, the operation currently input by the user is confirmed to be the first sliding operation, the sliding distance of the first sliding operation input is determined based on the touch data collected by the infrared touch frame, and the document body and the first batch of annotations are synchronously slid based on the sliding distance, so that the document body and the first batch of annotations at the relative position are displayed together in the visible area.

[0049] In another embodiment, for an interactive tablet that is not provided with an elastic wave module, when the document display application is in different modes, there are different response processes to the first sliding operation. Exemplarily, when the document display application is in selection mode, the first sliding event of the document body is monitored by the first document object model of the document body, and the document body and the corresponding first batch of annotations are controlled to slide synchronously according to the first sliding distance corresponding to the first sliding event. The first sliding event refers to a sliding event triggered by the user sliding the document body by scrolling the mouse, dragging the mouse, or touching the screen when the document display application is in selection mode and the application interface displays the document body. When the first document object model monitors the first sliding event of the document body, the first sliding distance is determined according to the first sliding operation. In this embodiment, the document body is displayed in the form of full rendering, that is, the document body will be rendered in advance in a canvas that matches the size of the document body, and the sliding of the document body can be achieved by sliding the canvas by the first sliding distance.

[0050] The computational complexity of full rendering is large. If both the document body and the first batch of annotations are fully rendered, it will affect the performance of the interactive tablet. Therefore, incremental rendering can be used to render the first batch of annotations only in the visible area. The specific implementation process is: the document body in the visible area is slid by the first document object model according to the first sliding distance; the first sliding distance is synchronized to the blackboard drawing layer, so that the blackboard drawing layer incrementally renders the first batch of annotations that are slid into the visible area according to the first sliding distance. Figure 5 is a second schematic diagram of the document body provided by an embodiment of the present application. As shown in Figure 5, when the user slides the document body 11 downward through the first sliding operation, the first sliding distance generated corresponding to the first sliding operation is k, and the first document object module slides the document body 11 on the canvas from bottom to top by k to slide the document body 11 in the visible area 12. After the sliding is completed, the content in the rectangular area of ​​(A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body is displayed in the visible area 12. At the same time, the first document object model synchronizes the first sliding distance to the blackboard drawing layer. The blackboard drawing layer determines the first batch of annotations at the relative position of the document body in the visible area based on the first sliding distance, and renders the first batch of annotations. For example, the third coordinate of the elliptical circle 16 in Figure 5 falls within the coordinate range of the rectangular area of ​​(A+k, 0), (A+k, N), (B+k, 0) and (B+k, N) in the document body 11, that is, the elliptical circle 16 is the first batch of annotations located in this rectangular area. When this rectangular area of ​​the document body 11 slides into the visible area 12, the blackboard drawing layer renders the elliptical circle 16 so that the elliptical circle 16 is displayed in the visible area 12, achieving a follow-up sliding effect between the document body 11 and the first batch of annotations at the relative position.

[0051] When the document display application is in brush mode, the blackboard drawing layer monitors for a second slide event and controls the synchronized sliding of the document body and the corresponding first batch of annotations based on a second slide distance corresponding to the second slide event. The second slide event is triggered by a first slide operation on the document body, such as mouse scrolling, mouse dragging, or multi-finger touch, while the document display application is in brush mode and the application interface displays the document body. When the user uses multi-finger touch to slide the document, the infrared touch frame simultaneously captures multiple, discontinuous slide touch data. If the document display application is in brush mode and the application interface displays the document body, the user input can be determined to be the first slide operation, not the first batch of annotations. When the blackboard drawing layer monitors for the second slide event, it determines a second slide distance based on the first slide operation and controls the synchronized sliding of the document body and the corresponding first batch of annotations by the second slide distance. The synchronized sliding is implemented by: the blackboard drawing layer incrementally renders the first batch of annotations that slide into the visible area based on the second slide distance; and synchronizing the second slide distance to the first document object model, causing the first document object model to slide the document body within the visible area based on the second slide distance. Exemplarily, when the user slides the document body downward by a second sliding distance of k through a first sliding operation, the blackboard drawing layer determines the first batch of annotations at the relative position of the document body in the visible area based on the second sliding distance, and renders the first batch of annotations. At the same time, the blackboard drawing layer synchronizes the second sliding distance to the first document object model, and the first document object model slides the canvas from bottom to top by k to slide the document body in the visible area. After the sliding is completed, the content of the rectangular area (A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body is displayed in the visible area 12, and the visual effect after the sliding is completed is also shown in Figure 5. Therefore, regardless of whether the document display application is in selection mode or erase mode, after the user inputs the first sliding operation, the document display application can control the document body and the first batch of annotations in the application interface to slide synchronously, achieving the visual effect that the first batch of annotations slides with the document body.

[0052] S120. In response to the attachment display operation, save the first batch of annotations corresponding to the document body and clear the first batch of annotations displayed on the blackboard drawing layer, confirm the attachment elements selected in the attachment display operation in the document file, and display the attachment elements in full screen.

[0053] Exemplarily, when an attachment element embedded in the document body slides into the visible area, the contents of the attachment element are displayed in the visible area, with the attachment element embedded in the document body in thumbnail form. Referring to Figure 5 , the first attachment element 13 is an image, and the second attachment element 14 is a third-party service page embedded in the document body 11 via an inline frame (iframe), such as an online document or online video from a video website. In this embodiment, the attachment display operation refers to the operation of displaying the attachment element in full screen. For attachment elements directly embedded in the document body, such as images, the user double-clicks the attachment element to enter the attachment display operation. For attachment elements embedded in the document body via inline frames, such as third-party service pages, the user can click the full-screen display control 17 (shown in Figure 5) set in the inline frame to enter the attachment display operation, and confirm the selected attachment element according to the attachment display operation. At the same time, the first coordinate of the document body within the visible area is saved so that after subsequently exiting the full-screen display state of the attachment element, the visible area can still display the content of the document body in the visible area before the attachment element was displayed in full screen, making it easier for the user to continue reading the content of the document body. All the first batch of annotations corresponding to the document body are saved so that the annotation traces in the document body can be restored when the full-screen display state of the attachment element is subsequently exited, avoiding the user from repeatedly entering the same annotation content and improving the user's annotation experience. When saving the first batch of annotations of the document body, the first batch of annotations are marked by the unique identifier of the document body, and then the first batch of annotations are stored in a storage unit. The first batch of annotations corresponding to the document body can be read from the storage unit based on the unique identifier of the document body. After saving the first batch of annotations of the document body and the first coordinates in the visible area, the first batch of annotations in the blackboard drawing layer are cleared, and the attachment element selected by the attachment display operation is displayed in full screen. It should be noted that the attachment element displayed in full screen in this embodiment can be displayed in full screen in the application interface of the document display application, or in full screen in the screen of the interactive tablet. When the attachment element is displayed in full screen on the screen of the interactive tablet, the application interface is also displayed in full screen on the screen of the interactive tablet. The attachment element expanded to full screen is still displayed in the display layer of the document file. The blackboard drawing layer is located above the attachment element. After clearing the first batch of annotations in the blackboard drawing layer, the blackboard drawing layer will not affect the user's reading of the attachment element, and the user does not need to erase the first batch of annotations before adding annotations to the attachment element, which optimizes the user's annotation experience of the attachment element.

[0054] In this embodiment, the third-party service page refers to the web page provided by the third-party server corresponding to the attachment element. For example, when the attachment element is an online document, the corresponding third-party server is the document website that provides the online document. When the attachment element is an online video, the corresponding third-party server is the video website that provides the online video. The attachment element is embedded in the document body through an inline frame, that is, the page provided by the third-party server is loaded in the form of an inline frame embedded in the web page. At this time, there may be a cross-domain problem between the URL of the document display application and the URL of the third-party server. The cross-domain problem will cause the document display application to be unable to control the page sliding of the third-party server when responding to the sliding operation, that is, the blackboard drawing layer and the third-party service page as the attachment element cannot achieve sliding tracking, affecting the user's reading experience of the attachment element.

[0055] To solve the cross-domain problem, a proxy server in the same domain as the document display application can be built, and the URL of the inline frame can be changed to point to the proxy server, and the resources of the third-party server can be loaded through the proxy server. The specific implementation process is: the source address of the inline frame call is configured to the address of the proxy server, so as to obtain and return the third-party service page through the proxy server, and the third-party service page returned by the proxy server is loaded through the inline frame; the second document object model of the third-party service page is obtained through the inline frame. Exemplarily, the source address (src) of the inline frame is configured to point to the address of the proxy server, so as to load the web page of the third-party server through the proxy server, and the proxy server returns the web page of the third-party server to the document display application. The document display application loads the web page of the third-party server through the inline frame, so as to realize that the third-party service page is embedded in the document body through the inline frame. After loading the third-party service page, the inline frame can obtain the second document object model of the third-party service page through iframe.contentWindow.document, and control the sliding of the third-party service page based on the second document object model to realize the sliding follow-up of the blackboard drawing layer and the third-party service page, thereby optimizing the user's reading and annotation experience of the attachment elements.

[0056] S130 , generating second annotation content of the attachment element in response to the second annotation operation, and displaying the second annotation content in the cleared blackboard drawing layer.

[0057] Among them, the second annotation operation can be understood as the operation input by the user when annotating the attachment element. After the attachment element is displayed in full screen on the application interface of the document display application, the user uses the stylus tip to write in the application interface, and the elastic wave module determines that the current touch object is the stylus tip, and switches the document display application to brush mode. When the document display application is in brush mode and the application interface displays the attachment element in full screen, it is confirmed that the operation currently input by the user is the second annotation operation, and the touch data collected by the infrared touch frame is used to generate the second annotation content corresponding to the second annotation operation, and the second annotation content is rendered in the blackboard drawing layer that has cleared the first batch of annotation content. At this time, the blackboard drawing layer only displays the second annotation content corresponding to the attachment element. It should be noted that both the document body and the attachment elements after full screen are rendered in the document file display layer.

[0058] Similarly, in order to enable the second annotation content to be displayed at the relative position of the corresponding attachment element when the attachment element is displayed full screen again later, its coordinates in the corresponding attachment element will be determined when the second annotation content is generated and rendered, so as to restore the display of the second annotation content based on the coordinates. Figure 6 is a flowchart of generating and rendering the second annotation content provided by an embodiment of the present application. As shown in Figure 6, the steps of displaying the second annotation content specifically include S1301-S1302:

[0059] S1301: Determine, based on the fourth coordinate of the attachment element in the visible area and the fifth coordinate of the second annotation operation in the visible area, the sixth coordinate of the second annotation content generated by the second annotation operation in the attachment element.

[0060] FIG7 is a first schematic diagram of the second attachment element provided in an embodiment of the present application. As shown in FIG7 , the horizontal coordinate range of the second attachment element 14 is 0 to W, and the vertical coordinate range is 0 to H. The content within the rectangular area (0,0), (0,W), (C,0) and (C,W) of the second attachment element 14 is displayed in the visible area 12, and (0,0), (0,W), (C,0) and (C,W) can be determined as the fourth coordinates of the second attachment element 14 within the visible area 12. It should be noted that from a visual effect point of view, the user can only see the second attachment element 14 within the visible area 12, that is, the content within the rectangular area (0,0), (0,W), (C,0) and (C,W) of the second attachment element 14. The content in other areas is not visible. FIG7 shows the content of other areas only for the convenience of describing the embodiment. The coordinates marked in FIG7 are generated based on the coordinate system of the second attachment element 14. The coordinate system of the second attachment element 14 takes the upper left corner of the second attachment element 14 as the origin.

[0061] When a user uses the stylus tip to annotate second accessory element 14 within visible area 12, triggering a second annotation operation, the second annotation content is generated through touch data collected by the infrared touch frame. The coordinates of the second annotation operation on the interactive flat panel display are determined based on the touch data. Based on the coordinates of the second annotation operation on the interactive flat panel display and the coordinates of the visible area on the interactive flat panel display, the fifth coordinates of the second annotation operation within visible area 12 are determined. The fifth coordinates are generated based on the coordinate system of visible area 12. Subsequently, the fifth coordinates of the second annotation operation within visible area 12 are converted from the coordinate system of visible area 12 to the coordinate system of second accessory element 14, obtaining the sixth coordinates of the second annotation content within second accessory element 14.

[0062] S1302. Draw the second annotation content generated corresponding to the second annotation operation in the blackboard drawing layer according to the fifth coordinate of the second annotation operation in the visible area. The blackboard drawing layer is configured in the visible area.

[0063] Exemplarily, the coordinate system of the blackboard drawing layer is the same as the coordinate system of the visible area. Based on the fifth coordinate of the second annotation operation in the visible area, the pixel coordinates of the second annotation content generated in the blackboard drawing layer can be determined. Based on the pixel coordinates of the second annotation content in the blackboard drawing layer, the second annotation content is rendered in the blackboard drawing layer. Referring to Figure 7, if the second annotation operation is to underline a line of text in the second attachment element 14 in the visible area 12, then when the second annotation content generated by the second annotation operation is rendered in the blackboard drawing layer, an underline is displayed at the corresponding text.

[0064] If the length of the attachment element when displayed in full screen is greater than the length of the visible area, the visible area cannot fully display the content of the attachment element. At this time, the user can input a second sliding operation to the interactive tablet to adjust the content displayed in the visible area. Similarly, for an interactive tablet equipped with an elastic wave module, when the elastic wave module recognizes that the current touching object is a finger, the finger sliding is determined based on the touch data collected by the infrared touch frame, and when it is determined that the attachment element is displayed in full screen in the visible area, the operation currently input by the user is confirmed to be the second sliding operation. The sliding distance of the second sliding operation input is determined based on the touch data collected by the infrared touch frame, and the attachment element and the second annotation content are synchronously slid based on the sliding distance, so that the attachment element and the second annotation content at the relative position are displayed together in the visible area.

[0065] In another embodiment, for an interactive tablet without an elastic wave module, when the document display application is in different modes, different responses are made to the second sliding operation. This embodiment is described using the example of the second sliding operation corresponding to sliding an attachment element as a third-party service page.

[0066] When the document display application is in selection mode, the third sliding event of the third-party service page is listened to through the second document object model of the third-party service page, and the third-party service page and the corresponding second annotation content are controlled to slide synchronously according to the third sliding distance corresponding to the third sliding event. Among them, the third sliding event refers to the sliding event triggered by the user's second sliding operation of sliding the attachment element by scrolling the mouse, dragging the mouse, or touching the screen when the document display application is in selection mode and the application interface displays the attachment element in full screen. When the second document object model listens to the third sliding event through iframe.contentWindow.document.addEventListener('scroll'), the third sliding distance is determined according to the second sliding operation. In this embodiment, the attachment element is displayed in the form of full rendering, that is, the attachment element will be rendered in advance in a canvas that matches the size of the attachment element, and the sliding of the attachment element is achieved by sliding the canvas by the third sliding distance. The second annotation content is rendered only in the visible area in the form of incremental rendering. The specific implementation process is: slide the third-party service page in the visible area according to the third sliding distance through the second document object model; synchronize the third sliding distance to the blackboard drawing layer, so that the blackboard drawing layer renders the second annotation content that slides into the visible area according to the third sliding distance increment. Figure 8 is a second schematic diagram of the second attachment element provided by an embodiment of the present application. As shown in Figure 8, when the user slides the second attachment element 14 downward through the second sliding operation, the third sliding distance generated corresponding to the second sliding operation is g, and the second document object module slides the second attachment element 14 on the canvas from bottom to top by g to slide the second attachment element 14 in the visible area 12. After the sliding is completed, the content in the rectangular area of ​​(g,0), (g,W), (C+g,0) and (C+g,W) of the second attachment element is displayed in the visible area 12. At the same time, the second document object model synchronizes the second sliding distance to the blackboard drawing layer, and the blackboard drawing layer determines the second annotation content at the relative position of the second attachment element 14 in the visible area according to the third sliding distance, and renders the second annotation content. For example, the sixth coordinate of the rectangle 19 in Figure 8 in the second attachment element 14 falls within the coordinate range of the rectangular area of ​​(g, 0), (g, W), (C+g, 0) and (C+g, W), that is, the rectangle 19 is the second annotation content located in this rectangular area. When this rectangular area of ​​the second attachment element 14 slides into the visible area 12, the blackboard drawing layer renders the rectangle 19 so that the rectangle 19 is displayed in the visible area 12, achieving the follow-up sliding effect of the second attachment element 14 and the second annotation content at the relative position.

[0067] When the document display application is in brush mode, the blackboard drawing layer monitors for a fourth slide event and, based on the fourth slide distance corresponding to the fourth slide event, controls the synchronization of sliding the third-party service page and the corresponding second annotation content. The fourth slide event is triggered by a second slide operation (mouse scrolling, mouse dragging, or multi-finger touch screen) that involves sliding the document when the document display application is in brush mode and the application interface displays the attachment element in full screen. When the user uses multi-finger touch screen to slide the document, the infrared touch frame captures multiple, discontinuous slide touch data. If the document display application is in brush mode and the application interface displays the attachment element in full screen, the user can determine that the current user input is the second slide operation, not the second annotation operation. When the blackboard drawing layer monitors the fourth slide event, it determines the fourth slide distance based on the second slide operation and controls the synchronization of sliding the attachment element and the corresponding second annotation content by the second slide distance. The specific implementation process of the synchronized sliding is as follows: the blackboard drawing layer incrementally renders the second annotation content that slides into the visible area based on the fourth slide distance; and synchronizes the fourth slide distance with the second document object model, causing the second document object model to slide the document body within the visible area based on the fourth slide distance. Exemplarily, when the user slides the second attachment element downward by the fourth sliding distance g through the second sliding operation, the blackboard drawing layer determines the second annotation content at the relative position of the second attachment element in the visible area based on the fourth sliding distance, and renders the second annotation content. At the same time, the blackboard drawing layer synchronizes the fourth sliding distance to the second document object model, and the second document object model slides the canvas from bottom to top by g to slide the second attachment element in the visible area. After the sliding is completed, the content of the rectangular area (g,0), (g,W), (C+g,0) and (C+g,W) of the second attachment element is displayed in the visible area 12, and the visual effect after the sliding is completed is also shown in Figure 8. Therefore, regardless of whether the document display application is in selection mode or erase mode, after the user inputs the second sliding operation, the document display application can control the attachment element and the second annotation content in the application interface to slide synchronously, achieving the visual effect that the second annotation content slides with the attachment element.

[0068] S140. In response to the attachment closing operation, save the second annotation content corresponding to the attachment element and clear the second annotation content on the blackboard drawing layer, display the document body and display the saved first annotation content on the blackboard drawing layer.

[0069] Referring to Figure 7, after the second attachment element 14 is displayed full screen in the application interface of the document display application, an attachment close control 18 is provided in the application interface. The user clicks the attachment close control to enter the attachment close operation, exits the full screen display state of the second attachment element 14 according to the attachment close operation, and redisplays the document body. When the attachment close operation is received, the fourth coordinate of the attachment element in the visible area is saved so that when the second attachment element returns to the full screen display state, the visible area can still display the content of the second attachment element displayed in the visible area before exiting the full screen display state, facilitating the user to continuously read the content of the attachment element. All second annotation content corresponding to the attachment element is saved so that when the attachment element returns to the full screen display state, the annotation traces in the attachment element can be restored, avoiding the user from repeatedly entering the same annotation content and improving the user's annotation experience. When saving all second annotation content corresponding to the attachment element, the corresponding second annotation content is marked using the unique identifier of the attachment element, and then the second annotation content is stored in a storage unit. The corresponding second annotation content can be subsequently read from the storage space based on the unique identifier of the attachment element. Furthermore, after saving the fourth coordinate of the attachment element in the visible area and the second annotation content, the second annotation content in the blackboard drawing layer is cleared, the full-screen display state of the attachment element is exited, and the document body is displayed in the visible area. It can be understood that after clearing the second annotation content in the blackboard drawing layer, the blackboard drawing layer will not affect the user's reading experience of the document body, and the user does not need to erase the second annotation content before adding annotations to the document body, which optimizes the user's annotation experience of the document body.

[0070] After exiting the full-screen display state of the attachment element, the document body at the relative position can be displayed in the visible area based on the saved first coordinates, and the first batch of annotations at the relative position can be rendered in the blackboard drawing layer configured in the visible area. Referring to Figure 5, when the rectangular area (A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body is displayed in the visible area 12, the user double-clicks the first attachment element 13 to enter the attachment display operation, and based on the attachment display operation, the first coordinates of the document body 11 in the visible area 12 are saved, that is, the coordinates (A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body 11 are saved, and then the first attachment element 13 is displayed in full screen in the visible area. After exiting the full-screen display state of the first attachment element 13, the contents of the rectangular area of ​​(A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body 11 are displayed in the visible area 12 based on the saved (A+k,0), (A+k,N), (B+k,0) and (B+k,N). The third coordinate of the elliptical circle 16 in the document body 11 falls within the coordinate range of the rectangular area of ​​(A+k,0), (A+k,N), (B+k,0) and (B+k,N), that is, the elliptical circle 16 is the first batch of annotation content located in this rectangular area. The elliptical circle 16 is rendered by clearing the blackboard drawing layer after the second annotation content so that the elliptical circle 16 and the contents of the rectangular area of ​​(A+k,0), (A+k,N), (B+k,0) and (B+k,N) of the document body are displayed in the visible area together, so as to restore the annotation traces of the document body and optimize the user's reading and annotation experience of the document body.

[0071] After the user exits the full-screen display state of the attachment element, if he wants to view the content of the attachment element again, he can re-enter the attachment display operation for the attachment element. Based on the attachment display operation, the content of the attachment element displayed in the visible area during the last full-screen display state will be re-displayed in the visible area, and the second annotation content at the relative position will be displayed in the visible area. The specific implementation process is: in response to the new attachment display operation, the first batch of annotation content corresponding to the document body is saved and the first batch of annotation content displayed on the blackboard drawing layer is cleared, and the first coordinate of the document body in the visible area is saved; based on the saved fourth coordinate, the additional element at the relative position is displayed in the visible area, and the second annotation content at the relative position is rendered in the blackboard drawing layer configured in the visible area. 8 , when the rectangular area (g, 0), (g, W), (C+g, 0), and (C+g, W) of the second attachment element 14 is displayed in the visible area 12, the user clicks on the additional close control (not shown in FIG8 ) in the visible area to input the attachment close operation. Based on the attachment close operation, the fourth coordinates of the second attachment element 14 in the visible area are saved, that is, the coordinates (g, 0), (g, W), (C+g, 0), and (C+g, W) of the second attachment element 14 are saved, and then the full-screen display state of the second attachment element 14 is exited. After the user clicks on the full-screen display control 17 of the second attachment element 14 again to input a new attachment display operation, the first coordinates of the document body in the visible area are saved in response to the attachment display operation, the first batch of annotations of the document body are saved, and the first batch of annotations of the blackboard drawing layer are cleared. Then, based on the fourth coordinates of the second attachment element 14 saved last time in the visible area 12, namely (g,0), (g,W), (C+g,0), and (C+g,W), the contents of the rectangular area (g,0), (g,W), (C+g,0), and (C+g,W) of the second attachment element 14 are displayed in the visible area 12. The sixth coordinate of rectangle 19 in the second attachment element 14 falls within the coordinate range of the rectangular area (g,0), (g,W), (C+g,0), and (C+g,W), that is, rectangle 19 is the second annotation content located in this rectangular area. Rectangle 19 is rendered by clearing the blackboard drawing layer after the first annotation content is cleared so that rectangle 19 and the contents of the rectangular area (g,0), (g,W), (C+g,0), and (C+g,W) of the second attachment element 14 are displayed in the visible area together, thereby restoring the annotation traces of the second attachment element and optimizing the user's reading and annotation experience of the attachment element. If the user wants to modify the second annotation content at a certain position of the second attachment element, the second attachment element and the second annotation content at that position can be slid into the visible area through a second sliding operation, the document display application can be switched to eraser mode, the second annotation content can be erased through an erasing operation, and new second annotation content can be added through a second annotation operation.

[0072] When the document display application closes a document file, the document display application prompts the user through a pop-up window whether to save the annotation content of the document file. If the user selects yes, the first batch of annotation content and the second batch of annotation content stored in the storage unit are persistently saved so that the annotation traces of the document body and attachment elements can be restored based on the first batch of annotation content and the second batch of annotation content in the storage unit when the document file is opened next time. If the user selects no, the first batch of annotation content and the second batch of annotation content in the storage unit are cleared.

[0073] In summary, the annotation method provided by the embodiment of the present application can, when displaying the attachment elements in the document file in full screen, clear the first batch of annotation contents in the blackboard drawing layer after the document body is associated with the corresponding first batch of annotation contents and saved, that is, the first batch of annotation contents and the attachment elements expanded in full screen will not be displayed at the same time, avoiding confusion of the annotation contents. Since the blackboard drawing layer is always displayed in the upper layer of the document file display layer, even when the attachment elements are displayed in full screen, the blackboard drawing layer is still located in the upper layer of the attachment elements, and the second attachment content corresponding to the attachment elements can be displayed through the blackboard drawing layer, realizing the function of annotating the attachment elements displayed in full screen. When exiting the full-screen display state of the attachment elements, the second annotation content in the blackboard drawing layer can be cleared after the attachment elements are associated with the corresponding second annotation and saved, that is, the second annotation content and the document body will not be displayed at the same time, avoiding confusion of the annotation contents. Since the document file is associated with the corresponding first batch of annotation contents and saved, the document file can be displayed together with the corresponding first batch of annotation contents, retaining the annotation traces of the document file, avoiding users from repeatedly annotating the document file. Similarly, the attachment element is saved in association with the corresponding second annotation content. When the attachment element is displayed in full screen, the attachment element and the corresponding second annotation content can be displayed together, retaining the annotation traces of the attachment element and avoiding users from repeatedly annotating the attachment element. This application implements the annotation function of the document body and attachment elements by saving the annotation content corresponding to the document file and the attachment element, and promptly clearing the annotation content on the blackboard drawing layer. It solves the problem in the prior art that it is impossible to annotate attachment elements displayed in full screen, improves the annotation function of the document display application, improves the richness of annotation interaction and the reusability of annotation content, and optimizes the user's annotation and reading experience of documents.

[0074] Based on the above embodiment, FIG9 is a schematic diagram of the structure of an annotation device provided by the present application. Referring to FIG9 , the annotation device provided by the present embodiment specifically includes: a first annotation module 21 , an attachment display module 22 , a second annotation module 23 and a main body display module 24 .

[0075] The first batch annotation module 21 is configured to display the document body of the document file, generate the first batch annotation content of the document body in response to the first batch annotation operation, and display the first batch annotation content in the blackboard drawing layer, which is located above the display layer of the document file;

[0076] The attachment display module 22 is configured to save the first batch of annotations corresponding to the document body and clear the first batch of annotations displayed on the blackboard drawing layer in response to the attachment display operation, confirm the attachment element selected in the document file by the attachment display operation, and display the attachment element in full screen;

[0077] The second annotation module 23 is configured to generate second annotation content of the attachment element in response to the second annotation operation, and display the second annotation content in the cleared blackboard drawing layer;

[0078] The main body display module 24 is configured to save the second annotation content corresponding to the attachment element and clear the second annotation content on the blackboard drawing layer in response to the attachment closing operation, display the document main body and display the saved first annotation content on the blackboard drawing layer.

[0079] Based on the above embodiment, the first batch annotation module 21 includes: a first position determination unit, configured to determine the third coordinates in the document body corresponding to the first batch annotation content generated by the first batch annotation operation based on the first coordinates of the document body in the visible area and the second coordinates of the first batch annotation operation in the visible area; a first rendering unit, configured to render the first batch annotation content generated by the first batch annotation operation in the blackboard drawing layer based on the second coordinates of the first batch annotation operation in the visible area, and the blackboard drawing layer is configured in the visible area.

[0080] On the basis of the above embodiment, the annotation device also includes a first sliding module, which includes: a first synchronous sliding unit, which is configured to monitor the first sliding event of the document body through the first document object model of the document body when in selection mode, and control the document body and the corresponding first batch of annotation contents to slide synchronously according to the first sliding distance corresponding to the first sliding event; a second synchronous sliding unit, which is configured to monitor the second sliding event through the blackboard drawing layer when in brush mode, and control the document body and the corresponding first batch of annotation contents to slide synchronously according to the second sliding distance corresponding to the second sliding event.

[0081] Based on the above embodiment, the first synchronous sliding unit includes: a first main body sliding sub-unit, configured to slide the document main body within the visible area according to the first sliding distance through the first document object model; a first batch annotation sliding sub-unit, configured to synchronize the first sliding distance to the blackboard drawing layer, so that the blackboard drawing layer renders the first batch of annotation content that slides into the visible area according to the first sliding distance increment.

[0082] Based on the above embodiment, the second synchronous sliding unit includes: a second annotation sliding sub-unit, which is configured to render the first batch of annotation content that is slid into the visible area through the blackboard drawing layer according to the second sliding distance increment; a second body sliding sub-unit, which is configured to synchronize the second sliding distance to the first document object model so that the first document object model slides the document body within the visible area according to the second sliding distance.

[0083] Based on the above embodiment, the second annotation module 23 includes: a second position determination unit, configured to determine the sixth coordinate in the attachment element corresponding to the second annotation content generated corresponding to the second annotation operation based on the fourth coordinate of the attachment element in the visible area and the fifth coordinate of the second annotation operation in the visible area; a second rendering unit, configured to draw the second annotation content generated corresponding to the second annotation operation in the blackboard drawing layer based on the fifth coordinate of the second annotation operation in the visible area, and the blackboard drawing layer is configured in the visible area.

[0084] On the basis of the above embodiment, when the attachment element is a third-party service page, the attachment element is embedded in the document body through an inline frame; accordingly, the annotation device also includes an attachment element embedding module, and the attachment element embedding module includes: a proxy unit, which is configured to configure the source address of the inline frame call to the address of the proxy server before responding to the attachment display operation, so as to obtain and return the third-party service page through the proxy server, and load the third-party service page returned by the proxy server through the inline frame; a model acquisition unit, which is configured to obtain the second document object model of the third-party service page through the inline frame.

[0085] On the basis of the above embodiment, the annotation device also includes a second sliding module, which includes: a third synchronous sliding unit, which is configured to monitor the third sliding event of the third-party service page through the second document object model of the third-party service page when in selection mode, and control the third-party service page and the corresponding second annotation content to slide synchronously according to the third sliding distance corresponding to the third sliding event; a fourth synchronous sliding unit, which is configured to monitor the fourth sliding event through the blackboard drawing layer when in brush mode, and control the third-party service page and the corresponding second annotation content to slide synchronously according to the fourth sliding distance corresponding to the fourth sliding event.

[0086] Based on the above embodiment, the third synchronous sliding unit includes: a first attachment sliding unit, which is configured to slide the third-party service page in the visible area according to the third sliding distance through the second document object model; a third annotation sliding unit, which is configured to synchronize the third sliding distance to the blackboard drawing layer, so that the blackboard drawing layer renders the second annotation content that slides into the visible area according to the third sliding distance increment.

[0087] Based on the above embodiment, the fourth synchronous sliding unit includes: a fourth annotation sliding unit, which is configured to render the second annotation content that slides into the visible area through the blackboard drawing layer according to the fourth sliding distance increment; a second attachment sliding unit, which is configured to synchronize the fourth sliding distance to the second document object model so that the second document object model slides the document body within the visible area according to the fourth sliding distance.

[0088] Based on the above embodiment, the attachment display module 22 also includes: a first position saving unit, configured to save the first coordinate of the document body in the visible area after responding to the attachment display operation; correspondingly, the body display module 24 includes: a first batch annotation recovery unit, configured to display the document body at a relative position in the visible area based on the saved first coordinate, and render the first batch annotation content at the relative position in the blackboard drawing layer configured in the visible area.

[0089] On the basis of the above embodiment, the main body display module 24 further includes: a second position saving unit, which is configured to save the fourth coordinates of the attachment element in the visible area after responding to the attachment closing operation; correspondingly, the attachment display module 22 further includes: an information saving unit, which is configured to save the first batch of annotation content corresponding to the document body and clear the first batch of annotation content displayed on the blackboard drawing layer in response to a new attachment display operation, and save the first coordinates of the document body in the visible area. The second annotation recovery unit is configured to display the additional element at a relative position in the visible area based on the saved fourth coordinates, and render the second annotation content at the relative position in the blackboard drawing layer configured in the visible area.

[0090] As mentioned above, the annotation device provided by the embodiment of the present application can, when displaying the attachment elements in the document file in full screen, clear the first batch of annotation contents in the blackboard drawing layer after saving the document body in association with the corresponding first batch of annotation contents, that is, the first batch of annotation contents and the attachment elements expanded in full screen will not be displayed at the same time, avoiding confusion of the annotation contents. Since the blackboard drawing layer is always displayed in the upper layer of the document file display layer, even when the attachment elements are displayed in full screen, the blackboard drawing layer is still located in the upper layer of the attachment elements, and the second attachment content corresponding to the attachment elements can be displayed through the blackboard drawing layer, realizing the function of annotating the attachment elements displayed in full screen. When exiting the full-screen display state of the attachment elements, the second annotation content in the blackboard drawing layer can be cleared after saving the attachment elements in association with the corresponding second annotation, that is, the second annotation content and the document body will not be displayed at the same time, avoiding confusion of the annotation contents. Since the document file is saved in association with the corresponding first batch of annotation contents, the document file can be displayed together with the corresponding first batch of annotation contents, retaining the annotation traces of the document file, avoiding users from repeatedly annotating the document file. Similarly, the attachment element is saved in association with the corresponding second annotation content. When the attachment element is displayed in full screen, the attachment element and the corresponding second annotation content can be displayed together, retaining the annotation traces of the attachment element and avoiding users from repeatedly annotating the attachment element. This application implements the annotation function of the document body and attachment elements by saving the annotation content corresponding to the document file and the attachment element, and promptly clearing the annotation content on the blackboard drawing layer. It solves the problem in the prior art that it is impossible to annotate attachment elements displayed in full screen, improves the annotation function of the document display application, improves the richness of annotation interaction and the reusability of annotation content, and optimizes the user's annotation and reading experience of documents.

[0091] The annotation device provided in the embodiment of the present application can be used to execute the annotation method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0092] FIG10 is a schematic diagram of the structure of an annotation device provided in an embodiment of the present application. Referring to FIG10 , the annotation device includes: a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 in the annotation device can be one or more, and the number of memories 32 in the annotation device can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the annotation device can be connected via a bus or other means.

[0093] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the annotation method of any embodiment of the present application (for example, the first annotation module 21, the attachment display module 22, the second annotation module 23 and the main body display module 24 in the annotation device). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0094] The communication device 33 is used for data transmission.

[0095] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, implements the above-mentioned annotation method.

[0096] The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0097] The annotation device provided above can be used to execute the annotation method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0098] An embodiment of the present application also provides a storage medium containing computer-executable instructions, which are used to execute an annotation method when executed by a computer processor. The annotation method includes: displaying the document body of a document file, generating a first batch of annotation content of the document body in response to a first batch of annotation operations, and displaying the first batch of annotation content in a blackboard drawing layer, which is located above the display layer of the document file; responding to an attachment display operation, saving the first batch of annotation content corresponding to the document body and clearing the first batch of annotation content displayed on the blackboard drawing layer, confirming the attachment element selected in the document file by the attachment display operation, and displaying the attachment element in full screen; responding to a second annotation operation, generating a second annotation content of the attachment element, and displaying the second annotation content in the cleared blackboard drawing layer; responding to an attachment closing operation, saving the second annotation content corresponding to the attachment element and clearing the second annotation content on the blackboard drawing layer, displaying the document body and displaying the saved first batch of annotation content on the blackboard drawing layer.

[0099] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROMs, floppy disks, or tape drives; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or it may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0100] Of course, the storage medium containing computer-executable instructions provided in the embodiment of the present application, whose computer-executable instructions are not limited to the above annotation method, can also execute related operations in the annotation method provided in any embodiment of the present application.

[0101] The annotation apparatus, storage medium and annotation device provided in the above embodiments can execute the annotation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the annotation method provided in any embodiment of the present application.

[0102] The above are only preferred embodiments of the present application and the technical principles employed. The present application is not limited to the specific embodiments described herein, and any obvious changes, readjustments, and substitutions that are apparent to those skilled in the art will not depart from the scope of protection of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the claims.

Claims

1. A method for annotation, wherein: include: Display the document body of the document file, generate the first batch of annotation contents of the document body in response to the first batch of annotation operations, and display the first batch of annotation contents in a blackboard drawing layer, which is located on the upper layer of the display layer of the document file; In response to the attachment display operation, the first batch of annotations corresponding to the document body is saved and the first batch of annotations displayed on the blackboard drawing layer is cleared, the attachment element selected in the document file by the attachment display operation is confirmed, and the attachment element is displayed in full screen; In response to the second annotation operation, second annotation content of the attachment element is generated, and the second annotation content is displayed in the cleared blackboard drawing layer; In response to the attachment closing operation, the second annotation content corresponding to the attachment element is saved and the second annotation content on the blackboard drawing layer is cleared, the document body is displayed and the saved first annotation content is displayed on the blackboard drawing layer.

2. The annotation method according to claim 1, wherein: The step of generating the first batch of annotation contents of the document body in response to the first batch of annotation operations and displaying the first batch of annotation contents in the blackboard drawing layer comprises: Determine, based on the first coordinates of the document body in the visible area and the second coordinates of the first batch of annotation operations in the visible area, the third coordinates in the document body corresponding to the first batch of annotation contents generated corresponding to the first batch of annotation operations; According to the second coordinates of the first batch of annotation operations in the visible area, the first batch of annotation contents generated corresponding to the first batch of annotation operations are rendered in the blackboard drawing layer, and the blackboard drawing layer is arranged in the visible area.

3. The annotation method according to claim 1, wherein: The method further comprises: In the case of being in selection mode, monitoring a first sliding event of the document body through a first document object model of the document body, and controlling the document body and the corresponding first batch of annotation contents to slide synchronously according to a first sliding distance corresponding to the first sliding event; In the case of the brush mode, the second sliding event is monitored by the blackboard drawing layer, and the document body and the corresponding first batch annotation content are controlled to slide synchronously according to a second sliding distance corresponding to the second sliding event.

4. The annotation method according to claim 3, wherein: The step of controlling the document body and the corresponding first batch annotation content to slide synchronously according to the first sliding distance corresponding to the first sliding event includes: Sliding the document body in the visible area according to the first sliding distance by using the first document object model; The first sliding distance is synchronized to the blackboard drawing layer, so that the blackboard drawing layer renders the first batch of annotation contents that slide into the visible area according to the first sliding distance increment.

5. The annotation method according to claim 3, wherein: The step of controlling the document body and the corresponding first batch annotation content to slide synchronously according to the second sliding distance corresponding to the second sliding event includes: Rendering the first batch of annotation contents slid into the visible area according to the second sliding distance increment by the blackboard drawing layer; The second sliding distance is synchronized to the first document object model, so that the first document object model slides the document body in the visible area according to the second sliding distance.

6. The annotation method according to claim 1, wherein: The step of generating the second annotation content of the attachment element in response to the second annotation operation, and displaying the second annotation content in the cleared blackboard drawing layer includes: Determine, according to the fourth coordinate of the attachment element in the visible area and the fifth coordinate of the second annotation operation in the visible area, that the second annotation content generated corresponding to the second annotation operation corresponds to the sixth coordinate in the attachment element; According to the fifth coordinate of the second annotation operation in the visible area, the second annotation content generated corresponding to the second annotation operation is drawn in the blackboard drawing layer, and the blackboard drawing layer is arranged in the visible area.

7. The annotation method according to claim 1, wherein: In the case where the attachment element is a third-party service page, the attachment element is embedded in the document body through an inline frame; accordingly, before responding to the attachment display operation, the method further includes: configuring the source address of the inline frame call to be the address of the proxy server, so as to obtain and return the third-party service page through the proxy server, and loading the third-party service page returned by the proxy server through the inline frame; A second document object model of the third-party service page is obtained through the inline frame.

8. The annotation method according to claim 7, wherein: The method further comprises: In the case of being in the selection mode, monitoring a third sliding event of the third-party service page through the second document object model of the third-party service page, and controlling the third-party service page and the corresponding second annotation content to slide synchronously according to a third sliding distance corresponding to the third sliding event; In the case of the brush mode, the fourth sliding event is monitored through the blackboard drawing layer, and according to the fourth sliding distance corresponding to the fourth sliding event, the third-party service page and the corresponding second annotation content are controlled to slide synchronously.

9. The annotation method according to claim 8, wherein: The step of controlling the third-party service page to slide synchronously with the corresponding second annotation content according to the third sliding distance corresponding to the third sliding event includes: Sliding the third-party service page in the visible area according to the third sliding distance by using the second document object model; The third sliding distance is synchronized to the blackboard drawing layer, so that the blackboard drawing layer renders the second annotation content that slides into the visible area according to the third sliding distance increment.

10. The annotation method according to claim 8, wherein: The controlling the third-party service page to slide synchronously with the corresponding second annotation content according to the fourth sliding distance corresponding to the fourth sliding event includes: Rendering, by the blackboard drawing layer, second annotation content that slides into the visible area according to the fourth sliding distance increment; The fourth sliding distance is synchronized to the second document object model, so that the second document object model slides the document body in the visible area according to the fourth sliding distance.

11. The annotation method according to claim 1, wherein: After the display operation in response to the attachment, the method further includes: Save the first coordinate of the document body within the visible area; Correspondingly, the displaying of the document body and displaying the first batch of saved annotation contents on the blackboard drawing layer includes: Based on the saved first coordinates, the document body at the relative position is displayed in the visible area, and the first batch of annotation contents at the relative position is rendered in a blackboard drawing layer configured in the visible area.

12. The annotation method according to claim 1, wherein: After the closing operation in response to the attachment, the method further includes: Save the fourth coordinate of the attachment element in the visible area; Correspondingly, after displaying the document body and displaying the first batch of saved annotation content on the blackboard drawing layer, it also includes: In response to a new attachment display operation, the first batch of annotations corresponding to the document body is saved and the first batch of annotations displayed by the blackboard drawing layer is cleared, and the first coordinate of the document body in the visible area is saved; Based on the saved fourth coordinate, the additional element at the relative position is displayed in the visible area, and the second annotation content at the relative position is rendered in a blackboard drawing layer configured in the visible area.

13. An annotation device, wherein: include: The first batch annotation module is configured to display the document body of the document file, generate the first batch annotation content of the document body in response to the first batch annotation operation, and display the first batch annotation content in the blackboard drawing layer, which is located on the upper layer of the display layer of the document file; The attachment display module is configured to save the first batch of annotations corresponding to the document body and clear the first batch of annotations displayed by the blackboard drawing layer in response to the attachment display operation, confirm the attachment element selected in the document file by the attachment display operation, and display the attachment element in full screen; A second annotation module is configured to generate second annotation content of the attachment element in response to a second annotation operation, and display the second annotation content in the cleared blackboard drawing layer; The main body display module is configured to save the second annotation content corresponding to the attachment element and clear the second annotation content on the blackboard drawing layer in response to the attachment closing operation, display the document main body and display the saved first annotation content on the blackboard drawing layer.

14. An annotation device, wherein: include: one or more processors; The memory stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the annotation method according to any one of claims 1 to 12.

15. A storage medium comprising computer executable instructions, wherein: The computer executable instructions are used to perform the annotation method according to any one of claims 1 to 12 when executed by a computer processor.

Images