Annotating method and electronic device

Abstract

The application provides a comment method and an electronic device, and relates to the technical field of terminals, wherein the method comprises the following steps: displaying a first interface, the first interface comprising a first application interface and a second application interface displayed in split screens; receiving a first comment operation of a user on the first interface; in response to the first comment operation, displaying a first comment on the first interface; receiving a first scrolling operation of the user on the first interface; in response to the first scrolling operation, displaying a second interface, the second interface comprising the first application interface and the second application interface displayed in split screens, and the second interface being different from the first interface; receiving a second comment operation of the user on the second interface; in response to the second comment operation, displaying a second comment on the second interface; and generating a first comment picture, the first comment picture comprising the first comment, the second comment, the first interface and the second interface. The technical scheme provided by the application can enrich the application scenarios of the comment function and improve the flexibility of the user in making comments.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an annotation method and electronic device. Background Technology

[0002] Mobile phones, tablets and other electronic devices are increasingly used in people's work and study. To facilitate user use, some electronic devices provide annotation functions, which users can use to annotate text, images and other content.

[0003] For example, Figure 1 As shown in (a), the user can open the "Gallery" application (APP) on the tablet and control the tablet to enter annotation mode; after entering annotation mode, as shown in (a), the user can control the tablet to enter annotation mode. Figure 1 As shown in (b), users can annotate images on the annotation interface using a stylus. In addition, the annotation interface provides a toolbar, which users can use to adjust annotation content, etc.

[0004] However, the current annotation function has limited application scenarios, and users do not have enough freedom and flexibility in using it. Summary of the Invention

[0005] In view of this, this application provides an annotation method and electronic device to enrich the application scenarios of annotation functions and improve the flexibility of users in making annotations.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide an annotation method, including:

[0007] The first interface is displayed, which includes a first application interface and a second application interface displayed in a split screen.

[0008] Receive the first batch of annotation operations from the user on the first interface;

[0009] In response to the first batch of annotations operation, the first batch of annotations is displayed on the first interface;

[0010] Receive the user's first scrolling operation on the first interface;

[0011] In response to the first scrolling operation, a second interface is displayed, which includes a first application interface and a second application interface displayed in a split screen, and the second interface is different from the first interface.

[0012] Receive the user's second annotation operation on the second interface;

[0013] In response to the second annotation operation, the second annotation is displayed on the second interface;

[0014] Generate the first batch of annotation images, which include: the first batch of annotations, the second batch of annotations, the first interface, and the second interface.

[0015] When the electronic device displays the first application interface and the second application interface in a split-screen mode, it can receive the user's annotation trigger operation, enter the annotation mode, and display the annotation interface (i.e., the first interface mentioned above). The first interface can continue to display the first application interface and the second application interface in a split-screen mode, and can load an annotation layer on the displayed application interface. The annotation layer can be transparent and can also include a toolbar.

[0016] The first and second annotation operations mentioned above can be annotation operations performed by the user on the screen using a stylus or finger.

[0017] For example, when the third and fourth application interfaces are displayed side-by-side, if the user inputs an up or down scrolling operation, the electronic device can scroll the first and / or second application interfaces to achieve up and down scrolling annotation in a side-by-side split-screen scenario.

[0018] For example, when the third and fourth application interfaces are displayed in a split-screen manner, if the user inputs a scrolling operation to the left or right, the electronic device can scroll the first application interface and / or the second application interface to achieve left and right scrolling annotation in a split-screen scenario.

[0019] In the annotation method provided in this application embodiment, the electronic device can enter the annotation mode in a split-screen scenario, allowing the user to make annotations. In the annotation mode, the user can annotate the first and second application interfaces displayed in the split screen, and the user can scroll through the annotations. Therefore, this method can enrich the application scenarios of the annotation function, improve the flexibility of the user's annotation, and enhance the user's annotation experience.

[0020] In one possible implementation of the first aspect, displaying the second interface in response to the first scrolling operation specifically includes:

[0021] In response to the first scrolling operation, the first application interface displayed in split screen is switched from the first sub-interface of the first application to the second sub-interface of the first application.

[0022] In the above embodiments, the electronic device can respond to the user's scrolling operation and update the content of the displayed first application interface, so that the user can scroll the content of the first application interface in annotation mode, thereby improving the flexibility of the user's annotation.

[0023] In one possible implementation of the first aspect, the method further includes:

[0024] In response to the first scrolling operation, the second application interface displayed in split screen switches from the third sub-interface of the second application to the fourth sub-interface of the second application.

[0025] Through the above implementation method, the content of the interface in the first application interface and the second application interface can be scrolled synchronously, thereby improving the convenience of user use.

[0026] In one possible implementation of the first aspect, if the first scrolling operation is an upward scrolling operation, the first sub-interface in the first batch of annotation images is located above the second sub-interface, and the third sub-interface is located above the fourth sub-interface.

[0027] Through the above implementation method, users can scroll up to view the content of the interface behind the first application interface and the second application interface during the annotation process.

[0028] In one possible implementation of the first aspect, if the first scrolling operation is a downward scrolling operation, the first sub-interface in the first batch of annotation images is located below the second sub-interface, and the third sub-interface is located below the fourth sub-interface.

[0029] Through the above implementation method, users can scroll down to view the content of the interface in front of the first application interface and the second application interface during the annotation process.

[0030] In one possible implementation of the first aspect, if the first application interface is located to the left of the second application interface, the first sub-interface in the first batch of annotation images is located to the left of the third sub-interface, and the second sub-interface is located to the left of the fourth sub-interface.

[0031] If the first application interface is located to the right of the second application interface, then in the first batch of annotation images, the first sub-interface is located to the right of the third sub-interface, and the second sub-interface is located to the right of the fourth sub-interface.

[0032] The above implementation method allows users to perform scrolling annotations in a split-screen scenario.

[0033] In one possible implementation of the first aspect, the fourth sub-interface is a filled page. This allows users to utilize more annotation space for application interfaces that cannot be scrolled further.

[0034] In one possible implementation of the first aspect, if the first application interface further includes a first component fixedly displayed at the bottom of the first application interface, then the method further includes:

[0035] After receiving the user's first scrolling operation on the first interface, the first component is no longer displayed.

[0036] In the above embodiments, components such as the menu bar that are fixed at the bottom of the first application interface are not displayed during the scrolling annotation process. This allows users to view more interface content on the screen and improves the user's annotation experience.

[0037] In one possible implementation of the first aspect, if the first application interface further includes a second component fixedly displayed at the top of the first application interface, then the method further includes:

[0038] After receiving the user's first scrolling operation on the first interface, the second component is no longer displayed.

[0039] In the above embodiments, components such as the status bar that are fixedly displayed at the top of the first application interface are not displayed during the scrolling annotation process. This allows users to view more interface content on the screen and improves the user's annotation experience.

[0040] In one possible implementation of the first aspect, the method further includes:

[0041] The third interface is displayed, which includes a third application interface and a fourth application interface displayed in a split screen.

[0042] Receive the user's second scrolling operation on the third application interface displayed in the third interface;

[0043] In response to the second scrolling operation, the third application interface is displayed in full screen;

[0044] Receive user's third annotation operation on the third application interface;

[0045] In response to the third annotation operation, the third annotation is displayed on the third application interface;

[0046] A second annotation image is generated, which includes the third annotation and the third application interface.

[0047] For example, when the third and fourth application interfaces are displayed in a split-screen manner, the user can input an up or down scrolling operation, and the electronic device can display the application interface in full screen when the user scrolls to that application interface, thus realizing up and down scrolling annotation in a split-screen scenario.

[0048] For example, when the third and fourth application interfaces are displayed side-by-side, if the user inputs a scrolling operation to the left or right, the electronic device can display the application interface in full screen when the user scrolls, thus enabling left-right scrolling annotation in a side-by-side split-screen scenario.

[0049] The above implementation methods can further enrich the application scenarios of the annotation function, improve the flexibility of users in making annotations, and enhance the user's annotation experience.

[0050] In one possible implementation of the first aspect, the method further includes:

[0051] The fourth interface is displayed, which includes a fifth application interface and a sixth application interface displayed in split screen.

[0052] Receive user's fourth annotation operation on the fifth application interface or the sixth application interface;

[0053] In response to the fourth annotation operation, the fourth annotation is displayed on the fifth or sixth application interface;

[0054] Receive the user's third scrolling operation on the fifth application interface;

[0055] In response to the third scrolling operation, the fifth application interface is displayed in full screen;

[0056] A third annotation image is generated, which includes: the fifth application interface, the sixth application interface, and the fourth annotation, all displayed in full screen.

[0057] After the fifth application interface is displayed in full screen, the user can add annotations, scroll the interface, etc. on the fifth application interface. After the user enters the annotation and saves it, the electronic device can merge the fifth application interface displayed during the annotation process and the annotation added by the user on the fifth application interface into a single annotation image, and merge the sixth application interface displayed during the annotation process and the annotation added by the user on the sixth application interface into a single annotation image. Then, the two annotation images are stitched together and saved.

[0058] The above implementation methods can further enrich the application scenarios of the annotation function, improve the flexibility of users in making annotations, and enhance the user's annotation experience.

[0059] In one possible implementation of the first aspect, the method further includes:

[0060] The fifth interface is displayed, which includes a seventh application interface and an eighth application interface displayed in split screen.

[0061] Receive the user's fifth annotation operation on the fifth interface;

[0062] In response to the fifth annotation operation, the fifth annotation is displayed, which overlaps with the boundary line between the seventh and eighth application interfaces;

[0063] Receive the user's fourth scrolling operation on the seventh application interface;

[0064] In response to the fourth scrolling operation, a prompt interface is displayed for the user to choose whether to save the annotations;

[0065] In response to the user's selection, the seventh application interface is displayed in full screen;

[0066] When the selection operation is to select not to save annotations, a fourth annotation image is generated;

[0067] When the selection operation is to save the annotation, a fourth annotation image and a fifth annotation image are generated; the fourth annotation image includes the seventh application interface displayed in full screen and the fifth annotation, and the fifth annotation image includes the fifth interface and the fifth annotation.

[0068] The above implementation methods can further enrich the application scenarios of the annotation function, improve the flexibility of users in making annotations, and enhance the user's annotation experience.

[0069] In one possible implementation of the first aspect, the method further includes:

[0070] The sixth interface is displayed, which includes a ninth application interface that is displayed in full screen and a tenth application interface that is displayed floating on the ninth application interface.

[0071] In response to an annotation-triggered operation, the floating tenth application interface is hidden or the ninth and tenth application interfaces are displayed in a split-screen mode.

[0072] In the above embodiments, electronic devices can enable users to enter annotation mode in a floating window scenario, thereby further enriching the application scenarios of annotation functions, improving the flexibility of user annotation, and enhancing the user's annotation experience.

[0073] In a second aspect, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the first aspect or any embodiment of the first aspect when the computer program is invoked.

[0074] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any embodiment of the first aspect.

[0075] Fourthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in the first aspect or any embodiment of the first aspect.

[0076] Fifthly, embodiments of this application provide a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the method described in the first aspect or any embodiment of the first aspect. The chip system may be a single chip or a chip module composed of multiple chips.

[0077] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of an existing annotation scenario;

[0079] Figure 2 A flowchart illustrating the annotation method provided in the embodiments of this application;

[0080] Figure 3 A schematic diagram of an annotation scenario provided in an embodiment of this application;

[0081] Figure 4 This is a schematic diagram of the floating menu triggering process provided in an embodiment of this application;

[0082] Figures 5 to 17 Some annotation scenarios are illustrated in the embodiments of this application;

[0083] Figure 18 This is a schematic diagram of the structure of the interactive system provided in the embodiments of this application;

[0084] Figure 19 A schematic diagram of the software architecture of the electronic device provided in the embodiments of this application;

[0085] Figure 20 This is a schematic diagram illustrating the principle of scrolling annotations provided in an embodiment of this application.

[0086] Figure 21 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0087] Figure 22 This is a schematic diagram of the annotation device provided in the embodiments of this application. Detailed Implementation

[0088] To enrich the application scenarios of annotation functions and improve the flexibility of users in making annotations, this application provides a technical solution in which the electronic device can enter the annotation mode in a multi-window scenario for users to make annotations.

[0089] The aforementioned electronic devices may be display devices such as mobile phones, tablets, laptops, and e-readers. This application does not impose any particular limitation on these devices. For ease of explanation, this application will use a tablet as an example for illustrative purposes.

[0090] The multi-window scenario can include two or more windows. The multiple windows in the multi-window scenario can be arranged horizontally (e.g., left-right split-screen scenario), vertically (e.g., top-bottom split-screen scenario), or overlapped (e.g., floating window scenario). The technical solution of this application will be described below using left-right split-screen scenario, top-bottom split-screen scenario and floating window scenario as examples.

[0091] First, let's introduce the split-screen scenario.

[0092] Figure 2 This is a flowchart illustrating the annotation method provided in the embodiments of this application, as shown below. Figure 2 As shown, the annotation method provided in this embodiment may include the following steps:

[0093] S110, Display a split-screen interface, which includes a horizontally arranged first application interface and a second application interface.

[0094] For example, such as Figure 3 As shown in (a), in the split-screen interface displayed on the electronic device, the first application interface is the application interface of the "Gallery" APP, and the second application interface is the application interface of the "Notes" APP. The "Gallery" APP and the "Notes" APP are displayed side by side, with the first application interface located to the left of the second application interface. This positional relationship will also be used as an example for the following explanation.

[0095] It is understandable that the application interface of the same APP can be displayed in the left and right split screen. That is, the APP corresponding to the first application interface and the second application interface can be the same. The user can control the first application interface and the second application interface to display the same or different content. This display mode can also be called parallel view. In addition, the left and right split screen can also be used to divide the application interface of the same APP into columns. In this embodiment, the first application interface and the second application interface are application interfaces of different APPs for illustrative purposes.

[0096] S120. In response to the user's input annotation trigger operation, display the first annotation interface.

[0097] like Figure 3 As shown in (a) and (b), when the split-screen interface is displayed on the electronic device, the user can enter a comment to trigger the operation, which will cause the electronic device to enter the comment mode and display the comment interface (here referred to as the first comment interface).

[0098] As an optional implementation, annotation-triggered actions can be user actions performed on the screen of an electronic device.

[0099] Specifically, the annotation trigger operation can be a trigger operation on the target control in the floating menu.

[0100] like Figure 3 As shown in (a), after the electronic device establishes a connection with the stylus, it can display a floating menu 10 on the screen. The floating menu 10 can provide annotation controls (i.e. target controls), and the user can trigger the electronic device to enter annotation mode by clicking the control.

[0101] Figure 4 This demonstrates another way to implement a floating menu, such as... Figure 4 As shown in (a), the user can swipe inward from the first corner of the screen (exemplarily the upper right corner) to open the floating menu 10, as shown in (a). Figure 4 As shown in (b), the floating menu 10 can be in a semi-circular shape, displayed against the top and right edges of the screen; and Figure 3 Similarly, the floating menu 10 can provide annotation controls, which users can click to trigger the electronic device to enter annotation mode. That is, annotation triggering operations can include both displaying the floating menu on the electronic device and clicking the annotation control.

[0102] The operation that triggers the display of the floating menu on the electronic device is not limited to the above-mentioned swiping inward from the upper right corner of the screen. Other operations are also possible. For example, the user can also swipe outward on the screen to the edge area near the upper right corner of the screen to trigger the display of the floating menu on the electronic device.

[0103] When it is necessary to close the floating menu, users can trigger the electronic device to close the floating menu by clicking on an area outside the floating menu or by performing other actions.

[0104] like Figure 4 As shown in (b), the floating menu can also provide other functional controls, including, for example, the control corresponding to function A. When the floating menu includes many functional controls, some functional controls can be displayed, such as two functional controls, while others can be hidden. Users can slide along the direction of the floating menu, either upwards or downwards, to control the functional controls displayed in the floating menu.

[0105] User annotation triggers on the screen of an electronic device can also be a preset gesture operation. For example, a user can swipe inward from the second corner of the screen (such as the lower left corner) to trigger the electronic device to enter annotation mode; the user can also perform a target air gesture operation to trigger the electronic device to enter annotation mode. This embodiment does not particularly limit the gesture operation.

[0106] In this embodiment, the touch operation performed on the screen of the electronic device can be a touch operation performed by the user with their finger or a touch operation performed by the user with a stylus. In this embodiment, a stylus is used as an example for illustrative explanation.

[0107] As an alternative implementation, annotation-triggered operations can also be user actions performed on the stylus.

[0108] For example, users can double-click the stylus, shake the stylus, or press the target button on the stylus to trigger the electronic device to enter annotation mode.

[0109] like Figure 3 As shown in (b), after the electronic device enters annotation mode, the first annotation interface displayed may include the interface content 21 of the split-screen interface and the annotation layer covering the interface content 21. The annotation layer may include a transparent area 22 and a toolbar 23.

[0110] Users can view the interface content 21 below the annotation layer through the transparent area 22 and annotate the interface content 21 in the transparent area 22.

[0111] Toolbar 23 contains function options for adjusting annotation content. Figure 3 Example (b) shows several function options, from left to right: exit annotation mode, adjust line thickness, adjust line color, eraser, and save annotation.

[0112] Understandably, the function options in toolbar 23 are not limited to... Figure 3 The options shown may include more or fewer options, and this embodiment does not particularly limit this.

[0113] Toolbar 23 can be located at the bottom of the first batch of annotation interfaces by default, and users can also drag toolbar 23 to change its position.

[0114] S130: Receive and display the first batch of notes added by the user for the first batch of notes interface.

[0115] like Figure 3 As shown in (b), after the electronic device enters the annotation mode, the user can add annotations (i.e., the first batch of annotations) to the interface content 21 in the transparent area 22 using a stylus, and the electronic device can display the annotations added by the user in real time.

[0116] During the process of adding annotations, users can select the function options in toolbar 23 to adjust the thickness and color of the annotation lines or erase the annotation lines.

[0117] S140. In response to the user's scrolling input, the second annotation interface is displayed after the interface is scrolled.

[0118] S150: Receive and display the second annotation added by the user for the second annotation interface.

[0119] S160. In response to the annotation save operation, save the interface containing the first and second annotations.

[0120] During the annotation process, if the user wishes to continue browsing subsequent interface content in the first and second application interfaces, they can input a scrolling operation (i.e., a scrolling operation) on the screen. In this embodiment, after receiving the scrolling operation input by the user, the electronic device can scroll the interface and update the interface content displayed in the annotation interface; then, the user can continue to add annotations (i.e., second annotations) on the scrolled annotation interface (i.e., the second annotation interface); after completing the annotation, the annotation can be saved through the save annotation option in toolbar 23, or the electronic device can automatically save the application interface displayed during the annotation process and the annotations added by the user after each annotation operation is completed. In this embodiment, the example of the user saving annotations through toolbar 23 is used for illustrative explanation.

[0121] Specifically, scrolling operations can be swiping up or down on the screen, or swiping left or right on the screen. The scrolling annotation process corresponding to various scrolling operations is similar, with the main difference being the scrolling direction. The following example illustrates the swiping up operation.

[0122] As an optional implementation, after receiving a user's scrolling action, the electronic device can simultaneously scroll and display both the first and second application interfaces. Several possible scrolling annotation scenarios are illustrated below.

[0123] Scene 01: See Figure 5 In (a), the content of the first application interface is scrollable, such as email apps and weather apps. The content included in the application interface of these apps is usually not enough to be displayed on the screen. When the app responds to the user's scrolling operation, the content can be scrolled up and down in the application interface before the content reaches the bottom. The content of the second application interface is not scrollable, such as calculator apps and compass apps. The content included in the application interface of these apps is usually enough to be displayed on the screen, that is, the content has reached the bottom. When the app responds to the user's scrolling operation, the content does not scroll in the application interface.

[0124] like Figure 5As shown in (a), the interface content displayed in the first batch of annotation interfaces before scrolling includes: interface content 1-1 of the first application interface and interface content 2 of the second application interface; the annotation content includes: the first batch of annotations, wherein the first batch of annotations is located in the middle area of ​​the screen and spans across interface content 1-1 and interface content 2. The position of the first batch of annotations in the first batch of annotation interfaces in subsequent scenes 02 to 06 will also be explained using this example.

[0125] After the user inputs a scrolling operation, the electronic device can scroll upwards through the content of the first application interface and the content of the second application interface, while simultaneously scrolling the annotation layer. The content of the first application interface is scrollable, allowing new content to be displayed during scrolling; the content of the second application interface is not scrollable, meaning it has reached the bottom. In this case, preset content can be filled at the bottom of the content of the second application interface. For example, the filling content can be a white page or the background page of the second application interface; this embodiment uses a white page as an example for illustration.

[0126] like Figure 5 As shown in (b), the electronic device scrolls upwards through the interface content of the first application interface, the interface content of the second application interface, and the annotation layer. In the second annotation interface displayed after scrolling, the interface content of the first application interface, from top to bottom, may include: part of the interface content 1-1 of the first application interface displayed in the first annotation interface and the newly displayed interface content 1-2 of the first application interface; the interface content of the second application interface, from top to bottom, may include: part of the interface content 2 of the second application interface displayed in the first annotation interface and the filled content of the second application interface; the first annotation scrolls upwards synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0127] After scrolling through all the comments, users can continue to add second comments in the second comment interface, such as... Figure 5 As shown in (b), the second annotation is located on the newly displayed interface content in the second annotation interface. The second annotation spans the area where the interface content 1-2 of the first application interface is located and the area where the filled content of the second application interface is located. It can be understood that in the scrolled second annotation interface, the user can also add annotations on the interface content 1-1 of the first application interface and the interface content 2 of the second application interface.

[0128] After completing the annotation, the user can enter a save command to save the annotation interface. Upon receiving the user's save command, the electronic device can merge the content of the first application interface, the content of the second application interface, and the annotations in the annotation layer displayed throughout the annotation process, and then save the resulting annotation image.

[0129] like Figure 5As shown in (c), in the saved annotation image, the interface content of the first application interface from top to bottom may include: all interface content 1-1 of the first application interface displayed in the first batch of annotation interfaces and the interface content 1-2 of the first application interface newly displayed in the second annotation interface; the interface content of the second application interface from top to bottom may include: all interface content 2 of the second application interface displayed in the first batch of annotation interfaces and the fill content displayed in the second annotation interface; the annotation content from top to bottom may include: the first batch of annotations and the second batch of annotations added by the user.

[0130] When saving annotation images, electronic devices can save the annotation images to a default path, such as the photo album or memo. Users can also set the save path. For example, after the user selects the save annotation option in toolbar 23, the electronic device can display a pop-up window for the user to set the save path, and then save the annotation images to the path set by the user.

[0131] In addition, after generating annotated images, electronic devices can also display a preview interface. For example, similar to screenshots, the annotated image can be displayed in a pop-up window within the preview interface. If no user click on the annotated image is received within a preset time, the annotated image can be saved in a default path. If a user click on the annotated image is received within the preset time, an image editing interface can be displayed. The image editing interface can include the annotated image and some editing controls, such as delete, save, mosaic, and share controls. Users can delete the annotated image in the image editing interface, or make related edits and then save and / or share the image.

[0132] It is understandable that the content newly displayed after scrolling is related to the scrolling distance. Therefore, the content of the second annotation interface displayed after scrolling may be wholly or partially different from the content of the first annotation interface displayed before scrolling. This embodiment uses partial difference as an example for illustration. Furthermore, for ease of understanding, Figure 5 In (a) and (b), toolbar 23 is omitted, and the following figures are similar.

[0133] Scene 02: See Figure 6 In (a), the content of the first application interface and the second application interface can be scrolled. The content of the first batch of application interfaces before scrolling includes: the content of the first application interface 1-1 and the content of the second application interface 2-1.

[0134] After the user inputs a scrolling command, the electronic device can scroll upwards through the content of the first application interface, the content of the second application interface, and the annotation layer. Both the content of the first and second application interfaces can be scrolled, and new content can be displayed during the scrolling process.

[0135] like Figure 6 As shown in (b), in the second annotation interface displayed after scrolling, the interface content in the first application interface can include, from top to bottom: part of the interface content 1-1 of the first application interface displayed in the first annotation interface and the interface content 1-2 of the newly displayed first application interface; the interface content in the second application interface can include, from top to bottom: part of the interface content 2-1 of the second application interface displayed in the first annotation interface and the interface content 2-2 of the newly displayed second application interface; the first batch of annotations scrolls up synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0136] like Figure 6 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation spans across the interface content 1-2 of the first application interface and the interface content 2-2 of the second application interface. Figure 6 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image includes, from top to bottom, the interface content of the first application interface, which may include: all interface content 1-1 of the first application interface displayed in the first annotation interface and the interface content 1-2 of the newly displayed first application interface in the second annotation interface; the interface content of the second application interface, from top to bottom, may include: all interface content 2-1 of the second application interface displayed in the first annotation interface and the interface content 2-2 of the newly displayed second application interface in the second annotation interface; the annotation content, from top to bottom, may include: the first batch of annotations and the second batch of annotations added by the user.

[0137] Some apps include one or more fixed-position components in their interface. These components do not move when the app scrolls. Some components have scrollable content, while others do not. For example, the main text in gallery, contacts, and memo apps is scrollable, while the content in the navigation bar and menu bar is not. Some apps also include toolbars, whose content is usually non-scrollable. Each component can include one or more controls. For scrollable components, the controls and other content within them can move when the app scrolls. For non-scrollable components, the controls and other content within them do not move when the app scrolls.

[0138] For any APP displayed in the first batch of annotation interfaces, if the APP includes at least one scrollable component and at least one non-scrollable component with fixed positions, for each non-scrollable component, if the non-scrollable component is located to the left or right of the scrollable component, the interface can be filled at the bottom of the interface content of the non-scrollable component when scrolling; if the non-scrollable component is located at the bottom or top of the scrollable component, the interface content of the non-scrollable component can be fixedly displayed when scrolling, or the interface content of the non-scrollable component can be hidden. When saving, the non-scrollable component can be spliced ​​to the top or top of the interface content of the scrollable component, or the interface content of the non-scrollable component can be not saved. This embodiment will be illustrated by hiding and saving as an example.

[0139] If the app contains multiple non-scrollable components, then the interface can be filled at the bottom of the app's content.

[0140] The following section will continue to explain various possible scrolling annotation scenarios when the application interface includes multiple components in fixed positions.

[0141] Scene 03: See Figure 7 In (a), the application interface of the first application includes horizontally arranged components 1 and 3, with component 3 located to the left of component 1; the application interface of the second application includes component 2. The interface content of components 1 and 2 can be scrolled, while the interface content of component 3 cannot be scrolled. For example, component 1 is the component corresponding to the content area in the interface of the first application, and the main text content is displayed in component 1; component 3 is the side toolbar in the interface of the first application. When the first application responds to the scrolling operation, the interface content in component 1 can scroll accordingly, while the interface content in component 3 remains unchanged.

[0142] The interface content displayed in the first batch of annotations before scrolling includes: interface content 1-1 of component 1, interface content 2-1 of component 2, and interface content 3 of component 3; the annotation content includes: the first batch of annotations, wherein the first batch of annotations is located in the middle area of ​​the screen and spans across interface content 1-1 and interface content 2-1.

[0143] After the user inputs a scrolling action, the electronic device can scroll upwards through the interface content of component 1, the interface content of component 2, the interface content of component 3, and the annotation layer. The interface content of components 1 and 2 is scrollable, and new content is displayed during scrolling; the interface content of component 3 is not scrollable but can be filled with content.

[0144] like Figure 7As shown in (b), in the second annotation interface displayed after scrolling, the interface content of the first application interface can include two parts, left and right. The right part, from top to bottom, can include: part of the interface content 1-1 of component 1 displayed in the first annotation interface and the interface content 1-2 of the newly displayed component 1; the left part, from top to bottom, can include: part of the interface content 3 of component 3 displayed in the first annotation interface and the fill content of component 3. The interface content of the second application interface, from top to bottom, can include: part of the interface content 2-1 of the second application displayed in the first annotation interface and the interface content 2-2 of the newly displayed second application; the first annotation scrolls upwards synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0145] like Figure 7 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation spans across the interface content 1-2 of the first application interface and the interface content 2-2 of the second application interface. Figure 7 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image includes two parts: the interface content of the first application interface, which can be divided into left and right parts. The right part, from top to bottom, includes: all interface content 1-1 of the first application displayed in the first annotation interface and the newly displayed interface content 1-2 of the first application in the second annotation interface. The left part, from top to bottom, includes: all interface content 3 of component 3 displayed in the first annotation interface and the fill content of component 3 displayed in the second annotation interface. The interface content of the second application interface, from top to bottom, includes: all interface content 2-1 of the second application displayed in the first annotation interface and the newly displayed interface content 2-2 of the second application in the second annotation interface. The annotation content, from top to bottom, includes: the first and second annotations added by the user.

[0146] Scene 04: See Figure 8 In (a), the application interface of the first application includes vertically arranged components 1 and 3, with component 3 located at the bottom of component 1. For example, component 3 is the menu bar in the first application interface. The application interface of the second application includes component 2. The interface content of component 1 is scrollable, while the interface content of components 2 and 3 is not scrollable. The interface content displayed in the first batch of the interface before scrolling includes: interface content 1-1 of component 1, interface content 2 of component 2, and interface content 3 of component 3.

[0147] After the user inputs a scrolling action, the electronic device can scroll up the interface content of component 1, the interface content of component 2, and the annotation layer, while hiding and showing the interface content of component 3. The interface content of component 1 is scrollable, and new interface content can be displayed during scrolling; the interface content of component 2 is not scrollable but can be filled.

[0148] like Figure 8 As shown in (b), in the second annotation interface displayed after scrolling, the interface content in the first application interface from top to bottom may include: part of the interface content 1-1 of component 1 displayed in the first annotation interface and the interface content 1-2 of the newly displayed component 1; the interface content in the second application interface from top to bottom may include: part of the interface content 2 of the second application displayed in the first annotation interface and the fill content of the second application; the first annotation scrolls up synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0149] like Figure 8 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation spans across the interface content 1-2 of the first application interface and the filled content of the second application interface. For example... Figure 8 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image includes, from top to bottom, the interface content of the first application interface, which may include: all interface content 1-1 of the first application displayed in the first annotation interface, the newly displayed interface content 1-2 of the first application in the second annotation interface, and all interface content 3 of component 3 displayed in the first annotation interface; the interface content of the second application interface, from top to bottom, may include: all interface content 2 of the second application displayed in the first annotation interface and the fill content displayed in the second annotation interface; the annotation content, from top to bottom, may include the first and second annotations added by the user.

[0150] Scene 05: See Figure 9 In (a), the application interface of the first application includes vertically arranged components 1 and 3, with component 3 located at the bottom of component 1. The application interface of the second application includes component 2. The interface content in components 1 and 2 is scrollable, while the interface content in component 3 is not scrollable. The interface content displayed in the first batch of the application interface before scrolling includes: interface content 1-1 of component 1, interface content 2-1 of component 2, and interface content 3 of component 3.

[0151] After the user inputs a scrolling action, the electronic device can scroll up the interface content of component 1, the interface content of component 2, and the annotation layer, while hiding and showing the interface content of component 3. The interface content of components 1 and 2 can both be scrolled, and new interface content can be displayed during the scrolling process.

[0152] like Figure 9As shown in (b), in the second annotation interface displayed after scrolling, the interface content in the first application interface from top to bottom may include: part of the interface content 1-1 of component 1 displayed in the first annotation interface and the interface content 1-2 of the newly displayed component 1; the interface content in the second application interface from top to bottom may include: part of the interface content 2-1 of the second application displayed in the first annotation interface and the interface content 2-2 of the newly displayed second application; the first annotation scrolls up synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0153] like Figure 9 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation spans across the interface content 1-2 of the first application interface and the interface content 2-2 of the second application interface. Figure 9 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image includes, from top to bottom, the interface content of the first application interface, which may include: all interface content 1-1 of the first application displayed in the first annotation interface, the newly displayed interface content 1-2 of the first application in the second annotation interface, and all interface content 3 of component 3 displayed in the first annotation interface; the interface content of the second application interface, from top to bottom, may include: all interface content 2-1 of the second application displayed in the first annotation interface, the newly displayed interface content 2-2 of the second application in the second annotation interface, and the newly displayed interface content 2-3 of the second application corresponding to component 3; the annotation content, from top to bottom, may include the first and second annotations added by the user.

[0154] That is, when saving, if the interface content of component 3 is appended to the bottom of the interface content 1-2 of component 1, then a portion of the interface content can continue to be displayed after the interface content 2-2 of component 2, making the length of the first and second application interfaces in the saved annotation image the same. It is understandable that the interface can also be filled at the bottom of the interface content 2-2 of component 2.

[0155] Scene 06: See Figure 10 In (a), the application interface of the first application includes vertically arranged components 1 and 3, with component 3 located at the bottom of component 1. The application interface of the second application includes vertically arranged components 2 and 4, with component 4 located at the bottom of component 2. The interface content in components 1 and 2 is scrollable, while the interface content in components 3 and 4 is not scrollable. The interface content displayed in the first batch of the application interface before scrolling includes: interface content 1-1 of component 1, interface content 2-1 of component 2, interface content 3 of component 3, and interface content 4 of component 4.

[0156] After the user inputs a scrolling action, the electronic device can scroll upwards through the interface content of component 1, the interface content of component 2, and the annotation layer, while hiding and showing the interface content of components 3 and 4. The interface content of components 1 and 2 can both be scrolled, and new interface content can be displayed during the scrolling process.

[0157] like Figure 10 As shown in (b), in the second annotation interface displayed after scrolling, the interface content in the first application interface from top to bottom may include: part of the interface content 1-1 of component 1 displayed in the first annotation interface and the interface content 1-2 of the newly displayed component 1; the interface content in the second application interface from top to bottom may include: part of the interface content 2-1 of the second application displayed in the first annotation interface and the interface content 2-2 of the newly displayed second application; the first annotation scrolls up synchronously, moving from the middle area of ​​the screen to the upper area of ​​the screen.

[0158] like Figure 10 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation spans across the interface content 1-2 of the first application interface and the interface content 2-2 of the second application interface. Figure 10 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image includes, from top to bottom, the interface content of the first application interface, which may include: all interface content 1-1 of the first application displayed in the first annotation interface, the newly displayed interface content 1-2 of the first application in the second annotation interface, and all interface content 3 of component 3 displayed in the first annotation interface; the interface content of the second application interface, from top to bottom, may include: all interface content 2-1 of the second application displayed in the first annotation interface, the newly displayed interface content 2-2 of the second application in the second annotation interface, and all interface content 4 of component 4 displayed in the first annotation interface; the annotation content, from top to bottom, may include: the first and second annotations added by the user.

[0159] The above Figure 10 The example used here is that components 3 and 4 have the same height. It's understandable that components 3 and 4 can have different heights. In this case, to make the saved annotation image more aesthetically pleasing, a new portion of the interface content can be displayed above the component with the shorter height, ensuring that the lengths of the first and second application interfaces in the saved annotation image are consistent. For example, if the height of component 3 is lower than the height of component 4, a new portion of the interface content can be displayed after the first two sections of the interface content in component 1.

[0160] As an alternative implementation, after receiving a user's scrolling command, the electronic device can only scroll the application interface targeted by that command, while other application interfaces do not need to scroll; that is, the application interfaces scroll asynchronously. During saving, the interface can be filled to ensure that the length of each application interface is consistent. When the user adds annotations, cross-screen annotations can be disabled. The following example, Scenario 02, provides an illustrative illustration.

[0161] See Figure 11 In (a), the interface content of the first application and the second application can be scrolled. The interface content displayed in the first batch of annotations before scrolling includes: the interface content 1-1 of the first application and the interface content 2 of the second application; the first batch of annotations includes: annotation 11 and annotation 12, with annotation 11 located in the middle area of ​​interface content 1-1 and annotation 12 located in the middle area of ​​interface content 2.

[0162] When a user inputs a scrolling command in the first application interface (indicated by a thick, upward-pointing black arrow in the diagram), the electronic device can scroll upwards through the content of the first application interface, such as... Figure 11 As shown in (b), in the second annotation interface displayed after scrolling, the interface content in the first application interface can include, from top to bottom: part of the interface content 1-1 of the first application displayed in the first annotation interface and the newly displayed interface content 1-2 of the first application; the interface content in the second application interface remains unchanged.

[0163] like Figure 11 As shown in (b), the user can add a second annotation in the second annotation interface and then save the annotation interface. The second annotation includes annotation 21 and annotation 22. Annotation 21 is located on interface content 1-2, and annotation 22 is located on interface content 2, below annotation 12. Figure 11 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image may include, from top to bottom, the interface content of the first application interface: all interface content 1-1 of the first application displayed in the first annotation interface and the newly displayed interface content 1-2 of the first application in the second annotation interface; the interface content of the second application interface may include, from top to bottom, all interface content 2 of the second application displayed in the first annotation interface and the filled content of the second application; the annotation content may include: annotations 11 and 21 located in the first application interface, and annotations 12 and 22 located in the second application interface.

[0164] The above example of scenario 02 illustrates the asynchronous scrolling process. The asynchronous scrolling process in other scenarios is similar and will not be repeated here.

[0165] The following section introduces the split-screen scenario.

[0166] Similar to the left-right split screen, the top-bottom split screen can display the application interfaces of different apps or the application interface of the same app. This embodiment does not make any special limitations on this. The following example of displaying the first application interface on the top screen and the second application interface on the bottom screen illustrates the scrolling annotation process.

[0167] When an electronic device receives a scrolling operation from a user, the annotation interface may or may not contain user-added annotations. In addition, user-added annotations may span across the screen, that is, the annotations may span both the first and second application interfaces, or they may not span the screen, that is, the annotations may be located in either the first or second application interface. These situations will be explained below.

[0168] Scene 11: See Figure 12 In case (a), the annotation interface shows no annotations when the electronic device receives a scrolling action from the user. In this situation, the electronic device can display the app targeted by the user's scrolling action in full-screen mode, for example... Figure 12 As shown in (a) and (b), after the user inputs a scrolling operation in the first application interface, the electronic device displays the first application interface in full screen.

[0169] Similar to the current full-screen annotation process, users can add annotations to the first application interface on the screen, scroll through the interface, etc. After the user enters a save annotation, the electronic device can merge the content of the first application interface displayed during the annotation process with the user-added annotations into an annotation image for saving. For example, such as Figure 12 As shown in (b), after the user adds a comment on the first application interface, the comment interface is saved, wherein the user-added comment is located in the central area of ​​the first application interface; as shown in (b). Figure 12 As shown in (c), after the electronic device receives the user's annotation save operation, the saved annotation image may include: the interface content of the first application interface displayed in full screen and the annotation added by the user.

[0170] Scene 12: See Figure 13 In (a) of the example, when the electronic device receives a scrolling operation from the user, the annotation interface includes annotation 11 and annotation 12. Annotation 11 is located in the first application interface, and annotation 12 is located in the second application interface. Neither annotation 11 nor annotation 12 spans the entire screen. In this case, the electronic device can display the app targeted by the user's scrolling operation in full screen, for example... Figure 13 As shown in (a) and (b), after the user inputs a scrolling operation in the first application interface, the electronic device displays the first application interface in full screen.

[0171] Then, the user can add annotations and scroll through the interface on the first application screen. After the user saves the annotation, the electronic device can merge the first application screen displayed during the annotation process and the annotations added by the user on the first application screen into a first annotation image, and merge the second application screen displayed during the annotation process and the annotations added by the user on the second application screen into a second annotation image. The second annotation image is then stitched to the bottom of the first annotation image and saved. For example, such as... Figure 13 As shown in (b), after the user adds annotation 2 to the first application interface, the annotation interface is saved, where annotation 2 is located below annotation 11; as Figure 13 As shown in (c), after the electronic device receives the user's annotation save operation, the interface content in the saved annotation image can include, from top to bottom: the interface content of the first application interface displayed in full screen and the interface content of the second application interface displayed before scrolling. The annotation content can include: annotation 11 and annotation 2 located on the first application interface and annotation 12 located on the second application interface.

[0172] Scene 13: See Figure 14 In (a) of the example, when the electronic device receives a scrolling operation from the user on the first application interface, the annotation interface includes annotation 1, which spans the entire screen. In this case, as follows... Figure 14 As shown in (b), the electronic device can display a pop-up window prompting the user whether to save the current annotations. After the user selects this option, as shown in (b), the electronic device can display a pop-up window prompting the user whether to save the current annotations. Figure 14 As shown in (c), the first application interface is displayed in full screen.

[0173] Similar to the above scenario, users can add annotations and scroll through the interface on the screen. After the user enters a comment and saves it, the electronic device can merge the first application interface displayed during the annotation process and the annotations added by the user on the first application interface into a single annotation image for saving. Additionally, if the user... Figure 14 If the current annotation content is selected in the interface shown in (b), the electronic device can simultaneously save the current annotation content. Figure 14 The application interface shown in (a) and annotation 1 are combined into an annotation image and saved. For example, as shown in [example image]... Figure 14 As shown in (b) and (c), after the user selects the "Yes" option and adds annotation 2 to the first application interface, the annotation interface is saved, with annotation 2 located below annotation 1. Figure 14 As shown in (d) and (e), after the electronic device receives the user's annotation save operation, one of the saved annotation images may include: a first application interface displayed in full screen, annotation 1 and annotation 2; the other saved annotation image may include: the first application interface and the second application interface displayed from top to bottom before scrolling, and annotation 1 spanning the first application interface and the second application interface.

[0174] Understandable, Figure 14 In (c) of the document, when the electronic device displays the first application interface in full screen, annotation 1 may not be displayed.

[0175] Scene 14: See Figure 15 In (a) of the example, when the electronic device receives a scrolling operation from the user on the second application interface, the annotation interface includes annotation 1, which spans the entire screen. In this case, as... Figure 15 As shown in (b), the electronic device can scroll up the first application interface and the second application interface. During the scrolling process, Figure 15 In (a) shown in the figure, the content of the first application interface is gradually hidden from top to bottom, while the second application interface continuously displays new content.

[0176] like Figure 15 As shown in (b), after the user adds annotation 2 to the interface and scrolls the interface again, annotation 2 is located in the second application interface and below annotation 1. After the user scrolls the interface, as shown... Figure 15 As shown in (c) in the figure, Figure 15 In (a) above, the content of the first application interface is completely hidden, while the second application interface is displayed in full screen. Annotation 1 and Annotation 2 scroll up synchronously, with Annotation 1 scrolling to the top of the screen and its portion within the first application interface being hidden, and Annotation 2 scrolling to the upper area of ​​the screen. After the user adds Annotation 3, the annotation interface is saved, with Annotation 3 located below Annotation 2. After receiving the user's annotation save operation, the electronic device can merge the application interface displayed during the annotation process and the user-added annotations into an annotation image for saving; such as Figure 15 As shown in (d), the interface content in the saved annotation image can include, from top to bottom: the interface content of the first application interface and the second application interface displayed before scrolling, and the interface content of the second application interface newly displayed after two scrolls. The annotation content can include, from top to bottom: annotation 1 spanning the first application interface and the second application interface, and annotations 2 and 3 located in the second application interface.

[0177] The following section introduces the floating window scenario.

[0178] like Figure 16 As shown in (a), the electronic device displays the first application interface in full screen and the second application interface in a floating window. In this scenario, the user clicks the annotation control in the floating menu 10; as Figure 16 As shown in (b), after the electronic device receives the user's annotation trigger operation, it enters annotation mode and can hide the floating window.

[0179] Users can add annotations and scroll through the interface on the screen. After the user enters a comment and saves it, the electronic device can merge the first application interface displayed during the annotation process and the user's added comments into a single annotation image for saving. For example, such as... Figure 16 As shown in (b), after the user adds annotation 1 on the first application interface, the interface scrolls, with annotation 1 located in the center of the screen; as... Figure 16 As shown in (c), after receiving the user's scrolling operation, the electronic device scrolls the content of the first application interface, and the new content of the first application interface is displayed at the bottom of the screen. Simultaneously, annotation 1 scrolls up to the top of the screen. The user continues to add annotation 2 to the first application interface and then saves the annotation interface, where annotation 2 is located on the newly displayed content. Figure 16 As shown in (d), after the electronic device receives the user's annotation save operation, the interface content in the saved annotation image can include, from top to bottom, the interface content of the first application interface displayed before scrolling and the interface content of the first application interface newly displayed after scrolling, and the annotation content can include, from top to bottom, annotation 1 and annotation 2.

[0180] As another alternative implementation method, such as Figure 17 As shown in (a) and (b), the electronic device displays the first application interface in full screen and the second application interface in a floating window. In this scenario, the user triggers the operation by clicking the annotation control in the floating menu 10 to input annotations. After the electronic device enters annotation mode, it can display the first and second application interfaces in a split-screen manner. This example uses a left-right split-screen, but it can be understood that a top-bottom split-screen is also possible.

[0181] The scrolling annotation process in a split-screen scenario is similar to that described above. The following example, Scenario 01, will be used as an illustration.

[0182] like Figure 17 As shown in (b), the content 1-1 of the first application's interface is scrollable, while the content 2 of the second application's interface is not scrollable. The user adds a comment 1 and then inputs a scrolling operation. The comment 1 is located in the center of the screen and spans both the first and second application interfaces. Figure 17 As shown in (c), after receiving a scrolling operation, the electronic device scrolls upwards the content of the first application interface, the content of the second application interface, and the annotation layer, filling the interface at the bottom of the content of the second application interface. In the scrolled interface, the content of the first application interface, from top to bottom, may include: Figure 17The partial interface content 1-1 of the first application shown in (b) and the newly displayed interface content 1-2 of the first application, the interface content in the second application interface from top to bottom may include: the partial interface content 2 of the second application and the filling content of the second application, and the annotation 1 scrolls up synchronously from the middle of the screen to the top of the screen.

[0183] After the user adds comment 2 and enters the save operation, the comment interface is saved. Comment 2 spans the area containing content 1-2 in the first application interface and the area containing the filled content in the second application interface. For example... Figure 17 As shown in (d) above, the content of the first application interface in the saved annotation image, from top to bottom, can include: Figure 17 The complete interface content shown in (b) 1-1 and Figure 17 In (c), the newly displayed interface content 1-2 of the first application, the interface content of the second application from top to bottom may include: all interface content 2 of the second application and the filled content of the second application, and the annotation content from top to bottom may include: annotation 1 and annotation 2 added by the user.

[0184] The annotation method provided in this embodiment allows electronic devices to enter annotation mode in multi-window scenarios, enabling users to make annotations. This enriches the application scenarios of the annotation function, improves the flexibility of user annotation, and enhances the user's annotation experience.

[0185] Based on the same concept, as an implementation of the above method, this application provides an interactive system, please refer to... Figure 18 , Figure 18 This is a schematic diagram of the structure of the interactive system provided in an embodiment of this application. Figure 18 As shown, the interactive system may include: electronic device 100 and stylus 200.

[0186] Electronic device 100 can be a display device such as a mobile phone, tablet, laptop, or e-reader. Stylus 200 can be a resistive stylus 200 or a capacitive stylus 200, etc.

[0187] The electronic device 100 and the stylus 200 can communicate via wireless communication methods such as Bluetooth and Wi-Fi networks.

[0188] The stylus 200 can be equipped with one or more buttons, which can be physical or virtual. Users can control the stylus 200 to perform corresponding operations using the buttons. For example, after the user clicks the first button, the stylus 200 establishes a communication connection with the electronic device 100; after the user clicks the second button, the electronic device 100 enters annotation mode.

[0189] After the electronic device 100 establishes communication with the stylus 200, the user can input touch operations through the stylus 200. In annotation mode, the electronic device 100 can display the motion trajectory formed by the stylus 200 on the screen.

[0190] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to illustrate the software structure of electronic device 100.

[0191] Figure 19 As shown in Figure 19, the software architecture of the electronic device 100 provided in this application embodiment can be divided into several layers. In some embodiments, the Android system can be divided into the application layer, application framework layer, Android runtime and system library, and kernel layer from top to bottom.

[0192] The application layer can include a series of applications. For example... Figure 19 As shown, applications can include camera, gallery, calendar, call, WLAN, Bluetooth, music, memo, SMS, annotation, browser, and other applications.

[0193] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0194] like Figure 19 As shown, the application framework layer may include a window manager, content provider, view system, resource manager, notification manager, screenshot module, etc.

[0195] The window manager manages windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots. The content provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon can include views for displaying text and views for displaying images. The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, video files, etc. The notification manager allows applications to display notifications in the status bar. These notifications can be used to convey informational messages and can disappear automatically after a short pause without user interaction. For example, the notification manager can be used to notify of download completion or message alerts. The notification manager can also display notifications as icons or scrollbar text in the system's top status bar, such as notifications from background applications, or as dialog boxes on the screen. For example, displaying text messages in the status bar, emitting notification sounds, causing electronic devices to vibrate, or flashing indicator lights.

[0196] The screenshot module can take screenshots of the display interface of the electronic device 100.

[0197] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0198] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0199] System libraries can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and input modules.

[0200] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. The Media Library supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0201] The input module is used to pass the touch events generated by the user's touch input to the corresponding application.

[0202] The kernel layer is the layer between hardware and software, providing core system services to the Android kernel, such as security services, network services, memory management services, detection management services, and driver models. The kernel layer can contain display drivers, camera drivers, audio drivers, and sensor drivers.

[0203] After a user inputs a touch operation, the touch sensor in the hardware device can generate a corresponding hardware interrupt and send it to the kernel layer. The kernel layer can encapsulate the touch operation into touch events (such as touch down events, touch move events, and touch up events), and can report the touch events to the upper-layer APP through the input module in the system library.

[0204] As described in the previous embodiments, after the electronic device 100 enters annotation mode, it can load an annotation layer on the application interface. This way, touch events generated by the user's scrolling input will be processed by the annotation layer, and the APP located below the annotation layer will not receive the touch events, thus preventing interface scrolling. In this embodiment, interface scrolling in annotation mode can be achieved by simulating touch events. To implement the scrolling annotation function, a content senser can be added to the application framework layer. The annotation application can use the content senser to implement the scrolling annotation process. The principle of scrolling annotation is explained below.

[0205] Figure 20 This is a schematic diagram illustrating the principle of scrolling annotation provided in an embodiment of this application, such as... Figure 20 As shown, after the user inputs an annotation to trigger an operation, the input module can pass the touch event generated by the annotation trigger operation to the annotation application (step S1); after receiving the touch event, the annotation application can respond to the touch event and send an interface information request command to the window manager (step S2) to obtain the APP interface information.

[0206] After receiving the interface information request command, the window manager can pass the interface information of the currently displayed APP (referred to as the target APP) to the content perceiver (step S3); then, the content perceiver can pass the interface information of the target APP to the annotation application (step S4).

[0207] The target app can include one or more apps, for example, the target app can include the ones mentioned above. Figure 3 The processing procedures for the "Gallery" and "Notes" apps shown are similar for each target app; this example illustrates the process using one of them.

[0208] After obtaining the interface information of the target APP, the annotation application can determine the position of the content display control in the target APP interface (referred to as the first target position) based on the interface layout information. Then, it can send a screenshot command to the screenshot module (step S5) to notify the screenshot module to take a screenshot of the first target position.

[0209] After the screenshot module completes the screenshot, it can return the screenshot to the annotation application (step S6); then, the annotation application can load the annotation layer on the screenshot and display it (step S7).

[0210] If the user adds the first batch of annotations on the annotation layer, the input module can pass the touch event corresponding to the first batch of annotations to the annotation application (step S8); the annotation application can respond to the touch event and display the first batch of annotations added by the user on the annotation layer (step S9).

[0211] In addition, after the annotation application obtains the screenshot returned by the screenshot module, it can simulate the touch event corresponding to the scrolling operation (referred to as scroll touch event), and then send the simulated scroll touch event to the window manager (step S10), notifying the window manager to pass the simulated scroll touch event to the target APP.

[0212] After the window manager passes the simulated scrolling touch event to the target APP (step S11), the target APP can then execute the corresponding scrolling process (step S12).

[0213] Specifically, if the interface content is scrollable, the target app can scroll the interface content; if the interface content is not scrollable, that is, the interface content has reached the bottom, the target app can choose not to scroll the interface content.

[0214] After sending the simulated scrolling touch event to the window manager, the annotation application can determine the position of the scrollable control (referred to as the second target position) based on the previously obtained interface information of the target APP, and then send a screenshot command to the screenshot module (step S13) to notify the screenshot module to take a screenshot of the second target position.

[0215] Specifically, the annotation application can determine whether the interface content in a control is scrollable based on the control type. Scrollable controls can include RecycleView, ListView, ScrollView, PageView, and WebView. For example, the main page of a "Gallery" app generally uses a RecycleView control, news apps generally use a ListView control, "Contacts" apps generally use a ScrollView control, slideshow apps generally use a PageView control, and web-based apps generally use a WebView control.

[0216] Once the annotation application identifies the scrollable control, it can set the position of the scrollable control as the second target position and instruct the screenshot module to take a screenshot of that position.

[0217] After the screenshot module completes the screenshot, it can return the screenshot to the annotation application (step S14); then, the annotation application can perform image deduplication and image stitching, generate a long screenshot, and update the displayed screenshot (step S15).

[0218] Specifically, the annotation application can compare the screenshot captured this time with the screenshot captured last time. If the screenshot captured last time already contains all the interface content of the screenshot captured this time, it means that the target app's interface content has scrolled to the bottom. In this case, the bottom of the target app's interface content can be filled, for example, by filling it with a white page. If the screenshot captured last time does not include all the interface content of the screenshot captured this time, the newly loaded interface content of the screenshot captured this time can be stitched to the bottom of the interface content of the screenshot captured last time to generate a long screenshot.

[0219] If the user performs a scrolling operation on the annotation layer, the input module can pass the touch event corresponding to the scrolling operation to the annotation application (step S16); the annotation application can respond to the touch event, scroll the long screenshot and annotation layer, and update the displayed interface content (step S17).

[0220] In addition, after the annotation application completes step S15, if it determines that the content of the target APP's interface has not reached the bottom, that is, the content of the target APP's interface can continue to scroll, the annotation application can return to step S10 to update the long screenshot. That is, the electronic device 100 can repeat steps S10-S15 until the preset conditions are met.

[0221] The preset conditions could be that the target APP's interface content reaches the bottom, or that the electronic device exits the annotation mode.

[0222] When repeatedly executing steps S10-S15, if the preset conditions are not met, the next long screenshot update process can be performed immediately after each long screenshot update process is completed; or the next long screenshot update process can be performed after waiting for a preset time after each long screenshot update process is completed. The preset time between each two long screenshot update processes can be equal or unequal. For example, the preset time can be gradually increased.

[0223] Understandably, when implementing scrolling annotation functionality, the window manager can be modified to allow the annotation application to directly obtain app interface information from the window manager. Adding a content-aware interface to the application framework layer can reduce modifications to the existing framework structure and lower the implementation difficulty.

[0224] The structure of the electronic device 100 is described below.

[0225] See Figure 21 , Figure 21 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0226] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0227] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0228] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0229] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0230] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0231] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0232] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. The UART interface is a universal serial data bus used for asynchronous communication; this bus can be a bidirectional communication bus, converting the data to be transmitted between serial and parallel communication. The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and camera 193; MIPI interfaces include camera serial interface (CSI) and display serial interface (DSI). The GPIO interface can be configured via software; it can be configured as a control signal or a data signal. The USB interface 130 is a USB standard compliant interface, specifically a Mini USB interface, MicroUSB interface, or USB Type-C interface. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transfer between the electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0233] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0234] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0235] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0236] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0237] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0238] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0239] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0240] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0241] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GNSS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0242] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0243] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a Micro LED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0244] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0245] The ISP is used to process data fed back from camera 193. Camera 193 is used to capture still images or video. The digital signal processor is used to process digital signals; in addition to digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital video.

[0246] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0247] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0248] The external storage interface 120 can be used to connect external storage devices, such as Micro SD cards, to expand the storage capacity of the electronic device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0249] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0250] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, audio module 170 can be located in processor 110, or some functional modules of audio module 170 can be located in processor 110. Speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Receiver 170B, also called a "handpiece," is used to convert audio electrical signals into sound signals. Microphone 170C, also called a "microphone" or "microphone," is used to convert sound signals into electrical signals. Headphone jack 170D is used to connect wired headphones. Headphone jack 170D can be a USB interface 130, or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0251] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with electronic device 100. Electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0252] The interactive system provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0253] Based on the same concept, as an implementation of the above method, this application provides an annotation device. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0254] Figure 22 This is a schematic diagram of the annotation device provided in the embodiments of this application, as shown below. Figure 22 As shown, the apparatus provided in this embodiment includes:

[0255] Display module 210, input module 220, processing module 230 and communication module 240.

[0256] The display module 210 is used to support the electronic device in performing the interface display operations in the above embodiments and / or other processes used in the technology described herein. The display module may be a touch screen or other hardware or a combination of hardware and software.

[0257] The input module 220 is used to receive user input on the display interface of the electronic device, such as touch input, voice input, gesture input, etc. The input module is used to support the electronic device in performing the steps of receiving user call answering operations in the above embodiments and / or other processes used in the technology described herein. The input module may be a touch screen or other hardware or a combination of hardware and software.

[0258] The processing module 230 is used to support the electronic device in performing processing operations related to the annotation process in the above embodiments and / or other processes used in the techniques described herein.

[0259] The communication module 240 is used to support the electronic device in performing operations related to the communication process with the stylus in the above embodiments and / or other processes used in the technology described herein.

[0260] The apparatus provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0261] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0262] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in the above-described method embodiments.

[0263] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to implement the method described in the above-described method embodiments.

[0264] This application also provides a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the method described in the above-described method embodiments. The chip system may be a single chip or a chip module composed of multiple chips.

[0265] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, or magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0266] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium can include various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0267] The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0268] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0269] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0270] It should be understood that in the description of this application and the appended claims, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0271] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.

[0272] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0273] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0274] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0275] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0276] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An annotation method, characterized in that, include: The first interface is displayed, which includes a first application interface and a second application interface displayed in a split screen. Receive the first batch of annotation operations from the user on the first interface; In response to the first batch of annotations operation, the first batch of annotations is displayed on the first interface; Receive the user's first scrolling operation on the first interface; In response to the first scrolling operation, a second interface is displayed, which includes a first application interface and a second application interface displayed in a split screen, and the second interface is different from the first interface. Receive the user's second annotation operation on the second interface; In response to the second annotation operation, the second annotation is displayed on the second interface; Generate the first batch of annotation images, which include: the first batch of annotations, the second batch of annotations, the first interface, and the second interface.

2. The method according to claim 1, characterized in that, The step of displaying the second interface in response to the first scrolling operation specifically includes: In response to the first scrolling operation, the first application interface displayed in split screen is switched from the first sub-interface of the first application to the second sub-interface of the first application.

3. The method according to claim 2, characterized in that, The method further includes: In response to the first scrolling operation, the second application interface displayed in split screen switches from the third sub-interface of the second application to the fourth sub-interface of the second application.

4. The method according to claim 3, characterized in that, If the first scrolling operation is an upward scrolling operation, in the first batch of annotation images, the first sub-interface is located above the second sub-interface, and the third sub-interface is located above the fourth sub-interface.

5. The method according to claim 4, characterized in that, If the first application interface is located to the left of the second application interface, the first sub-interface in the first batch of annotation images is located to the left of the third sub-interface, and the second sub-interface is located to the left of the fourth sub-interface; If the first application interface is located to the right of the second application interface, then in the first batch of annotation images, the first sub-interface is located to the right of the third sub-interface, and the second sub-interface is located to the right of the fourth sub-interface.

6. The method according to claim 3, characterized in that, The fourth sub-interface is a populated page.

7. The method according to claim 3, characterized in that, If the first application interface further includes a first component that is fixedly displayed at the bottom of the first application interface, then the method further includes: After receiving the user's first scrolling operation on the first interface, the first component is no longer displayed.

8. The method according to claim 1, characterized in that, The method further includes: The third interface is displayed, which includes a third application interface and a fourth application interface displayed in a split screen. Receive the user's second scrolling operation on the third application interface displayed in the third interface; In response to the second scrolling operation, the third application interface is displayed in full screen; Receive user's third annotation operation on the third application interface; In response to the third annotation operation, the third annotation is displayed on the third application interface; A second annotation image is generated, which includes the third annotation and the third application interface.

9. The method according to claim 1, characterized in that, The method further includes: The fourth interface is displayed, which includes a fifth application interface and a sixth application interface displayed in split screen. Receive user's fourth annotation operation on the fifth application interface or the sixth application interface; In response to the fourth annotation operation, the fourth annotation is displayed on the fifth or sixth application interface; Receive the user's third scrolling operation on the fifth application interface; In response to the third scrolling operation, the fifth application interface is displayed in full screen.

10. The method according to claim 1, characterized in that, The method further includes: The fifth interface is displayed, which includes a seventh application interface and an eighth application interface displayed in split screen. Receive the user's fifth annotation operation on the fifth interface; In response to the fifth annotation operation, the fifth annotation is displayed, which overlaps with the boundary line between the seventh and eighth application interfaces; Receive the user's fourth scrolling operation on the fifth interface; In response to the fourth scrolling operation, a prompt interface is displayed for the user to choose whether to save the annotations.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: The sixth interface is displayed, which includes a ninth application interface that is displayed in full screen and a tenth application interface that is displayed floating on the ninth application interface. In response to an annotation-triggered operation, the floating tenth application interface is hidden or the ninth and tenth application interfaces are displayed in a split-screen mode.

12. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to execute the method as described in any one of claims 1-11 when the computer program is invoked.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

14. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-11.

15. A chip system, characterized in that, The chip system includes a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method as described in any one of claims 1-11.

Images