Displaying and storing method, device and equipment of comment information and storage medium

Abstract

The application discloses a display, storage method, device and equipment of annotation information and a storage medium, and belongs to the technical field of display. The method is applied to a first device, the first device is connected with a second device; the method comprises the following steps: receiving screen projection content sent by the second device; displaying the screen projection content, the screen projection content that has been displayed in history contains a target element, the target element is bound with annotation content; identifying the reproduction of the target element based on the screen projection content; in the case that the reproduction of the target element is identified, the annotation content bound with the target element is reproduced. By identifying the reproduction of the target element through the first device, the change of the screen projection content is monitored, compared with monitoring the change of the screen projection content through the second device, the resource processing amount of the second device is reduced. And compared with monitoring through the second device, the stability of monitoring the screen projection content is improved through the first device. In addition, the annotation content is reproduced when the target element is reproduced, and the use convenience is improved.

Description

Displaying and storing method, device and equipment of annotation information and storage medium TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of display technology, in particular to a displaying and storing method, device and equipment of annotation information and storage medium. BACKGROUND

[0002] In the field of display technology, the file content in the second device can be projected onto the display screen of the first device, so that the file content can be demonstrated through the first device. In addition, annotations may be needed for the file content during the demonstration to improve the quality of the demonstration through the annotations. Therefore, there is an urgent need for a method for displaying and storing the annotation information of the projected content.

[0003] SUMMARY

[0004] The present application provides a displaying and storing method, device and equipment of annotation information and storage medium to display and store the annotation information of the projected content. The technical solution is as follows.

[0005] In one aspect, a displaying method of annotation information is provided, which is applied to a first device connected with a second device. The method comprises the following steps.

[0006] Receiving the projected content sent by the second device;

[0007] Displaying the projected content, wherein the projected content has a target element which is historically displayed and has annotation content bound thereto;

[0008] Identifying the reproduction of the target element based on the projected content, wherein the target element has annotation content bound thereto;

[0009] Reproducing the annotation content bound to the target element when the reproduction of the target element is identified.

[0010] In one possible implementation, before the step of reproducing the annotation content bound to the target element, the method further comprises the step of determining the target element based on the projected content.

[0011] In one possible implementation, the step of determining the target element based on the projected content comprises the following steps.

[0012] Identifying at least one page element of the projected content to determine the target element;

[0013] Determining the target element from the at least one page element.

[0014] In a possible implementation, the determining the target element from the at least one page element comprises: determining the target element from the at least one page element based on a position of the annotation.

[0015] In a possible implementation, before the reproducing the annotation content bound by the target element, the method further comprises: storing the target element and the annotation content bound by the target element.

[0016] In a possible implementation, before the storing the target element and the annotation content bound by the target element, the method further comprises:

[0017] obtaining first position information of the target element;

[0018] determining the annotation content bound by the target element based on the first position information of the target element.

[0019] In a possible implementation, the determining the annotation content bound by the target element based on the first position information of the target element comprises:

[0020] obtaining second position information of each annotation content;

[0021] determining the annotation content bound by the target element according to the first position information and the second position information.

[0022] In a possible implementation, the original size of the target element is also stored, and before the reproducing the annotation content bound by the target element, the method further comprises:

[0023] determining a display size of the reproduced target element and the original size of the target element;

[0024] determining a scaling ratio based on the display size and the original size of the target element;

[0025] the reproducing the annotation content bound by the target element comprises:

[0026] displaying the reproduced annotation content bound by the target element according to the scaling ratio.

[0027] In a possible implementation, the relative position information between the target element and the annotation content bound by the target element is also stored, and before the reproducing the annotation content bound by the target element, the method further comprises:

[0028] determining an element position of the reproduced target element;

[0029] determining position information of the reproduced annotation content bound by the target element based on the element position of the target element and the relative position information;

[0030] the reproduced target element binding includes:

[0031] displaying the reproduced target element binding based on position information of the reproduced target element binding.

[0032] In a possible implementation, the reproducing the target element based on the screen projection content includes:

[0033] determining a similarity between an element in the screen projection content and the stored target element, the similarity including at least one of a content similarity or a position similarity;

[0034] in a case where the similarity is greater than a similarity threshold, determining that the element in the screen projection content is the reproduced target element.

[0035] In a possible implementation, the reproducing the target element binding includes:

[0036] obtaining display position information of the reproduced target element;

[0037] in a case where it is determined, according to the display position information, that a position other than the reproduced target element is a blank position, displaying the target element binding in the blank position.

[0038] In a possible implementation, a plurality of display windows are displayed in a screen displaying the screen projection content, and the screen projection content includes:

[0039] displaying the screen projection content through a target display window in the plurality of display windows.

[0040] In a possible implementation, the method further includes:

[0041] in response to a creation of a comment page operation, displaying a plurality of comment pages on the displayed screen projection content, any comment page in the plurality of comment pages being used to reproduce target element binding comment content on the any comment page in a case where the target element is reproduced.

[0042] In a possible implementation, the method further includes:

[0043] in response to a switching of a comment page operation, replacing comment content displayed on the screen projection content based on the switching of the comment page operation.

[0044] In a possible implementation, the first device and the second device are connected through a High Definition Multimedia Interface (HDMI) connection line.

[0045] In a possible implementation, the method further includes:

[0046] generating an annotation file based on the reproduced target element and the annotation content corresponding to the reproduced target element, the annotation file being used to share or display the reproduced target element and the annotation content corresponding to the reproduced target element.

[0047] In a possible implementation, the target element is an entire page of the screen projection content, and the method further includes:

[0048] comparing the screen projection content of the current page with stored screen projection content, the stored screen projection content having corresponding annotation content;

[0049] in a case where the screen projection content of the current page is consistent with the stored screen projection content, displaying the annotation content corresponding to the stored screen projection content.

[0050] In a possible implementation, the method further includes:

[0051] in response to a trigger operation of opening the annotation, comparing the screen projection content of the current page with stored target elements;

[0052] in a case where the screen projection content of the current page is inconsistent with the stored target elements, creating a screen projection file for the current page, the created screen projection file being used to store annotation content of the current page.

[0053] In a possible implementation, the screen projection content includes a plurality of page elements, the plurality of page elements including at least two target elements, and different target elements being bound to different annotation content.

[0054] In a possible implementation, before reproducing the annotation content bound to the target element in a case where the target element is identified to be reproduced, the method further includes:

[0055] binding the target element to a standard font corresponding to the annotation content bound to the target element and storing the target element and the standard font;

[0056] reproducing the annotation content bound to the target element in a case where the target element is identified to be reproduced, including:

[0057] in a case where the target element is identified to be reproduced, displaying the annotation content bound to the target element and a standard font corresponding to the annotation content.

[0058] In another aspect, a storage method of annotation information is provided. The method is applied to a first device connected with a second device. The method comprises:

[0059] receiving screen projection content sent by the second device;

[0060] determining a target element based on the screen projection content;

[0061] determining annotation content bound to the target element;

[0062] storing the target element and the annotation content bound to the target element.

[0063] In a possible implementation, the determining of the target element based on the screen projection content comprises:

[0064] performing page element recognition on the screen projection content to determine at least one page element;

[0065] determining the target element from the at least one page element.

[0066] In a possible implementation, the determining of the target element from the at least one page element comprises:

[0067] determining the target element from the at least one page element based on the position of the annotation.

[0068] In a possible implementation, the determining of the annotation content bound to the target element comprises:

[0069] obtaining first position information of the target element;

[0070] determining the annotation content bound to the target element based on the first position information of the target element.

[0071] In a possible implementation, the determining of the annotation content bound to the target element based on the first position information of the target element comprises:

[0072] obtaining second position information of each annotation content;

[0073] determining the annotation content bound to the target element according to the first position information and the second position information.

[0074] In a possible implementation, the screen projection content comprises a plurality of page elements, the plurality of page elements comprise at least two target elements, different target elements are bound to different annotation contents, and the storing of the target element and the annotation content bound to the target element comprises:

[0075] store the at least two target elements respectively with the bound comment content.

[0076] In a possible implementation, after the target element and the comment content bound with the target element are stored, the method further includes:

[0077] obtaining a display position of the target element;

[0078] in a case where there is a blank position around the display position, displaying the comment content bound with the target element in the blank position.

[0079] In a possible implementation, the storing of the target element and the comment content bound with the target element further includes:

[0080] binding and storing the target element with a standard font corresponding to the comment content bound with the target element;

[0081] The method further includes:

[0082] displaying the comment content bound with the target element and the standard font corresponding to the comment content.

[0083] In a possible implementation, the storing of the target element and the comment content bound with the target element includes:

[0084] storing the target element and the comment content bound with the target element in a first folder;

[0085] The method further includes:

[0086] displaying the comment content bound with the target element, and storing the comment content bound with the target element and newly generated comment content in a second folder, the second folder being a subfolder of the first folder.

[0087] A display device of comment information is also provided, the device being applied to a first device, the first device being connected with a second device; the device includes:

[0088] a receiving module, configured to receive screen projection content sent by the second device;

[0089] a display module, configured to display the screen projection content, wherein the screen projection content that has been displayed historically contains a target element, and the target element is bound with comment content;

[0090] an identification module, configured to identify reproduction of the target element based on the screen projection content, and the target element is bound with comment content;

[0091] The display module is further configured to reproduce the annotation content bound to the target element when it is identified that the target element is reproduced.

[0092] In a possible implementation, the identification module is further configured to determine the target element based on the screen projection content.

[0093] In a possible implementation, the identification module is configured to identify page elements of the screen projection content, and determine at least one page element; and determine the target element from the at least one page element.

[0094] In a possible implementation, the identification module is configured to determine the target element from the at least one page element based on a position of the annotation.

[0095] In a possible implementation, the apparatus further includes:

[0096] The first storage module is configured to store the target element and the annotation content bound to the target element.

[0097] In a possible implementation, the identification module is further configured to obtain first position information of the target element; and determine the annotation content bound to the target element based on the first position information of the target element.

[0098] In a possible implementation, the identification module is configured to obtain second position information of each annotation content; and determine the annotation content bound to the target element according to the first position information and the second position information.

[0099] In a possible implementation, the first storage module is further configured to store an original size of the target element.

[0100] The identification module is further configured to determine a display size of the reproduced target element and the original size of the target element; and determine a scaling ratio based on the display size and the original size of the target element.

[0101] The display module is configured to display the annotation content bound to the reproduced target element according to the scaling ratio.

[0102] In a possible implementation, the first storage module is further configured to store relative position information between the target element and the annotation content bound to the target element.

[0103] The identification module is further configured to determine an element position of the reproduced target element; and determine position information of the annotation content bound to the reproduced target element based on the element position of the target element and the relative position information.

[0104] The display module is configured to display the reproduced comment content of the target element binding based on position information of the reproduced comment content of the target element binding.

[0105] In a possible implementation, the identification module is configured to determine a similarity between the element in the screen projection content and the stored target element, the similarity including at least one of a content similarity or a position similarity; and determine that the element in the screen projection content is the reproduced target element when the similarity is greater than a similarity threshold.

[0106] In a possible implementation, the display module is configured to obtain display position information of the reproduced target element; and display the comment content of the reproduced target element binding at a blank position when it is determined according to the display position information that a position other than the reproduced target element is the blank position.

[0107] In a possible implementation, a plurality of display windows are displayed in a screen displaying the screen projection content, the plurality of display windows display different content, and the display module is configured to display the screen projection content through a target display window in the plurality of display windows.

[0108] In a possible implementation, the display module is further configured to display a plurality of comment pages on the displayed screen projection content in response to a comment page creation operation, and any comment page in the plurality of comment pages is used to reproduce the comment content of the target element binding on the any comment page when the reproduced target element is identified.

[0109] In a possible implementation, the display module is further configured to replace the comment content displayed on the screen projection content based on a comment page switching operation in response to the comment page switching operation.

[0110] In a possible implementation, the first device and the second device are connected through a high-definition multimedia interface (HDMI) connection line.

[0111] In a possible implementation, the apparatus further includes:

[0112] The generation module is configured to generate a comment file based on the reproduced target element and the comment content corresponding to the reproduced target element, the comment file being used to share or display the reproduced target element and the comment content corresponding to the reproduced target element.

[0113] In a possible implementation, the target element is the entire page of the screen projection content, and the display module is further configured to compare the screen projection content of a current page with stored screen projection content, and the stored screen projection content has corresponding annotation content; and in a case where the screen projection content of the current page is consistent with the stored screen projection content, display the annotation content corresponding to the stored screen projection content.

[0114] In a possible implementation, the apparatus further includes:

[0115] The creating module is configured to, in response to a trigger operation of starting annotation, compare screen projection content of a current page with stored target elements; and in a case where the screen projection content of the current page is inconsistent with the stored target elements, create a screen projection file for the current page, and the created screen projection file is used to store annotation content of the current page.

[0116] In a possible implementation, the screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, and different target elements are bound to different annotation content.

[0117] In a possible implementation, the apparatus further includes:

[0118] The second storage module is configured to bind and store the target element and a standard font corresponding to annotation content bound to the target element.

[0119] The display module is configured to, in a case where the target element is reproduced, display the annotation content bound to the target element and the standard font corresponding to the annotation content.

[0120] Also provided is a storage apparatus of annotation information, the apparatus being applied to a first device, the first device being connected with a second device; and the apparatus including:

[0121] The receiving module is configured to receive screen projection content sent by the second device.

[0122] The determining module is configured to determine a target element based on the screen projection content.

[0123] The determining module is further configured to determine annotation content bound to the target element.

[0124] The storage module is configured to store the target element and the annotation content bound to the target element.

[0125] In a possible implementation, the determining module is configured to identify page elements of the screen projection content, and determine at least one page element; and determine a target element from the at least one page element.

[0126] In a possible implementation, the determining module is configured to determine the target element from the at least one page element based on the position of the annotation.

[0127] In a possible implementation, the determining module is configured to obtain first position information of the target element, and determine the annotation content bound by the target element based on the first position information of the target element.

[0128] In a possible implementation, the determining module is configured to obtain second position information of each annotation content, and determine the annotation content bound by the target element according to the first position information and the second position information.

[0129] In a possible implementation, the screen projection content includes a plurality of page elements, the plurality of page elements include at least two target elements, different target elements are bound to different annotation contents, and the storing module is configured to store the at least two target elements and the bound annotation contents respectively.

[0130] In a possible implementation, the apparatus further includes:

[0131] The obtaining module is configured to obtain a display position of the target element.

[0132] The first displaying module is configured to display the annotation content bound by the target element in a blank position around the display position in a case where the blank position exists.

[0133] In a possible implementation, the storing module is further configured to bind and store the target element and a standard font corresponding to the annotation content bound by the target element.

[0134] The apparatus further includes:

[0135] The second displaying module is configured to display the annotation content bound by the target element and the standard font corresponding to the annotation content.

[0136] In a possible implementation, the storing module is configured to store the target element and the annotation content bound by the target element in a first folder.

[0137] The apparatus further includes:

[0138] The third displaying module is configured to display the annotation content bound by the target element.

[0139] The storing module is further configured to store the annotation content bound by the target element and a newly generated annotation content in a second folder, and the second folder is a subfolder of the first folder.

[0140] In another aspect, a computer device is provided, the computer device comprising a processor configured to load and execute at least one computer program to cause the computer device to implement any of the methods described above.

[0141] In another aspect, a computer-readable storage medium is provided, the non-transitory computer-readable storage medium having stored therein at least one computer program, the at least one computer program being loaded and executed by a processor to cause a computer to implement any of the methods described above.

[0142] In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any of the methods described above.

[0143] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

[0144] The technical solutions provided by the present application realize monitoring the change of the projection content by identifying the reproduction of the target element by the first device, and reproduce the annotation content bound to the target element when the reproduction of the target element in the projection content is identified, which improves the use convenience. Compared with monitoring the change of the projection content by the second device, the resource processing amount of the second device is reduced. Furthermore, compared with monitoring by the second device, monitoring by the first device improves the stability of monitoring the projection content. BRIEF DESCRIPTION OF DRAWINGS

[0145] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0146] FIG. 1 is a schematic diagram of an implementation environment of a display method of annotation information according to an embodiment of the present application;

[0147] FIG. 2 is a flowchart of a display method of annotation information according to an embodiment of the present application;

[0148] FIG. 3 is a schematic diagram of a connection relationship between devices according to an embodiment of the present application;

[0149] FIG. 4 is a schematic diagram of a plurality of display windows according to an embodiment of the present application;

[0150] Fig. 5 is a flow chart of a process of binding the content of a comment and a target element according to an embodiment of the present application;

[0151] Fig. 6 is a schematic diagram of a process of displaying a comment according to an embodiment of the present application;

[0152] Fig. 7 is a schematic diagram of a process of zooming and reproducing a comment according to an embodiment of the present application;

[0153] Fig. 8 is a schematic diagram of a display interface of a comment page according to an embodiment of the present application;

[0154] Fig. 9 is a schematic diagram of a scenario of a display method of comment information according to an embodiment of the present application;

[0155] Fig. 10 is a flow chart of a storage method of comment information according to an embodiment of the present application;

[0156] Fig. 11 is a schematic diagram of a structure of a display device of comment information according to an embodiment of the present application;

[0157] Fig. 12 is a schematic diagram of a structure of a storage device of comment information according to an embodiment of the present application;

[0158] Fig. 13 is a schematic diagram of a structure of a server according to an embodiment of the present application;

[0159] Fig. 14 is a schematic diagram of a structure of a terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0160] In order to make the objects, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0161] It should be noted that the terms "first", "second" and the like (if any) in the description of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application.

[0162] In the field of display technology, the connection between a first device (such as a smart TV, a projector or a dedicated display screen) and a second device (such as a smartphone, a tablet computer or a notebook computer) becomes more convenient. It enriches the content display mode in various scenarios such as conferences, teaching and entertainment, and promotes the efficient dissemination and interaction of information. After the file content is projected from the second device to the display screen of the first device, the user can explain or demonstrate the content displayed by the first device. In order to further improve the interactivity, clarity and understanding of the demonstration, the projected content will be annotated in real time. Through annotation, the user can directly add highlights, arrows, text boxes, notes and other elements to the projected document, picture or video, so as to highlight the key points, explain the difficulties or guide the audience's sight, thereby enhancing the attraction and effect of the demonstration.

[0163] In the related art, the screen projection software is installed on both the screen projection end (i.e. the second device) and the receiving end (i.e. the first device) to capture, encode, transmit and decode the screen content. The screen projection software of the second device is used to listen to the screen content changes, such as page turning, and the screen projection software of the first device is used to receive and display the screen content. The first device is installed with annotation software, and the annotation software and the screen projection software cooperate to run, the first device receives the page information from the second device and the handwriting input of the display screen of the first device, and generates and saves the annotation content. Thus, the annotation content can be displayed in real time and synchronized with the projection content, realizing the function of annotating with the page. However, in the related art, the second device needs to listen to the changes of the screen content, and send the changes of the screen content to the first device. The sending process may be disturbed, and the content listened to by the second device may be delayed too much compared with the content displayed by the first device.

[0164] The embodiment of the present application provides a display method of annotation information, which is used for realizing annotation of projection content. Please refer to FIG. 1, which shows an implementation environment schematic diagram of the display method of annotation information provided by the embodiment of the present application. The implementation environment can include a computer device 101 and a display device 102, and the computer device 101 and the display device 102 are connected. Optionally, the display device 102 includes a display screen, which can be a screen installed on the display device 102, in which case the display screen is part of the display device 102.

[0165] The embodiments of the present application do not limit the type of the computer device 101. For example, the computer device 101 can be a terminal or a server. Alternatively, the terminal can be any electronic product that can interact with a user through one or more of a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, a handwriting device, and the like, such as a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart car, a smart television, a smart speaker, and the like. The server can be a single server or a server cluster composed of multiple servers, or a cloud computing service center.

[0166] The embodiments of the present application do not limit the type of the display screen of the display device 102. For example, the display screen can be an LED (Light Emitting Diode) screen or an LCD (Liquid Crystal Display) screen, and the like.

[0167] Those skilled in the art should understand that the computer device 101 and the display device 102 described above are only examples, and other existing or future computer devices and display devices, such as those applicable to the present application, should also be included in the protection scope of the present application and are hereby incorporated by reference.

[0168] Referring to FIG. 2, FIG. 2 is a flowchart of a display method of annotation information according to an embodiment of the present application. The display method of annotation information is described by taking a first device as an example. The first device is connected with a display screen, and the first device is connected with a second device. For example, the first device can be the display device 102 shown in FIG. 1, and the second device can be the computer device 101 shown in FIG. 1. As shown in FIG. 2, the display method of annotation information includes, but is not limited to, the following steps 201-204.

[0169] In step 201, the first device receives the screen projection content sent by the second device.

[0170] In the embodiments of the present application, the first device is a device responsible for displaying the screen projection content, such as a large screen terminal, a smart all-in-one machine, an electronic blackboard, etc. The first device has a display screen for displaying the screen projection content in a conference room, a classroom, or a display site, etc. Optionally, the first device is installed with a source display software. The source display software is used to receive and display the screen projection content from the second device. The second device is a device initiating the screen projection, for example, a notebook computer, a desktop computer, a mobile phone, or a tablet computer, etc. The second device can play static or dynamic screen projection content, such as a PPT (PowerPoint), a Word document, a video, etc. The second device transmits the played screen projection content to the first device. The embodiments of the present application do not limit the specific content of the screen projection content, nor the timing of the second device sending the screen projection content. In addition, the second device and the first device can establish a connection through wired or wireless technology, thereby ensuring that the second device can transmit the played screen projection content to the first device.

[0171] In a possible implementation, the first device and the second device are connected through a connection line. Compared with wireless screen projection realized by software, the signal transmission between the first device and the second device connected through the connection line is more stable. The embodiments of the present application do not limit the type of the connection line, for example, the connection line can be an HDMI (High Definition Multimedia Interface) connection line, and the HDMI connection line has high signal stability. The HDMI provides a high-speed and stable data transmission channel, reducing the screen projection problems caused by network delay or signal interference. In addition, the HDMI-based screen projection has good compatibility, which can support various operating systems and device types.

[0172] Referring to a connection relationship between a first device and a second device shown in FIG. 3, the second device (i.e., a screen projection terminal) is connected to the first device (i.e., a receiving terminal) through an HDMI connection line. The second device sends the screen projection content through the HDMI connection line, and the first device receives the screen projection content. The first device can make annotations and share the screen projection content, wherein the annotation and sharing operations can be referred to the subsequent related description.

[0173] In step 202, the screen projection content is displayed, wherein the historical screen projection content contains a target element, and the target element is bound with annotation content.

[0174] In the embodiments of the present application, the first device is installed with HDMI source display software for receiving and displaying HDMI signals from the second device. After the second device transmits the screen projection content to the first device through the HDMI connection line or other screen projection technology, the HDMI interface of the first device receives the transmitted screen projection content. The first device decodes and processes the received HDMI signals through the HDMI source display software, and the decoded audio and video data are sent to the display screen of the first device for display.

[0175] In a possible implementation, the screen displaying the screen projection content displays a plurality of display windows, and the plurality of display windows display different contents. Displaying the screen projection content includes displaying the screen projection content through a target display window in the plurality of display windows.

[0176] The display screen of the first device is divided into a plurality of regions, and each region corresponds to a display window. The display window can be fixed size, adjustable size or floating, depending on the design of the operating system and the needs of user configuration. Each display window can independently display different contents, such as different documents, pictures, videos or application interfaces. Independent display of each display window enables the user to handle multiple tasks or view multiple information at the same time. Optionally, among the plurality of display windows, the user can specify one or more display windows as target display windows for displaying content from the second device. The specification process of the target display window can be automatic, for example, based on the system configuration of the first device or the preset rules to indicate the target display window, or manual, for example, selected by the user according to the current needs to determine the target display window.

[0177] When the screen projection content is transmitted to the first device through the HDMI connection line or other screen projection technology, the first device recognizes and captures the screen projection content. For example, the HDMI source display software recognizes and captures the screen projection content. Then, according to the preset or user-specified rules, the screen projection content is directed to the selected target display window for display. Optionally, in order to ensure that the screen projection content can be correctly and beautifully displayed in the target display window, the first device may perform content synchronization and adaptation processing. For example, adjusting the resolution, scaling ratio, color setting, etc. of the screen projection content to match the size and characteristics of the target display window.

[0178] Optionally, the user can switch the target display window at any time according to needs, or adjust the size, position and other attributes of the target display window. The multi-window display mode enables the user to view the screen projection content while continuing to perform other tasks or view other information in other windows, thereby improving work efficiency and user experience.

[0179] Exemplarily, referring to a multi-display window diagram shown in FIG. 4, the display screen includes four display windows, window 1, window 2, window 3 and window 4, and the user can freely select any one of the windows as a target display window. Different annotation operations can be performed for different display windows. The content under different display windows is stored separately.

[0180] In the embodiment of the present application, the screen projection content can be a previously displayed screen projection content, that is, the screen projection content is a historically displayed screen projection content, and the historically displayed screen projection content contains a target element. The target element refers to an element that is bound with annotation content. Exemplarily, the target element can be a page element in the page of the screen projection content that is bound with annotation content, and the page element is a basic unit of the page of the screen projection content. The embodiment of the present application does not limit the division manner of the basic unit of the page, and the granularity of the division is different, and the page elements are different. In addition, the page element can also be of multiple types, including but not limited to a text format page element, a picture format page element, a chart format page element and a video format page element, etc.

[0181] For example, the screen projection content is a document, the page element can be each paragraph in the document, and the target element can be a certain paragraph in the document that is bound with annotation content. For another example, the screen projection content is a picture, the page element is a certain region in the picture, and the target element is a certain content in the region that is bound with annotation content. For another example, the screen projection content is a chart, the page element is each data point in the chart, and the target element is a certain data point that is bound with annotation content. In addition, the target element can also be the entire page of the screen projection content, and the embodiment of the present application does not limit the target element. Furthermore, the embodiment of the present application also does not limit the number of times of historical display of the screen projection content, and all the historically displayed screen projection content can have the possibility of reproduction of the target element.

[0182] In step 203, reproduction of the target element is identified based on the screen projection content.

[0183] In the embodiments of the present application, after the first device receives the screen projection content, the target element is determined based on the screen projection content by identifying the received screen projection content. For example, the target element can be determined by artificial selection, or can be determined automatically without artificial selection. In a possible implementation, the target element is determined based on the screen projection content, including: identifying page elements of the screen projection content, and determining at least one page element; and determining the target element from the at least one page element. The page element can be a basic unit of a page, for example, the page element can be a picture, a character, or the like in the entire page. The embodiments of the present application do not limit the division manner of the page element, which can be a minimum unit of the page, or a combination of the minimum units. For example, the minimum unit is a character, and the page element is a line of text composed of multiple characters.

[0184] Regardless of the division of the page element, the first device can identify the page element in the screen projection content by image recognition, text recognition, or the like, and then identify the target element to which the annotation content is bound from the page element. As described in step 202, the target element refers to an element to which the annotation content is bound. For example, the target element can be a certain page element in the page of the screen projection content, such as a certain paragraph in a document, a certain area in a picture, or a certain data point in a chart, and can also be a character, or the entire page of the screen projection content. Regardless of the target element, the first device can identify the page element according to a preset rule or a user-specified condition, and identify the page element as the target element based on the annotation content bound to the page element. The embodiments of the present application do not limit the identification manner. For example, the page element can be identified based on the stored characteristics such as the type, position, size, color, or the like of the element, and then the target element to which the annotation content is bound is determined.

[0185] For example, after the method provided in the embodiments of the present application adds the annotation content to the target element in the screen projection content, the annotation content can be bound to the target element and stored. For example, the target element and the annotation content are bound and stored according to the position. Then, the target element is determined from the at least one page element, which can be determined based on the position of the annotation. The position of the annotation includes, but is not limited to, the initial position of the annotation, or the center position of the handwriting corresponding to the annotation, and the like. The initial position of the annotation can be the position of the annotation when the annotation is generated. For example, after the screen projection content is displayed, the user selects a certain page element in the screen projection content to add the annotation content, and the position of the generated annotation content is the initial position of the annotation. In addition, the annotation content can be handwriting, and the center position of the handwriting can be used as the position of the annotation. The determination manner of the center position of the handwriting is not limited in the embodiments of the present application. Regardless of the manner of representing the position of the annotation, the position of the annotation can be understood as the position of the annotation content.

[0186] For example, when the target element is determined from the at least one page element based on the position of the annotation, the page element closest to or coinciding with the position of the annotation in the page element can be determined as the target element.

[0187] In addition, the process of storing the target element and the annotation content bound to the target element by the first device can include the first device taking a screenshot of the screen mirroring content, and assigning a custom page number to each screenshot obtained by the screenshot. The page number is used to identify the order and position of the screenshot. In a page with handwriting display, i.e., with annotation content, the page number is usually incremented to reflect the progress of the user reading. For example, the first device extracts a unique feature value from the current screenshot by using an image comparison algorithm. The feature value needs to have sufficient discrimination so that the corresponding page picture, i.e., the screenshot of the screen mirroring content, can be quickly found by comparing the feature value. In the embodiments of the present application, the feature value is closely related to the content of the screenshot and can uniquely identify the screenshot in the database.

[0188] Optionally, for the storage process, the screenshot is saved in a common picture format, such as png (Portable Network Graphics), jpg (joint photographic experts group), etc. in the first device. The saved file path can be recorded for subsequent access and reference. The annotation file contains the handwriting data of the user on the current page, which is the annotation content, and can be in a picture format (such as a screenshot of the user's handwriting) or a vector data (such as the coordinate point information of the handwriting).

[0189] Then, the first device saves the page number, the feature value, the screenshot file path, the annotation file path, and the synthetic picture path, etc. in the database according to the corresponding relationship. The structured storage method facilitates subsequent query and quick comparison, improving the efficiency of data management. While saving the current screenshot data or after saving the current screenshot data, the first device extracts the feature value of the current image by using the image comparison algorithm, and queries whether there is annotation content corresponding to the feature value in the database. If the query result shows that there is corresponding annotation content, the first device will display the last annotation content. In this way, the user can see the previous handwriting and the new screenshot content on the same page, which helps to maintain the coherence of reading and the integrity of annotation.

[0190] In another possible implementation, the target element includes element information, the annotation content bound to the target element includes annotation information, and the target element and the annotation content bound to the target element are stored based on correspondence between the element information and the annotation information; wherein the element information includes at least one of an element identifier (ID), an element type, element content, or an element position, and the annotation information includes at least one of an annotation identifier (ID), annotation content, or an annotation position. The element identifier (ID) can be used to uniquely identify the corresponding target element, the element type is used to indicate the type of the target element, for example, whether the target element is a picture, text, a table, or the like. The element content is the content presented by the target element, for example, if the target element is a picture, the element content is the content presented by the picture, and if the target element is text, the element content is the content presented by the text. The element position is only the position of the target element in the screen projection content, which can be represented by coordinates or a relative position. The embodiments of the present application do not limit the element information, and any element information that can query and identify the corresponding target element is acceptable. The annotation information is similar to the element information, and any annotation information that can identify and query the annotation content is acceptable, and will not be described again.

[0191] Optionally, before reproducing the annotation content bound to the target element, the method further includes: obtaining element information of the reproduced target element, the element information including an element position; obtaining the annotation content bound to the reproduced target element based on the element information; and reproducing the annotation content bound to the target element, including displaying the reproduced annotation content bound to the target element at a display position corresponding to the element position. Since the target element and the bound annotation content are stored correspondingly and are bound to each other, the reproduced annotation content bound to the target element is displayed at the display position corresponding to the element position, so that they are displayed correspondingly and more clearly, facilitating viewing.

[0192] In addition, during the screen projection process, due to the dynamic nature of the screen projection content (such as PPT page turning, video playing, etc.), the first device needs to track and confirm the reproduction of the target element in the screen projection content in real time, so as to ensure that the annotation content can be accurately associated with the target element in the current screen projection page. The annotation content is additional information input by a user when watching the screen projection content through handwriting, typing, or other means. The annotation content is used to explain, supplement, or emphasize the screen projection content.

[0193] In a possible implementation, before identifying the reproduction of the target element based on the screen projection content, the method further includes: obtaining a plurality of screen projection contents at different times; determining an instruction type of the screen projection content sent by the second device according to the screen projection contents at the consecutive at least two times, the instruction type including a page turning instruction and a non-page turning instruction; and storing the target element and the annotation content bound to the target element, wherein the target element and the annotation content bound to the target element in the plurality of screen projection contents at different times are stored based on the instruction type. Thereafter, the annotation content bound to the reproduced target element can be obtained from the stored target element and the annotation content bound to the target element.

[0194] The first device continuously receives the screen projection content at different time points from the second device, or receives the screen projection content at intervals. The time points or intervals can be fixed (for example, 5 seconds). The first device determines the instruction type of the screen projection content according to the screen projection content at at least two consecutive time points. The instruction type includes two types: a page turning instruction and a non-page turning instruction. The page turning instruction indicates that the content has obvious changes between pages, such as PPT page turning, etc. The non-page turning instruction indicates that the content has local changes or no obvious changes within the page, such as playing elements in PPT, document sliding, etc.

[0195] For example, the first device can use image comparison, text difference analysis and other algorithms to identify the instruction type. The image comparison, text difference analysis and other algorithms can analyze the similarity and difference between the continuous screen projection content, so as to determine the instruction type thereof.

[0196] After determining the instruction type of the screen projection content, the first device performs target element identification on the screen projection content at the current time point. For each identified target element, the first device binds the annotation content associated with the target element and stores it in the database. Optionally, the design of the database should support efficient query and retrieval operations, so as to quickly obtain the target element and its annotation content when needed.

[0197] In a possible implementation, the first device can use different storage strategies according to different instruction types. For example, for the screen projection content of the page turning instruction, the first device can store all elements in the entire screen projection content as target elements; and for the screen projection content of the non-page turning instruction, the first device can store single or multiple elements as target elements.

[0198] When new screen projection content arrives, the first device detects whether the new screen projection content contains a reproduction of the previously stored target element. In the case of detecting the reproduction of the target element, the first device retrieves the annotation content bound to the target element from the database. The retrieval process can be based on the unique identifier of the target element (such as page number, coordinates, text content, etc.) or other information. The first device superimposes or displays the retrieved annotation content on the current screen projection content, so that the user can view the screen projection content and the related annotation content at the same time.

[0199] In a possible implementation, the at least two continuous times include a first time and a second time, and the determining, according to the screen projection content of the at least two continuous times, of the instruction type of the screen projection content includes: in a case where the screen projection content of the first time is replaced by the screen projection content of the second time, determining that the instruction type is a page turning instruction. The first time refers to a first screen projection content time point before the second time. The second time refers to another screen projection content time point immediately after the first time. In the case of the page turning instruction, the screen projection content of the second time completely replaces the screen projection content of the first time. For example, a page turning, a document scrolling to the next page, or switching to completely different content, and the like. For example, in a PPT presentation process, page turning is a common operation, and the first device can determine that the instruction type is a page turning instruction by recognizing the page turning operation, and adjust the display strategy of the annotation content accordingly.

[0200] In a possible implementation, the at least two continuous times include a first time and a second time, and the storing, based on the instruction type, of the target element in the plurality of screen projection contents of different times and the annotation content bound to the target element includes: in a case where the instruction type is a page turning instruction, storing the screen projection content of the first time and the annotation content bound to the screen projection content of the first time, the screen projection content of the first time being the target element of the first time.

[0201] When the first device determines that the current instruction type is a page turning instruction, the screen projection content of the first time has been completely replaced by the screen projection content of the second time, and the user is viewing a new page or content block. In this case, the first device stores the screen projection content of the first time and the annotation content bound thereto. That is, the entire screen projection content of the first time can be used as the target element of the first time.

[0202] Optionally, in a case where the screen projection content of the first time is not replaced by the screen projection content of the second time, and at least one element content in the screen projection content of the second time is updated compared with the screen projection content of the first time, it is determined that the instruction type is a non-page turning instruction. In the case of the non-page turning instruction, the screen projection content of the second time does not completely replace the screen projection content of the first time, but at least one page element therein is updated. The update can include text modification, picture replacement, data update, and the like, but the structure and most of the content of the entire page or document remain unchanged.

[0203] In a possible implementation, the at least two continuous times include a first time and a second time, and the storing, based on the instruction type, of the target element in the plurality of screen projection contents of different times and the annotation content bound to the target element includes: in a case where the instruction type is a non-page turning instruction, obtaining element content in the screen projection content of the first time, storing the element content and the annotation content bound to the element content, the element content being the target element of the first time.

[0204] In this case, when the first device determines that the current instruction type is a non-page turning instruction, it means that the screen projection content at the second time does not completely replace the screen projection content at the first time, but only updates part of the elements in the screen projection content at the first time. In this case, the first device first identifies the updated page element in the screen projection content at the first time as the target element, and then obtains the specific content of the target element. The first device stores the updated element content and the annotation content bound to the element content.

[0205] In a possible implementation, before storing the target element and the annotation content bound to the target element, the method further includes: obtaining first position information of the target element; and determining the annotation content bound to the target element based on the first position information of the target element. For example, determining the annotation content bound to the target element based on the first position information of the target element includes: obtaining second position information of each annotation content; and determining the annotation content bound to the target element according to the first position information and the second position information.

[0206] For example, the first position information includes the coordinates of the target element, such as the pixel positions of the X-axis and the Y-axis, the size of the target element, such as the width and the height, and other possible positioning parameters. The second position information refers to the specific position of the annotation content added by the user to the target element on the display screen. For example, the second position information includes the coordinates and the size of the annotation.

[0207] After obtaining the position information of the target element and the annotation content, the first device determines which annotation content is bound to which target element according to the position information. The embodiments of the present application do not limit the method of determining the binding relationship according to the position information. For example, if the position of the annotation content is close to or overlaps with the position of a target element, it can be determined that the annotation content is bound to the target element.

[0208] Referring to a process flow diagram of binding of annotation content and target element shown in FIG. 5, the process includes the following steps 501-505.

[0209] Step 501: Analyzing the entire layout of the page in response to an annotation instruction.

[0210] For example, the user triggers the annotation instruction through an interface operation (such as clicking, long pressing, sliding, etc. on the display screen). After receiving the annotation instruction, the first device first analyzes the layout of the entire page, and identifies different regions and page elements in the page, such as text paragraphs, pictures, tables, charts, etc., that is, identifies the page elements in the screen projection content.

[0211] Step 502, identify the element type based on the layout analysis algorithm, and determine the element type according to the corresponding text, image and table recognition algorithm to recognize the page element.

[0212] The element type identification is based on the results of layout analysis, and different types of page elements (such as text, image, table) are distinguished.

[0213] Step 503, determine the calibration point of the page element.

[0214] For example, according to the shape and size of the page element, the appropriate calibration point is selected. For rectangular elements (such as text paragraphs, table cells), the center point, top left corner, bottom right corner, etc. can be selected as the calibration point.

[0215] Step 504, determine the position of the annotation handwriting coordinates relative to the calibration point of the page element.

[0216] For example, the offset of the annotation coordinates (x1-x0, y1-y0) relative to the calibration point is calculated, where (x0, y0) is the coordinate of the calibration point, and (x1, y1) is a point of the annotation coordinates, indicating the position of the annotation content. The distance l and angle a of the annotation point and the calibration point are calculated, so as to accurately determine the position relationship between the annotation content and the page element.

[0217] Step 505, bind and store the position of the annotation handwriting coordinates relative to the calibration point of the page element.

[0218] For example, a unique content ID (Identification, identity information) is generated for each recognized page element, which can correspond to the element type, page element summary (such as the first few words of the text, the hash value of the image, the data of a certain row and column of the table, etc.), calibration point information (such as top left corner, bottom right corner coordinates). The ID of the annotation is bound and stored with the corresponding content ID, and the element type, page element summary, calibration point information, etc. corresponding to the ID can also be bound and stored. Optionally, related information can also be stored, including but not limited to the coordinate offset or distance and angle information of the annotation. Then, by using a database or a file system to store the annotation ID and the content ID, it can be ensured that it can be quickly retrieved and updated.

[0219] Optionally, there may be multiple annotation contents close to or overlapping with the same page element position, or a single annotation close to multiple page element positions. In this case, the first device may need to use a more complex correlation algorithm, such as considering the relative position of the annotation and the page element, the size and shape of the annotation, the user's historical operation records, etc. For example, there is an arrow-shaped part in the annotation, which can be determined to point to the page element corresponding to the annotation. In addition, in the above steps, the page element bound with the annotation content is the target element.

[0220] In a possible implementation, the identifying the reproduction of the target element based on the screen projection content comprises: determining a similarity between an element in the screen projection content and the stored target element, the similarity comprising at least one of content similarity or position similarity; and determining that the element in the screen projection content is the reproduction of the target element when the similarity is greater than a similarity threshold.

[0221] The similarity can be a comprehensive evaluation based on various factors such as visual features, text content, structural information, and the like. In order to determine whether two elements are similar enough to be considered as different versions or representations of the same element, the first device needs to set a first similarity threshold. When the first similarity is greater than the threshold, the first device determines that the two elements are similar. The first similarity threshold can be freely set by the user, and can also be adjusted according to the application scenario.

[0222] If the first similarity is greater than the first similarity threshold, the first device determines that the current screen projection content is consistent with a certain target element stored, and displays the annotation content bound to the target element.

[0223] In a possible implementation, when the instruction type is a page turning instruction, the first device compares the similarity between the current screen projection content (i.e., the reproduction of the target element) and any target element stored.

[0224] When the instruction type is a non-page turning instruction, the first device compares the similarity between the target element (i.e., the reproduction of the target element) in the current screen projection content and the stored target element, i.e., the second similarity. The calculation method of the similarity can be the same as or different from that of the similarity under the page turning instruction. Similarly, in order to determine whether two page elements are similar enough, the first device sets a second similarity threshold. When the second similarity is greater than the second similarity threshold, the first device determines that the two element contents are similar. The second similarity threshold can be freely set by the user, and can also be set based on experience and adjusted according to the application scenario. The second similarity threshold and the first similarity threshold can be the same or different.

[0225] If the second similarity is greater than the second similarity threshold, the first device determines that the target element in the current screen projection content is consistent with the target element content stored, and displays the annotation content bound to the target element content.

[0226] The embodiments of the present application do not limit the determination manner of the first similarity and the second similarity, for example, the image comparison algorithm is implemented. The image comparison algorithm is to extract the key information or features in the image, and compare the differences of the features in two pictures. Among them, the feature matching is a comparison method based on the local feature points of the image. The matching between images is realized by detecting the key points (such as corner points, edge points, etc.) in the image and calculating the feature vectors of the key points.

[0227] The embodiments of the present application do not limit the feature matching algorithm, for example, the SIFT (Scale-Invariant Feature Transform) algorithm, the SIFT algorithm is a scale-invariant feature transform algorithm, which can detect local feature points in an image and generate a unique descriptor for each feature point. The descriptor has certain invariance to the scale scaling, rotation and even affine transformation of the image. The SURF (Speeded Up Robust Features) algorithm, the SURF algorithm is an improvement of the SIFT algorithm, aiming to improve the calculation speed and robustness. The integral image and box filter are used to approximate the Gaussian second derivative in the SIFT algorithm, so as to speed up the detection of feature points and the generation of descriptors. The ORB (Oriented FAST and Rotated BRIEF) algorithm, the ORB algorithm combines the advantages of the FAST feature point detector and the descriptor and is improved. The FAST (Features from accelerated segment test) algorithm is used to quickly detect the feature points in the image and calculate the direction information (Oriented) of the feature points, and then the improved descriptor is used to describe the feature points.

[0228] In the feature matching process, the algorithm calculates the similarity (such as Euclidean distance, Hamming distance, etc.) between the feature point descriptors in two pictures, and judges the similarity between the pictures according to the number and quality of the matching. If the number of matched feature points is enough and the distribution is reasonable, it is considered that the two pictures may represent different views of the same page (that is, no page turning); otherwise, it is considered that the page turning has occurred.

[0229] Optionally, with the development of deep learning technology, Convolutional Neural Networks (CNN) are applied in image recognition, classification, and retrieval, etc. Using a pre-trained CNN model for image comparison, high-level feature representations of images can be extracted, and similarity judgments can be made based on these features. The similarity judgment process can include a Siamese network based on a deep learning feature extractor (such as VGG (Visual Geometry Group), ResNet (Residual Neural Network), etc., where VGG is a type of convolutional neural network) and a loss function specifically designed for image retrieval or verification (such as cosine similarity, Triplet Loss, etc.) to extract high-level feature representations of images, and based on the distance or similarity of the feature vectors to determine whether the images are the same or similar.

[0230] The Siamese network is a special neural network structure composed of two or more identical sub-networks that share the same weights and bias terms. In the image comparison task, two pictures to be compared can be input into the two sub-networks of the Siamese network, their feature vectors can be extracted, and the distance or similarity between the two feature vectors can be calculated to determine whether the pictures are the same or similar. Deep learning-based feature extractors, such as VGG, ResNet, etc., deep learning models, extract high-level feature representations from input images. During the training of deep learning models, loss functions can be used to evaluate the difference between the model's prediction results and the true labels. In the image comparison task, loss functions such as cosine similarity, Triplet Loss, etc. can be used to optimize the model so that it can more accurately extract and compare image features.

[0231] Step 204, in the case of identifying the reproduction of the target element, reproducing the annotation content bound to the target element.

[0232] In one possible implementation, reproducing the annotation content bound to the target element includes: obtaining display position information of the reproduced target element; and in a case where a position outside the reproduced target element determined according to the display position information is a blank position, displaying the annotation content bound to the reproduced target element at the blank position.

[0233] Illustratively, the first device first obtains display position information of the currently reproduced target element on the display screen. The display position information can include the coordinates of the element, such as the pixel positions of the X and Y axes, the size, such as the width and height, and possibly the bounding box or contour information. The display position information of the target element can be obtained by parsing the image, document structure, or screen rendering data of the screen mirroring content.

[0234] After the display position information of the reproduced target element is determined, the first device further analyzes other regions on the display screen to determine whether there is sufficient blank position to display the annotation content. The blank position refers to the screen region that is not occupied by other elements and is suitable for displaying additional information, and the manner of determining the blank position is not limited in the embodiments of the present application.

[0235] Based on the determination of the blank position, the first device displays the annotation content bound to the reproduced target element in the blank position, thereby ensuring that the annotation content does not block or interfere with the user's viewing of the main screen content, while allowing the user to conveniently view and refer to the annotation information.

[0236] Exemplarily, referring to a display process of annotation content shown in FIG. 6, in FIG. 6A, the annotation content bound to element 1 is annotation 1, which is displayed in a blank position outside the position of element 1. In FIG. 6B, element 2 appears on the basis of element 1, at this time, annotation 1 partially overlaps with element 2, that is, annotation 1 is not in the blank position. In FIG. 6C, in the case of FIG. 6B, the position of annotation 1 is re-determined, so that annotation 1 is located in a blank position outside the positions of element 1 and element 2. Thus, annotation 1 does not interfere with element 1 and element 2.

[0237] In a possible implementation, the method provided by the embodiments of the present application further stores the original size of the target element, and before reproducing the annotation content bound to the target element, the method further includes: determining the display size of the reproduced target element and the original size of the stored target element; determining the scaling ratio based on the display size and the original size of the target element; and reproducing the annotation content bound to the target element, including: displaying the reproduced annotation content bound to the target element according to the scaling ratio.

[0238] The target element displayed by the first device can be freely enlarged or reduced based on the user's needs, and thus the first device can compare the difference in size between the display size of the reproduced target element and the original size of the stored target element to determine the scaling ratio. The first device determines the scaling size of the annotation content through the scaling ratio, thereby ensuring that the annotation content can adapt to the size of the current screen content when displayed, while maintaining appropriate readability and visibility. The manner of determining the scaling size of the annotation content can include that the first device uses a predefined scaling algorithm or rule to calculate the scaling ratio of the annotation content. For example, in the case that the reproduced target element is 50% of the stored target element, the corresponding annotation content can also be reduced to 50% of the stored annotation content.

[0239] Referring to a scaling reproduction process of annotation content shown in FIG. 7, the process includes steps 701-704.

[0240] Step 701, the layout of the page as a whole is analyzed to determine the element types of the current layout and the page elements of each element type.

[0241] In this step, the first device first parses the currently displayed page, and uses image recognition or other related technologies to identify various page elements on the page. The page elements include but are not limited to text, pictures, charts, video boxes, buttons, etc.

[0242] Step 702, compare the page elements with the contents of the target elements stored in the storage to determine the reproduced target elements.

[0243] After the first device completes the page analysis, the first device compares the page elements of the current page with the target elements pre-stored in the storage. The data in the storage can contain detailed information of the previous page version, including the ID, position, size, type, etc. of the page elements. Through comparison, the first device can identify which page elements in the current page correspond to the stored target elements, thereby determining the element information of the reproduced target elements. The way to determine the reproduced target elements can refer to the related description of determining the reproduced target elements above, which will not be repeated here.

[0244] Step 703, according to the element information of the reproduced target elements, confirm the annotation information of the annotation content bound with the target elements.

[0245] The element information of the target elements includes the original size of the target elements, element position information, etc. It can also be the relative position information of the target elements and the annotation content bound with the target elements. For the reproduced annotation content, the first device determines the calibration point associated with the annotation. The calibration point can be the center point, vertex or other feature point of the target element bound with the reproduced annotation content. The position information of the calibration point is the position information of the reproduced target elements, which is recorded and stored when the annotation is created. The first device calculates the position where the annotation content bound with the reproduced target elements should appear in the current page according to the calibration point and the relative position relationship between the annotation and the calibration point, i.e. the relative position relationship between the reproduced target elements and the target elements and the bound annotation content.

[0246] Step 704, according to the display size of the reproduced target elements and the original size of the stored target elements, determine the scaling ratio, and scale the reproduced annotation content bound with the reproduced target elements according to the scaling ratio.

[0247] In case the size of the target element is detected to change, the first device should recalculate the display size of the reproduced target element (denoted as b) and compare it with the stored original size (denoted as a). The scaling ratio can be calculated by the formula: scaling ratio = b / a. According to the calculated scaling ratio, the first device traverses all the annotation content bound to the target element and adjusts the coordinates of the bound annotation content. The adjustment of the annotation coordinates can be achieved by multiplying the original coordinates by the scaling ratio.

[0248] In a possible implementation, the first device also stores the relative position information between the target element and the annotation content bound to the target element. Before reproducing the annotation content bound to the target element, the method further includes: determining the element position of the reproduced target element; determining the position information of the reproduced annotation content bound to the target element based on the element position of the reproduced target element and the relative position information; and accordingly, reproducing the annotation content bound to the target element includes: displaying the reproduced annotation content bound to the target element based on the position information of the reproduced annotation content bound to the target element. In this way, the reproduced target element and the bound annotation content are displayed in accordance with the relative position relationship.

[0249] The embodiments of the present application do not limit the relative position information between the target element and the annotation content bound to the target element. For example, after the position information of the target element is determined, the position of the annotation content bound to the target element relative to the target element is represented by the position information of the target element, for example, the relative position information is represented by the position information of the target element and the offset between the target element and the bound annotation content. Other expression forms capable of indicating the relative position can also be used, as long as the relative position information can be used to determine the target element and the bound annotation content.

[0250] In a possible implementation, the method further includes: generating an annotation file based on the reproduced target element and the annotation content corresponding to the reproduced target element, and the annotation file is used to share or display the reproduced target element and the annotation content corresponding to the reproduced target element.

[0251] The embodiments of the present application do not limit the format of the generated annotation file, and can record and share the reproduced target element and the annotation content bound to the target element. For example, the stored target element, the position information of the target element, and the annotation content corresponding to the target element and the position information of the annotation are integrated into a unified file format by generating an annotation file. Thus, the user can share or display the target element and the annotation content bound to the target element without relying on the original screen projection device or application. Alternatively, the format of the annotation file can be designed according to specific needs. For example, the PDF (Portable Document Format) can be used in such a scenario due to its good cross-platform compatibility and non-editability.

[0252] When the annotation file is generated, the first device can provide a sharing function, allowing the user to send the file to other users or other devices through various ways such as email, cloud storage service, instant messaging tool, etc. The receiver can view the annotation content without the original screen projection device or application. In addition to the sharing function, the first device can also provide a built-in display function, allowing the user to view the annotation file directly in the application of the first device.

[0253] Exemplarily, in team collaboration, members often need to share and discuss screen projection content such as documents, pictures, etc., and add annotations to mark important information or suggest modifications. By generating an annotation file and sharing it with team members, the collaboration efficiency can be improved, and all members can see the latest annotation information. Among them, the reproduced annotation content can be consistent with the position of the stored annotation content, thereby improving the user experience. Alternatively, due to the flipping, scrolling, or zooming when reading the document, only when the element content is completely presented, the annotation content bound to the element content will appear, and if the element content only displays part of the content, which is not complete compared with the stored element content, the annotation content bound to it will not appear.

[0254] In addition, in the embodiment of the present application, the screen projection content includes a plurality of page elements, the plurality of page elements include at least two target elements, and different target elements are bound to different annotation content. The display method of the reproduction of each target element and the bound annotation content is the same, and will not be repeated here. For the case where the target element is the entire page of the screen projection content, the method further includes: comparing the current page of the screen projection content with the stored screen projection content, and the stored screen projection content has corresponding annotation content; in the case where the current page of the screen projection content is consistent with the stored screen projection content, the corresponding annotation content of the stored screen projection content is displayed. In this case, the screen projection content is the reproduced target element. When comparing the current page of the screen projection content with the stored screen projection content, the comparison method can also be the same as the comparison method of the target element and the stored target element, and the principle is the same, which will not be repeated here.

[0255] It should be noted that, whether the target element is one or more page elements in the screen projection content or the entire screen projection content, the method further includes: in response to the trigger operation of starting the annotation, comparing the current page of the screen projection content with the stored target element; in the case where the current page of the screen projection content is inconsistent with the stored target element, creating a screen projection file for the current page, and the created screen projection file is used to store the annotation content of the current page. Wherein, the current page of the screen projection content is inconsistent with the stored target element, which means that the current screen projection content is not stored, and the current screen projection content can also not have added annotation content, so that after the trigger operation of starting the annotation, the addition of the annotation can be started by creating a screen projection file, and the annotation content can be recorded through the screen projection file. In the embodiment of the present application, the created screen projection file can also be referred to as creating an annotation page, and the annotation page is used to store the annotation content of the current page. In the embodiment of the present application, the trigger operation of starting the annotation can also be referred to as the operation of creating an annotation page.

[0256] In a possible implementation, the method further includes: in response to the operation of creating an annotation page, displaying a plurality of annotation pages on the displayed screen projection content, and any annotation page in the plurality of annotation pages is used to reproduce the annotation content bound to the target element on any annotation page in the case where the target element is reproduced.

[0257] For example, the user triggers the operation of creating an annotation page through certain interface interaction (such as clicking a button, selecting a menu item, using a shortcut key, etc.). The operation of creating an annotation page can be performed at any time during the display of the screen projection content, so that the user can flexibly add and manage the annotation page according to needs. By creating an annotation page, a special layer is provided to display the annotation information bound to the screen projection content. Each annotation page can be regarded as an independent view for storing and displaying annotation content of a specific theme or classification.

[0258] In response to the creation of the annotation page operation, the first device displays one of the plurality of annotation pages on the displayed screen mirroring content. In addition, in order to improve the user experience, the first device can visually design the annotation pages, such as using different colors, icons, borders, and other elements to distinguish different annotation pages. By creating multiple annotation pages, the writing range of the annotation content can be expanded, thereby storing more annotation content. Alternatively, in the case of multiple annotation pages, the stored annotation content of the target element can be multiple, that is, one target element corresponds to multiple annotation contents.

[0259] In one possible implementation, the method further includes: in response to the switching annotation page operation, replacing the annotation content displayed on the screen mirroring content based on the switching annotation page operation.

[0260] Exemplarily, the user triggers the switching annotation page operation through some interface interaction (such as clicking the annotation page label, using the shortcut key, gesture operation, etc.). The switching operation is performed during the display of the screen mirroring content, aiming to enable the user to quickly switch to different annotation pages to view or edit the annotation content associated with the page. When the first device receives the switching annotation page operation, the first device immediately responds to the switching operation and replaces the annotation content displayed on the screen mirroring content according to the annotation page selected by the user. Exemplarily, the user clicks the page number of the first annotation page, and the annotation content included in the first annotation page is displayed on the screen mirroring content.

[0261] Referring to FIG. 8, an annotation page display interface schematic diagram is shown, wherein the display content is the screen mirroring content, and the displayed annotation pages are the plurality of annotation pages corresponding to the screen mirroring content, including annotation page 1, annotation page 2, annotation page 3, and annotation page 4. In response to the user's click operation on the label of each annotation page, the annotation content in the corresponding annotation page can be switched to be displayed.

[0262] Exemplarily, the first device supports multiple types of windows, including a window displaying HDMI source content and a multi-application window. The HDMI source window can display video or image content from different HDMI sources, and the multi-application window allows the user to open and display multiple different application programs, such as WPS documents, picture viewers, etc. An independent annotation window will be displayed on the upper layer of each main window (whether it is a source window or an application window). The annotation window allows the user to directly annotate, underline, or add text on the content of the corresponding window. The annotation window will be dynamically updated as the annotation content changes.

[0263] In a possible implementation, each window has its own independent coordinate system, that is, the annotation operation in one window does not affect the content or layout of other windows. Thus, it is ensured that the user can work independently in each window without worrying about interference between operations. The function of each window is completely independent. The user can open different applications, display different source contents, and perform independent annotation operations on each window in different windows, thereby improving the flexibility and usability of the first device.

[0264] In addition, the annotations and screenshot contents of each window can be saved separately. That is, each window has a corresponding saving path or folder for storing the annotation file, screenshot file, and composite picture of the window. In order to facilitate the user to manage and search for files, the first device can automatically create a folder for each window and save all related files of the window in the folder. Thus, the user can quickly locate the required file by browsing the folder.

[0265] In a possible implementation, the method further includes: in a case where the target element is not identified, in response to an annotation operation on the screen projection content, obtaining an annotation point corresponding to the annotation operation; in a case where no next annotation operation is received at an interval reference time, determining annotation text corresponding to the annotation point based on the annotation point; and displaying the annotation text on the screen projection content.

[0266] In the above scenario, the first device is displaying the screen projection content, but has not identified the target element, that is, the screen projection content itself does not contain the target element, that is, the displayed screen projection content is new and no annotation operation has been performed before. The user triggers an annotation operation on the screen projection content through certain interface interaction (such as clicking, touching, handwriting input, etc.). The annotation operation can be performed at any position of the screen projection content and is not limited by whether the target element is reproduced. The first device responds to the annotation operation and obtains an annotation point corresponding to the annotation operation. The annotation point refers to the specific position selected by the user on the screen projection content for annotation, which can be a point or a line.

[0267] The interval reference time is a preset time threshold for determining whether the user has completed the current annotation operation and is ready to start the next operation. The time threshold can be set according to actual needs to balance the fluency of user operation and the timeliness of the response of the first device. Within the interval reference time, the first device waits for the next annotation operation that the user may perform. If no new annotation operation is received within this time, the first device considers that the user has completed the current annotation and is ready for the next step of processing.

[0268] In the case that the next batch operation is not received at the interval reference time, the first device determines the batch text associated with the point according to the batch point. The user directly inputs the text (such as through a keyboard, handwriting recognition, etc.) when performing the batch operation, and the text will be directly used as the batch text. After the batch text is determined, the first device displays the batch text on the screen content. The display mode of the batch text can be set according to actual needs, such as being presented in the form of a floating window, a text box, a highlight display, etc. Meanwhile, the first device can also provide a function of adjusting the position, size, color, etc. of the batch text, so as to allow the user to make personalized settings as needed.

[0269] Exemplarily, the handwriting batch content is generated by capturing the pen trace points on the screen. The pen trace points contain the position information of the pen trace on the screen and are recorded in chronological order. This ensures the continuity and accuracy of the pen trace. When the user completes a piece of handwriting batch (i.e. does not perform the pen writing again within a certain time after lifting the pen, and the time can be set to 500-1000 milliseconds), the first device automatically triggers the recognition algorithm. The recognition algorithm converts the handwriting into a standard font by using the previously recorded pen trace point data. After the recognition is completed, the obtained standard font content is displayed on the screen. The user can choose to replace the handwriting font at the original position, or choose to display the standard font in a separate window.

[0270] In a possible implementation of the present application, before the target element is reproduced, the method further comprises: binding and storing the target element with the standard font corresponding to the batch content bound to the target element; and correspondingly, reproducing the batch content bound to the target element comprises: displaying the batch content bound to the target element and the standard font corresponding to the batch content in the case that the target element is reproduced.

[0271] For example, for the recognized standard font text, the standard font will be saved one by one corresponding to the page. Optionally, the first device provides a text search function, which allows the user to find the corresponding standard font text by inputting a keyword, which not only makes the display mode more diversified, but also better meets the user's use requirements.

[0272] For ease of understanding, the present application provides a scene schematic diagram of a batch information display method, as shown in FIG. 9.

[0273] Step 1: The screen casting end (i.e. the second device) plays the screen casting static page content (such as by playing a ppt).

[0274] Step 2: The screen casting end and the receiving end (i.e. the first device) are connected through an HDMI line, and the signal source window (display window) of the receiving end displays the screen casting content.

[0275] Step 3, the receiving end opens the annotation software, and a transparent or semi-transparent floating window (annotation page) is superimposed on the source window through the annotation software, the floating window can receive the touch event of the user to generate the handwriting content, and the handwriting content is displayed on the floating window. The receiving end can intercept the screen content displayed on the source window through the screenshot module.

[0276] Step 4, the receiving end sets the annotation function through the annotation function setting module (for example, annotation pen mode, handwriting color, handwriting thickness, etc.), and opens the page annotation function.

[0277] Step 5, the annotation module intercepts the picture content of the source window through the screenshot module at a fixed time, and the interception time can be 5 seconds.

[0278] Step 6, whether the page has been turned relative to the last screenshot is determined through the image comparison algorithm (comparison is performed once for each screenshot), wherein the image comparison algorithm can be deployed on the terminal side (the receiving end) or the cloud side.

[0279] Step 7, the first device can annotate the screen projection information content through the annotation software.

[0280] Step 8, for the page turning case, the screen projection content and the annotation content bound to the screen projection content are stored through the data storage module.

[0281] Step 9, when the user needs to share the annotation content, the saved screen projection content and the annotation content bound to the screen projection content can be used to generate a PDF file or the like for sharing, for example, sharing can be performed through an instant messaging application, a local area network, or other software forms. The related content of steps 1-9 can be referred to the annotation information display method shown in FIG. 2, and will not be described here.

[0282] In summary, the annotation information display method provided by the embodiment of the present application can realize the monitoring of the change of the screen projection content by identifying the reproduction of the target element through the first device, and can reproduce the annotation content bound to the target element when the reproduction of the target element in the screen projection content is identified, thereby reducing the resource processing amount of the second device compared with monitoring the change of the screen projection content through the second device. In addition, the monitoring through the first device improves the stability of monitoring the screen projection content compared with the monitoring through the second device. Furthermore, the first device and the second device are connected through the HDMI line, which is more stable than the wireless connection realized through software, and the development of the screen projection software is not required, thereby reducing the development cost.

[0283] Referring to FIG. 10, the embodiment of the present application provides a storage method of annotation information, which is applied to a first device, and the first device is connected with a second device; for example, the first device can be the display device 102 shown in FIG. 1, and the second device can be the computer device 101 shown in FIG. 1. Referring to FIG. 10, the method comprises steps 1001-1004.

[0284] In step 1001, the screen projection content sent by the second device is received.

[0285] The embodiment of the present application does not limit the screen projection content sent by the second device, and does not limit the time when the second device sends the screen projection content. For details, refer to the related description of step 201 in FIG. 2, which will not be repeated here.

[0286] In step 1002, the target element is determined based on the screen projection content.

[0287] In one possible implementation, the target element is determined based on the screen projection content, which comprises: identifying the page elements of the screen projection content, and determining at least one page element; and determining the target element from the at least one page element. The determination of the target element from the at least one page element includes but is not limited to determining the target element from the at least one page element based on the position of the annotation.

[0288] For details of the implementation of step 1002, refer to the related description of step 203 in FIG. 2, which will not be repeated here.

[0289] In step 1003, the annotation content bound to the target element is determined.

[0290] In one possible implementation, the annotation content bound to the target element is determined, which comprises: obtaining the first position information of the target element; and determining the annotation content bound to the target element based on the first position information of the target element.

[0291] In one possible implementation, the annotation content bound to the target element is determined based on the first position information of the target element, which comprises: obtaining the second position information of each annotation content; and determining the annotation content bound to the target element according to the first position information and the second position information.

[0292] For details of the implementation of step 1003, refer to the related description of step 203 in FIG. 2, which will not be repeated here.

[0293] In step 1004, the target element and the annotation content bound to the target element are stored.

[0294] The screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, different target elements are bound to different annotation content, and the target elements and the annotation content bound to the target elements are stored, including: storing the at least two target elements and the annotation content bound to the target elements respectively. The implementation of storing the target elements and the annotation content bound to the target elements can refer to the related description of step 203 in FIG. 2, which is not described here again.

[0295] In a possible implementation, after the target elements and the annotation content bound to the target elements are stored, the method further includes: obtaining a display position of the target element; and displaying the annotation content bound to the target element at a blank position around the display position in a case where the blank position exists. On the basis of determining the blank position, the first device displays the reproduced annotation content bound to the target element at the blank position, so that the annotation content does not block or interfere with the user from viewing the main screen projection content, and the user can conveniently view and refer to the annotation information. The process can refer to the related description of step 204 in FIG. 2, which is not described here again.

[0296] In the embodiment of the application, storing the target elements and the annotation content bound to the target elements further includes: binding and storing a standard font corresponding to the target element and the annotation content bound to the target element; and correspondingly, the method further includes: displaying the annotation content bound to the target element and the standard font corresponding to the annotation content. For example, the standard font corresponding to the target element and the annotation content bound to the target element is bound and stored, and through a text search function, the user is allowed to find the standard font corresponding to the annotation content by inputting a keyword, so that the display mode is more diversified, and is more in line with the use demand of the user. The process can refer to the related description of step 204 in FIG. 2, which is not described here again.

[0297] Exemplarily, storing the target elements and the annotation content bound to the target elements includes: storing the target elements and the annotation content bound to the target elements in a first folder; and correspondingly, the method further includes: displaying the annotation content bound to the target element, and storing the annotation content bound to the target element and newly generated annotation content in a second folder, the second folder being a subfolder of the first folder.

[0298] The embodiments of the present application do not limit the storage manner of the target element and the comment content bound by the target element in the first folder, and can ensure that the two are stored correspondingly, and the target element bound comment content can be quickly queried from the first folder subsequently. In addition, after the target element bound comment content is displayed, the target element bound comment content and the newly generated comment content are stored in the second folder, the reproduced target element bound comment content can be stored separately, and the second folder is a subfolder of the first folder, hierarchical storage is realized, and searching and classification are facilitated. For example, the target element and the bound comment content are stored in the hard disk or the temporary folder of the all-in-one machine, an independent folder is created under a specific directory after the page comment is opened and the reproduced comment content is displayed each time, the folder name is unique, and then all generated files are annotated in the folder and the subfolder, and hierarchical storage is realized.

[0299] It should be noted that the storage method of the comment information in FIG. 10 can be applied to the display method of the comment information in FIG. 2, and the corresponding embodiments in FIG. 10 and FIG. 12 can be referred to each other, and the same content will not be described again.

[0300] The method provided by the embodiments of the present application can store the target element and the comment content bound by the target element after the target element and the comment content bound by the target element are determined, so that after the target element is reproduced subsequently, the bound comment content can be reproduced quickly, and the display efficiency of the comment content is improved.

[0301] Referring to FIG. 11, FIG. 11 is a structural schematic diagram of a display device of comment information provided by an embodiment of the present application, as shown in FIG. 11, the device is applied to a first device, the first device is connected with a second device; the device comprises:

[0302] The receiving module 1101 is configured to receive the screen projection content sent by the second device.

[0303] The display module 1102 is configured to display the screen projection content, wherein the target element is contained in the screen projection content that has been displayed historically, and the target element is bound with the comment content.

[0304] The identification module 1103 is configured to identify reproduction of the target element based on the screen projection content, and the target element is bound with the comment content.

[0305] The display module 1102 is further configured to reproduce the comment content bound by the target element in a case where the target element is identified to be reproduced.

[0306] In a possible implementation manner, the identification module 1103 is further configured to determine the target element based on the screen projection content.

[0307] In a possible implementation, the identification module 1103 is configured to identify page elements of the screen mirroring content, determine at least one page element, and determine a target element from the at least one page element.

[0308] In a possible implementation, the identification module 1103 is configured to determine the target element from the at least one page element based on a position of the annotation.

[0309] In a possible implementation, the apparatus further includes:

[0310] The first storage module is configured to store the target element and annotation content bound to the target element.

[0311] In a possible implementation, the identification module 1103 is further configured to obtain first position information of the target element, and determine annotation content bound to the target element based on the first position information of the target element.

[0312] In a possible implementation, the identification module 1103 is configured to obtain second position information of each annotation content, and determine annotation content bound to the target element based on the first position information and the second position information.

[0313] In a possible implementation, the first storage module is further configured to store an original size of the target element.

[0314] The identification module 1103 is further configured to determine a display size of the reproduced target element and the original size of the target element, and determine a scaling ratio based on the display size and the original size of the target element.

[0315] The display module 1102 is configured to display the annotation content bound to the reproduced target element according to the scaling ratio.

[0316] In a possible implementation, the first storage module is further configured to store relative position information between the target element and the annotation content bound to the target element.

[0317] The identification module 1103 is further configured to determine an element position of the reproduced target element, and determine position information of the annotation content bound to the reproduced target element based on the element position of the target element and the relative position information.

[0318] The display module 1102 is configured to display the annotation content bound to the reproduced target element based on the position information of the annotation content bound to the reproduced target element.

[0319] In a possible implementation, the identification module 1103 is configured to determine a similarity between the element in the screen projection content and the stored target element, the similarity including at least one of a content similarity or a position similarity; and determine that the element in the screen projection content is the reproduced target element in a case where the similarity is greater than a similarity threshold.

[0320] In a possible implementation, the display module 1102 is configured to obtain display position information of the reproduced target element; and display the annotation content bound to the reproduced target element on a blank position in a case where it is determined according to the display position information that a position other than the reproduced target element is the blank position.

[0321] In a possible implementation, the screen in which the screen projection content is displayed includes a plurality of display windows, the plurality of display windows display different contents, and the display module 1102 is configured to display the screen projection content through a target display window in the plurality of display windows.

[0322] In a possible implementation, the display module 1102 is further configured to display a plurality of annotation pages on the displayed screen projection content in response to a creation of an annotation page operation, and any annotation page in the plurality of annotation pages is used to reproduce the annotation content bound to the target element on the any annotation page in a case where the target element is reproduced.

[0323] In a possible implementation, the display module 1102 is further configured to replace the annotation content displayed on the screen projection content based on a switching of the annotation page operation in response to the switching of the annotation page operation.

[0324] In a possible implementation, the first device and the second device are connected through a high-definition multimedia interface (HDMI) connection line.

[0325] In a possible implementation, the apparatus further includes:

[0326] The generation module is configured to generate an annotation file based on the reproduced target element and the annotation content corresponding to the reproduced target element, and the annotation file is used to share or display the reproduced target element and the annotation content corresponding to the reproduced target element.

[0327] In a possible implementation, the target element is an entire page of the screen projection content, and the display module 1102 is further configured to compare the screen projection content of a current page with stored screen projection content, the stored screen projection content has corresponding annotation content; and display the annotation content corresponding to the stored screen projection content in a case where the screen projection content of the current page is consistent with the stored screen projection content.

[0328] In a possible implementation, the apparatus further includes:

[0329] The creating module is configured to, in response to a trigger operation of opening the annotation, compare the screen projection content of the current page with the stored target element, and create a screen projection file for the current page in a case where the screen projection content of the current page is inconsistent with the stored target element, the created screen projection file being configured to store the annotation content of the current page.

[0330] In a possible implementation, the screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, and different target elements are bound to different annotation content.

[0331] In a possible implementation, the apparatus further includes:

[0332] The second storage module is configured to bind and store the target element and the standard font corresponding to the annotation content bound to the target element.

[0333] The display module 1102 is configured to display the annotation content bound to the target element and the standard font corresponding to the annotation content in a case where the reproduction of the target element is identified.

[0334] The apparatus provided in the embodiments of the present application can identify the reproduction of the target element by the first device, monitor the change of the screen projection content, reproduce the annotation content bound to the target element in a case where the reproduction of the target element in the screen projection content is identified, and reduce the resource processing amount of the second device compared with monitoring the change of the screen projection content by the second device. In addition, monitoring by the first device improves the stability of monitoring the screen projection content compared with monitoring by the second device. Furthermore, the first device and the second device are connected by the HDMI line, which is more stable than wireless connection realized by software, and the development of screen projection software is not required, thereby reducing the development cost.

[0335] Referring to FIG. 12, FIG. 12 is a structural schematic diagram of an annotation information storage apparatus provided in the embodiments of the present application, as shown in FIG. 12, the apparatus is applied to a first device, the first device is connected with a second device; the apparatus includes:

[0336] The receiving module 1201 is configured to receive the screen projection content sent by the second device.

[0337] The determining module 1202 is configured to determine the target element based on the screen projection content.

[0338] The determining module 1202 is further configured to determine the annotation content bound to the target element.

[0339] The storage module 1203 is configured to store the target element and the annotation content bound to the target element.

[0340] In a possible implementation, the determining module 1202 is configured to identify page elements of the screen projection content, and determine at least one page element; and determine a target element from the at least one page element.

[0341] In a possible implementation, the determining module 1202 is configured to determine the target element from the at least one page element based on a position of the annotation.

[0342] In a possible implementation, the determining module 1202 is configured to obtain first position information of the target element; and determine annotation content bound by the target element based on the first position information of the target element.

[0343] In a possible implementation, the determining module 1202 is configured to obtain second position information of each annotation content; and determine annotation content bound by the target element according to the first position information and the second position information.

[0344] In a possible implementation, the screen projection content includes a plurality of page elements, the plurality of page elements include at least two target elements, different target elements are bound to different annotation content, and the storage module 1203 is configured to store the at least two target elements and the bound annotation content respectively.

[0345] In a possible implementation, the apparatus further includes:

[0346] The obtaining module is configured to obtain a display position of the target element.

[0347] The first display module is configured to display the annotation content bound by the target element at a blank position in a case where the blank position exists around the display position.

[0348] In a possible implementation, the storage module 1203 is further configured to bind and store a standard font corresponding to the target element and the annotation content bound by the target element.

[0349] The apparatus further includes:

[0350] The second display module is configured to display the annotation content bound by the target element and the standard font corresponding to the annotation content.

[0351] In a possible implementation, the storage module 1203 is configured to store the target element and the annotation content bound by the target element in a first folder.

[0352] The apparatus further includes:

[0353] The third display module is configured to display the annotation content bound by the target element.

[0354] The storage module 1203 is further configured to store the target element and the comment content bound to the target element in a second folder, the second folder being a subfolder of the first folder.

[0355] The apparatus provided by the embodiments of the present application, after determining the target element and the comment content bound to the target element, stores the target element and the comment content bound to the target element, so that the bound comment content can be quickly reproduced after the target element is reproduced, and the display efficiency of the comment content is improved.

[0356] It should be noted that the display apparatus of the comment information provided by the embodiment of FIG. 11 and the storage apparatus of the comment information provided by FIG. 12 are only used as examples to divide the above-mentioned functional modules when implementing the functions, and in actual functions, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus provided by the above-mentioned embodiments and the corresponding method embodiments belong to the same concept, and the specific implementation process can be seen from the method embodiments.

[0357] FIG. 13 is a structural schematic diagram of a server provided by an embodiment of the present application. The server can be quite different due to different configurations or performances, and can include one or more processors 1301 and one or more memories 1302. At least one computer program is stored in the one or more memories 1302, and the at least one computer program is loaded and executed by the one or more processors 1301, so that the server implements the display method of the comment information provided by each of the above-mentioned method embodiments. Of course, the server can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for realizing the functions of the device, and here, details are not described.

[0358] FIG. 14 is a structural schematic diagram of a terminal provided by an embodiment of the present application, so that the terminal implements the display method of the comment information provided by each of the above-mentioned method embodiments. The terminal can be, for example, a smart phone, a tablet computer, a player, a notebook computer or a desktop computer. The terminal can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, and other names.

[0359] Generally, the terminal includes a processor 1401 and a memory 1402.

[0360] The processor 1401 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1401 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 1401 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 1401 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 1401 can further include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0361] The memory 1402 can include one or more computer-readable storage media that can be non-transitory. The memory 1402 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1402 is used to store at least one instruction for being executed by the processor 1401 to enable the terminal to implement the display method of annotation information or the storage method of annotation information provided by the method embodiments in the present application.

[0362] In some embodiments, the terminal can also optionally include a peripheral device interface 1403 and at least one peripheral device. The processor 1401, the memory 1402, and the peripheral device interface 1403 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1403 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1404, a display screen 1405, a camera assembly 1406, an audio circuit 1407, and a power supply 1408.

[0363] The peripheral interface 1403 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 1401 and the memory 1402. In some embodiments, the processor 1401, the memory 1402 and the peripheral interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1401, the memory 1402 and the peripheral interface 1403 can be implemented on a separate chip or circuit board, and the present embodiments are not limited in this regard.

[0364] The radio frequency circuit 1404 is configured to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1404 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 1404 converts electric signals into electromagnetic signals for transmission, or converts electromagnetic signals received into electric signals. Optionally, the radio frequency circuit 1404 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 1404 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to, a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1404 can also include NFC (Near Field Communication) related circuitry, and the present application is not limited in this regard.

[0365] The display screen 1405 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 1405 is a touch display screen, the display screen 1405 is further configured to capture touch signals on or above the surface of the display screen 1405. The touch signals can be input to the processor 1401 as control signals for processing. In this case, the display screen 1405 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 1405 can be one, disposed on the front panel of the terminal; in other embodiments, the display screen 1405 can be at least two, respectively disposed on different surfaces of the terminal or in a folding design; in other embodiments, the display screen 1405 can be a flexible display screen, disposed on a curved surface or a folding surface of the terminal. Even, the display screen 1405 can also be disposed in an irregular shape, i.e. a special-shaped screen. The display screen 1405 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0366] The camera assembly 1406 is configured to capture images or videos. Optionally, the camera assembly 1406 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 1406 can further include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0367] The audio circuit 1407 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 1401 for processing, or input to the radio frequency circuit 1404 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the terminal. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 1401 or the radio frequency circuit 1404 into sound waves. The speaker can be a traditional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 1407 can also include a headphone jack.

[0368] The power supply 1408 is used to supply power to each component in the terminal. The power supply 1408 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 1408 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0369] In some embodiments, the terminal also includes one or more sensors 1409. The one or more sensors 1409 include but are not limited to: an acceleration sensor 1410, a gyroscope sensor 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0370] The acceleration sensor 1410 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 1410 can be used to detect the components of gravitational acceleration in three coordinate axes. The processor 1401 can control the display screen 1405 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1410. The acceleration sensor 1410 can also be used for game or user motion data collection.

[0371] The gyroscope sensor 1411 can detect the body direction and rotation angle of the terminal. The gyroscope sensor 1411 can cooperate with the acceleration sensor 1410 to collect 3D actions of the user on the terminal. The processor 1401 can realize the following functions according to the data collected by the gyroscope sensor 1411: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0372] The pressure sensor 1412 can be disposed at a side frame of the terminal and / or under the display screen 1405. When the pressure sensor 1412 is disposed at the side frame of the terminal, a holding signal of the terminal by a user can be detected, and left-hand or right-hand recognition or a shortcut operation can be performed by the processor 1401 according to the holding signal collected by the pressure sensor 1412. When the pressure sensor 1412 is disposed under the display screen 1405, a pressure operation of the display screen 1405 by a user can be performed by the processor 1401 to control an operable control on a UI interface. The operable control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0373] The optical sensor 1413 is configured to collect ambient light intensity. In an embodiment, the processor 1401 can control display brightness of the display screen 1405 according to the ambient light intensity collected by the optical sensor 1413. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1405 is increased, and when the ambient light intensity is low, the display brightness of the display screen 1405 is decreased. In another embodiment, the processor 1401 can also dynamically adjust a shooting parameter of the camera assembly 1406 according to the ambient light intensity collected by the optical sensor 1413.

[0374] The proximity sensor 1414, also referred to as a distance sensor, is usually disposed at a front panel of the terminal. The proximity sensor 1414 is configured to collect a distance between a user and a front face of the terminal. In an embodiment, when the proximity sensor 1414 detects that the distance between the user and the front face of the terminal gradually decreases, the display screen 1405 is switched from a bright screen state to a dark screen state by the processor 1401, and when the proximity sensor 1414 detects that the distance between the user and the front face of the terminal gradually increases, the display screen 1405 is switched from the dark screen state to the bright screen state by the processor 1401.

[0375] Those skilled in the art can understand that the structure shown in FIG. 14 does not constitute a limitation on the terminal, and can include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0376] In an example embodiment, a computer device is also provided, which includes a processor and a memory having at least one computer program stored therein. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the above methods.

[0377] In an example embodiment, a computer readable storage medium is also provided, which has at least one computer program stored therein. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above methods.

[0378] In a possible implementation manner, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like. Alternatively, the computer readable storage medium can be a non-transitory computer readable storage medium.

[0379] In an example embodiment, a computer program product or computer program including computer instructions stored in a computer readable storage medium is also provided. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any of the above methods.

[0380] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the screen projection content involved in the present application is obtained under full authorization.

[0381] It should be understood that "multiple" referred to herein refers to two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0382] The above only describes example embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

A display method of comment information, characterized by The method is applied to a first device connected with a second device, and comprises: receiving screen projection content sent by the second device; displaying the screen projection content, wherein the screen projection content has a target element that has been displayed in history, and the target element is bound with comment content; identifying the reproduction of the target element based on the screen projection content; reproducing the comment content bound with the target element when the reproduction of the target element is identified. The method of claim 1, wherein Before the reproduction of the comment content bound with the target element, the method further comprises: determining the target element based on the screen projection content. The method according to claim 2, characterized in that The determination of the target element based on the screen projection content comprises: identifying page elements of the screen projection content to determine at least one page element; determining the target element from the at least one page element. The method according to claim 3, characterized in that The determination of the target element from the at least one page element comprises: determining the target element from the at least one page element based on the position of the comment. The method of claim 1, wherein Before the reproduction of the comment content bound with the target element, the method further comprises: storing the target element and the comment content bound with the target element. The method according to claim 5, characterized in that Before the storage of the target element and the comment content bound with the target element, the method further comprises: obtaining first position information of the target element; determining the comment content bound with the target element based on the first position information of the target element. The method according to claim 6, characterized in that The determination of the comment content bound with the target element based on the first position information of the target element comprises: obtaining second position information of each comment content; determining the comment content bound with the target element according to the first position information and the second position information. The method according to claim 5, characterized in that The original size of the target element is also stored, and before the reproduction of the comment content bound with the target element, the method further comprises: determining the display size of the reproduced target element and the original size of the target element; determining a scaling ratio based on the display size and the original size of the target element; The reproduction of the comment content bound with the target element comprises: displaying the reproduced comment content bound with the target element according to the scaling ratio. The method according to claim 5, characterized in that The relative position information between the target element and the comment content bound with the target element is also stored, and before the reproduction of the comment content bound with the target element, the method further comprises: determining the element position of the reproduced target element; determining the position information of the reproduced comment content bound with the target element based on the element position of the target element and the relative position information; The reproduction of the comment content bound with the target element comprises: displaying the reproduced comment content bound with the target element based on the position information of the reproduced comment content bound with the target element. The method according to claim 5, characterized in that The identification of the reproduction of the target element based on the screen projection content comprises: determining the similarity between the element in the screen projection content and the stored target element, wherein the similarity comprises at least one of content similarity or position similarity; when the similarity is greater than a similarity threshold, determining that the element in the screen projection content is the reproduced target element. The reproduction of the comment content bound with the target element comprises: The method of claim 1, wherein obtaining display position information of the reproduced target element; ​ In a case where a position other than the reproduced target element according to the display position information is a blank position, the reproduced target element is displayed in the blank position. The method according to any one of claims 1 to 11, characterized in that The screen displaying the screen projection content displays a plurality of display windows, and the plurality of display windows display different content. The displaying of the screen projection content includes: Displaying the screen projection content through a target display window in the plurality of display windows. The method according to any one of claims 1 to 11, characterized in that The method further includes: In response to a creation of a comment page operation, displaying a plurality of comment pages on the displayed screen projection content, and any comment page in the plurality of comment pages is used to reproduce the comment content bound to the target element in a case where the target element is identified to be reproduced. The method of claim 13, wherein The method further includes: In response to a switching of a comment page operation, replacing the comment content displayed on the screen projection content based on the switching of the comment page operation. The method according to any one of claims 1 to 11, characterized in that The first device and the second device are connected through a high-definition multimedia interface (HDMI) connection line. The method according to any one of claims 1 to 11, characterized in that The method further includes: Generating a comment file based on the reproduced target element and the comment content corresponding to the reproduced target element, and the comment file is used to share or display the reproduced target element and the comment content corresponding to the reproduced target element. The method according to any one of claims 1 to 11, characterized in that The target element is an entire page of the screen projection content, and the method further includes: Comparing the screen projection content of a current page with stored screen projection content, and the stored screen projection content has corresponding comment content; In a case where the screen projection content of the current page is consistent with the stored screen projection content, displaying the comment content corresponding to the stored screen projection content. The method according to any one of claims 1 to 11, characterized in that The method further includes: In response to a trigger operation of opening a comment, comparing the screen projection content of a current page with stored target elements; In a case where the screen projection content of the current page is inconsistent with the stored target elements, creating a screen projection file for the current page, and the created screen projection file is used to store the comment content of the current page. The method according to any one of claims 1 to 11, characterized in that The screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, and different target elements are bound to different comment content. The method according to any one of claims 1 to 11, characterized in that Before the reproducing of the comment content bound to the target element in a case where the target element is identified to be reproduced, the method further includes: Binding the target element to a standard font corresponding to the comment content bound to the target element and storing the same; The reproducing of the comment content bound to the target element in a case where the target element is identified to be reproduced includes: In a case where the target element is identified to be reproduced, displaying the comment content bound to the target element and the standard font corresponding to the comment content. A storage method of comment information, characterized by, The method is applied to a first device connected with a second device, and the method includes: Receiving screen projection content sent by the second device; Determining a target element based on the screen projection content; Determining comment content bound to the target element; Storing the target element and the comment content bound to the target element. The method of claim 21, wherein The determining of the target element based on the screen projection content includes: Identifying at least one page element from the screen projection content; Determining a target element from the at least one page element. The method of claim 22, wherein The determining the target element from the at least one page element comprises: The determining the target element from the at least one page element based on the position of the annotation. The method of claim 21, wherein The determining the annotation content bound by the target element comprises: Obtaining first position information of the target element; The determining the annotation content bound by the target element based on the first position information of the target element comprises: The method of claim 24, wherein Obtaining second position information of each annotation content; The determining the annotation content bound by the target element based on the first position information and the second position information comprises: Obtaining second position information of each annotation content; The method of claim 21, wherein The storing the target element and the annotation content bound by the target element comprises: Storing the at least two target elements and the annotation content bound by the target element respectively. The method according to any one of claims 21-26, characterized in that The storing the target element and the annotation content bound by the target element further comprises: Obtaining a display position of the target element; In a case that there is a blank position around the display position, displaying the annotation content bound by the target element in the blank position. The storing the target element and the annotation content bound by the target element further comprises: The method according to any one of claims 21-26, characterized in that Binding and storing a standard font corresponding to the target element and the annotation content bound by the target element. The method further comprises: Displaying the annotation content bound by the target element and the standard font corresponding to the annotation content. The storing the target element and the annotation content bound by the target element comprises: The method according to any one of claims 21-26, characterized in that Storing the target element and the annotation content bound by the target element in a first folder; The method further comprises: Displaying the annotation content bound by the target element, and storing the annotation content bound by the target element and newly generated annotation content in a second folder, the second folder being a subfolder of the first folder. The device is applied to a first device, the first device being connected with a second device; the device comprises: A display device of comment information characterized by A receiving module configured to receive screen projection content sent by the second device; A display module configured to display the screen projection content, wherein the screen projection content has been displayed historically and contains a target element, the target element being bound with annotation content; An identification module configured to identify reproduction of the target element based on the screen projection content, the target element being bound with annotation content; The display module is further configured to reproduce the annotation content bound by the target element in a case that the target element is identified to be reproduced. The identification module is further configured to determine the target element based on the screen projection content. The apparatus of claim 30, wherein The identification module is configured to identify page elements of the screen projection content, and determine at least one page element; and determine the target element from the at least one page element. The apparatus of claim 31, wherein The identification module is configured to determine the target element from the at least one page element based on the position of the annotation. The apparatus of claim 32, wherein The device further comprises: The apparatus of claim 30, wherein ​ The first storage module is configured to store the target element and the comment content bound to the target element. The apparatus of claim 34, wherein The identification module is further configured to acquire first position information of the target element, and determine the comment content bound to the target element based on the first position information of the target element. The apparatus of claim 35, wherein The identification module is configured to acquire second position information of each comment content, and determine the comment content bound to the target element according to the first position information and the second position information. The apparatus of claim 34, wherein The first storage module is further configured to store an original size of the target element. The identification module is further configured to determine a display size of the reproduced target element and the original size of the target element, and determine a scaling ratio based on the display size and the original size of the target element. The display module is configured to display the comment content bound to the reproduced target element according to the scaling ratio. The apparatus of claim 34, wherein The first storage module is further configured to store relative position information between the target element and the comment content bound to the target element. The identification module is further configured to determine an element position of the reproduced target element. The display module is configured to display the comment content bound to the reproduced target element based on the position information of the comment content bound to the reproduced target element. The identification module is configured to determine a similarity between an element in the screen projection content and the stored target element, the similarity including at least one of a content similarity or a position similarity, and determine that the element in the screen projection content is the reproduced target element when the similarity is greater than a similarity threshold. The apparatus of claim 34, wherein The display module is configured to acquire display position information of the reproduced target element, and display the comment content bound to the reproduced target element at a blank position when it is determined according to the display position information that a position other than the reproduced target element is the blank position. The apparatus of claim 30, wherein The display module is configured to display the screen projection content through a target display window in a plurality of display windows in a screen in which the screen projection content is displayed, the plurality of display windows displaying different contents. The apparatus of any of claims 30-40, wherein The display module is further configured to display a plurality of comment pages on the displayed screen projection content in response to a comment page creation operation, and reproduce the comment content bound to the target element on any comment page in the plurality of comment pages when the target element is reproduced. The apparatus of any of claims 30-40, wherein The display module is further configured to replace the comment content displayed on the screen projection content based on a comment page switching operation in response to the comment page switching operation. The apparatus of claim 42, wherein The first device and the second device are connected through a high-definition multimedia interface (HDMI) connection line. The apparatus of any of claims 30-40, wherein The apparatus further includes: A generation module configured to generate a comment file based on the reproduced target element and the comment content corresponding to the reproduced target element, the comment file being used to share or display the reproduced target element and the comment content corresponding to the reproduced target element. The apparatus of any of claims 30-40, wherein ​ ​ The apparatus of any of claims 30-40, wherein The target element is the whole page of the screen projection content, and the display module is further configured to compare the screen projection content of a current page with stored screen projection content, and the stored screen projection content has corresponding annotation content; and in a case where the screen projection content of the current page is consistent with the stored screen projection content, display the annotation content corresponding to the stored screen projection content. The apparatus of any of claims 30-40, wherein The device further includes: A creating module configured to, in response to a trigger operation of starting annotation, compare the screen projection content of a current page with stored target elements; and in a case where the screen projection content of the current page is inconsistent with the stored target elements, create a screen projection file for the current page, and the created screen projection file is configured to store annotation content of the current page. The apparatus of any of claims 30-40, wherein The screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, and different target elements are bound with different annotation content. The apparatus of any of claims 30-40, wherein The device further includes: A second storage module configured to bind and store the target elements with standard fonts corresponding to the annotation content bound with the target elements; The display module is configured to, in a case where the target elements are reproduced, display the annotation content bound with the target elements and the standard fonts corresponding to the annotation content. A storage device of comment information characterized by comprising: The device is applied to a first device, the first device is connected with a second device, and the device includes: A receiving module configured to receive screen projection content sent by the second device; A determining module configured to determine target elements based on the screen projection content; The determining module is further configured to determine annotation content bound with the target elements; A storage module configured to store the target elements and the annotation content bound with the target elements. The determining module is configured to identify page elements of the screen projection content, determine at least one page element, and determine target elements from the at least one page element. The apparatus of claim 50, wherein The determining module is configured to determine target elements from the at least one page element based on positions of the annotation. The apparatus of claim 51, wherein The determining module is configured to obtain first position information of the target elements, and determine annotation content bound with the target elements based on the first position information of the target elements. The apparatus of claim 50, wherein The determining module is configured to obtain second position information of each annotation content, and determine annotation content bound with the target elements according to the first position information and the second position information. The apparatus of claim 53, wherein The screen projection content includes a plurality of page elements, and the plurality of page elements include at least two target elements, and different target elements are bound with different annotation content, and the storage module is configured to store the at least two target elements and the bound annotation content respectively. The apparatus of claim 50, wherein The device further includes: The apparatus of any of claims 50-55, wherein An obtaining module configured to obtain a display position of the target elements; A first display module configured to, in a case where there is a blank position around the display position, display annotation content bound with the target elements in the blank position. The storage module is further configured to bind and store the target elements with standard fonts corresponding to the annotation content bound with the target elements; The device further includes: The apparatus of any of claims 50-55, wherein A second display module configured to display the annotation content bound with the target elements and the standard fonts corresponding to the annotation content. ​ ​ The apparatus of any of claims 50-55, wherein The storage module is configured to store the target element and the comment content bound to the target element in a first folder. The device further includes: A third display module configured to display the comment content bound to the target element. The storage module is further configured to store the comment content bound to the target element and the newly generated comment content in a second folder, which is a subfolder of the first folder. A computer device, characterized in that, The computer device includes a processor configured to load and execute at least one computer program to enable the computer device to implement the method of any one of claims 1 to 29. A computer-readable storage medium, characterized by, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor to enable the computer to implement the method of any one of claims 1 to 29. A computer program product, characterized by The computer program product includes computer program code, which is loaded and executed by the computer to enable the computer to implement the method of any one of claims 1 to 29.

Images