Annotation synchronization method, device and annotation synchronization method of teaching system

Abstract

The present invention discloses a method, device and annotation synchronization method for a teaching system. The annotation synchronization method includes the following steps: receiving a real-time screen video stream of a first terminal; playing the video stream on a base layer of the local terminal; reading the marking information of the video stream, wherein the marking information is information marked on the video stream by the first terminal based on the received annotation information; matching the annotation content on the transparent layer of the local terminal according to the marking information, and deleting the annotation content on the transparent layer of the local terminal based on preset rules when the marking information matches the annotation content on the current transparent layer of the local terminal, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by the local terminal. The method forms a feedback mechanism between the main speaker and each listening terminal without adding other response mechanisms, and simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening terminal.

Description

Technical Field

[0001] The present invention relates to the field of synchronous display of data, and more particularly to a method and device for synchronizing annotations and an annotation synchronization method for a teaching system. Background Art

[0002] With the development of information network technology, education has realized real-time or time-sharing distance learning with the help of information networks and related supporting technical equipment. In distance learning, electronic whiteboards can realize joint annotations between the main speaker and the audience, which greatly facilitates the interaction between the main speaker and the audience. However, in existing whiteboards, when the user at the audience makes annotations on the local device, the annotation handwriting needs to be sent to the remote main speaker device through the network for display, and then the annotation handwriting can be transmitted back to the local device together with the annotation handwriting displayed on the remote main speaker for presentation. In this process, since the user's annotation information needs to be transmitted through the network twice, the local display device displays the annotation handwriting significantly lags behind the user operation, which brings great inconvenience especially when the network status is poor.

[0003] In order to solve the technical problem of handwriting lag, the patent "Screen Annotation Method and System" with patent number ZL201710467197.3 proposed a solution: the local terminal is divided into a first canvas and a second canvas, the second canvas is used to receive screenshots of the remote main terminal, when the handwriting annotation is made on the local terminal, the handwriting annotation is first displayed on the first canvas, and the handwriting annotation is sent to the main terminal. After the main terminal displays the handwriting annotation, a screenshot of the main terminal is taken, and the screenshot is sent to the second canvas, and the handwriting annotation first displayed on the first canvas is deleted. However, this technical solution does not limit the timing of deleting handwriting. Deleting too early will result in blank handwriting for a period of time, and deleting too late will result in overlapping handwriting for a period of time. These situations will reduce the user experience. Summary of the Invention

[0004] The present invention aims to overcome at least one of the above-mentioned drawbacks of the prior art and provides a method, device and teaching system for synchronizing annotations. The technical solution adopted by the present invention is as follows.

[0005] In a first aspect, the present invention provides a method for synchronizing annotations, comprising the steps of:

[0006] Receiving a real-time screen video stream of the first terminal;

[0007] Play the video stream on the local end's bottom layer;

[0008] Reading marking information of the video stream, wherein the marking information is information marked on the video stream by the first terminal based on the received annotation information;

[0009] Match the annotation content on the transparent layer of this end according to the marking information. When the marking information matches the annotation content on the current transparent layer of this end, delete the annotation content on the transparent layer of this end based on preset rules, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by this end.

[0010] In one embodiment, the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, includes the steps of:

[0011] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0012] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted.

[0013] In one embodiment, the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, includes the steps of:

[0014] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0015] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted after a preset time period.

[0016] In a second aspect, the present invention provides a method for synchronizing annotations in a teaching system, comprising the steps of:

[0017] Outputting a real-time screen image video stream to a second terminal, wherein the real-time screen image video stream includes: a base layer and a transparent layer located above the base layer;

[0018] receiving annotation information sent by the second terminal;

[0019] Generate annotation content on the transparent layer of the local end according to the received annotation information;

[0020] Marking information is added to the output real-time screen picture video stream based on the received annotation information.

[0021] In one embodiment, the marking information includes part or all of the information in the received annotation information.

[0022] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0023] In a third aspect, the present invention provides a method for synchronizing annotations in a teaching system, the teaching system comprising: a lecture terminal and an auditory terminal, wherein both the lecture terminal and the auditory terminal comprise a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer; the method for synchronizing annotations comprises the following steps:

[0024] The speaker outputs real-time screen video stream to the listener;

[0025] The listening terminal plays the real-time screen image video stream on the base layer of the local terminal;

[0026] The listening end receives the annotation information;

[0027] The listening end generates annotation content on the transparent layer of the local end according to the received annotation information;

[0028] The listening end sends the received annotation information to the speaking end;

[0029] The main speaker generates annotation content on the transparent layer of the local end according to the received annotation information;

[0030] The main speaker adds marking information to the real-time screen video stream output to the listening end based on the received annotation information;

[0031] The listening terminal reads the marking information on the real-time screen video stream;

[0032] The listening terminal matches the annotation content on the local transparent layer according to the marking information, and deletes the annotation content on the local transparent layer based on preset rules when the marking information matches the annotation content on the current transparent layer of the local terminal.

[0033] In one embodiment, the listening terminal matches the annotation content on the local transparent layer according to the marking information, and when the marking information matches the annotation content on the current transparent layer of the local terminal, the process of deleting the annotation content on the local transparent layer based on a preset rule includes the steps of:

[0034] The listening end compares the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the listening end;

[0035] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal.

[0036] In one embodiment, the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, includes the steps of:

[0037] The listening end compares the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the listening end;

[0038] When the marking information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal after a preset time period.

[0039] In one embodiment, the marking information added by the speaker terminal includes part or all of the information in the annotation information received by the speaker terminal.

[0040] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0041] In a fourth aspect, the present invention provides a lecture and listening terminal with synchronized annotations, comprising:

[0042] A receiving module, configured to receive a real-time screen video stream of a first terminal;

[0043] A playback module, configured to play the video stream on the local end's base layer;

[0044] a reading module, configured to read the marking information of the video stream, wherein the marking information is information marked on the video stream by the first terminal based on the received annotation information;

[0045] A deletion module is used to match the annotation content on the transparent layer of this end according to the marking information. When the marking information matches the annotation content on the current transparent layer of this end, the annotation content on the transparent layer of this end is deleted based on preset rules, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by this end.

[0046] In a fifth aspect, the present invention provides a speaker terminal with synchronized annotations, comprising:

[0047] A video output module, configured to output a real-time screen image video stream to a second terminal, wherein the real-time screen image video stream includes: a base layer and a transparent layer located above the base layer;

[0048] A receiving module, configured to receive annotation information sent by the second terminal;

[0049] An annotation generation module, used to generate annotation content on the transparent layer of the local terminal according to the received annotation information;

[0050] The marking module is used to add marking information to the output real-time screen image video stream based on the received annotation information.

[0051] In a sixth aspect, the present invention provides a teaching system, comprising: a lecture terminal and an audience terminal, wherein each of the lecture terminal and the audience terminal comprises a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer; the audience terminal further comprises:

[0052] The playback module is used to receive the real-time screen video stream output by the main speaker terminal and play the real-time screen video stream on the background layer of the local terminal;

[0053] The first annotation generation module is used to receive annotation information and generate annotation content on the local transparent layer according to the received annotation information;

[0054] The sending module is used to send the received annotation information to the main speaker;

[0055] A tag reading module, configured to read tag information on the real-time screen video stream;

[0056] An annotation deletion module is used to match the annotation content on the local transparent layer according to the marking information, and when the marking information matches the annotation content on the current transparent layer of the local end, delete the annotation content on the local transparent layer based on a preset rule;

[0057] The main speaker also includes:

[0058] Video output module, used to output real-time screen video stream to the listening end;

[0059] The receiving module is used to receive the annotation information sent by the listening end;

[0060] The second annotation generation module is used to generate annotation content on the transparent layer of the local terminal according to the received annotation information;

[0061] The marking module is used to add marking information to the output real-time screen image video stream based on the received annotation information.

[0062] In a seventh aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of the above-mentioned embodiments when executing the program.

[0063] In an eighth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method of any of the above-mentioned embodiments when the program is executed by a processor.

[0064] In the present invention, marking information is added to the video stream sent from the main speaker to the listening end, and the listening end deletes the corresponding annotation content on the transparent layer of the local end according to the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the first terminal and each listening end, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening end. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of embodiment 1.

[0066] Figure 2 It is a schematic diagram of the transmission process of Example 1.

[0067] Figure 3 It is a flow chart of the second embodiment.

[0068] Figure 4 It is a flow chart of embodiment 3.

[0069] Figure 5 It is a structural diagram of embodiment 4.

[0070] Figure 6 It is a structural diagram of embodiment 5.

[0071] Figure 7 It is a structural diagram of embodiment 6. DETAILED DESCRIPTION

[0072] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0073] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first, second, ..." may interchangeably represent a specific order or precedence, where permitted. It should be understood that the objects distinguished by "first, second, ..." may be interchangeable, where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0074] Example 1

[0075] See Figure 1 , Figure 1This is a flow chart of a method for synchronizing annotations provided in the first embodiment of the present invention, which includes steps S110, S120, S130, and S140. It should be noted that steps S110, S120, S130, and S140 are merely reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps in the method in this embodiment.

[0076] Step S110: receiving a real-time screen video stream of the first terminal.

[0077] Step S120: Play the video stream on the local end's base layer.

[0078] Generally speaking, this method is used on the student end or the lecture end, so the local end in this method is the student end or the lecture end, and the first end is the teacher end or the lecture end. A lecture end is usually connected to one or more lecture ends. Figure 2 As shown, this end (listening end) has a transparent layer 11 and a base layer 12. The user performs annotation operations on his own listening end, such as drawing handwriting, inputting text, etc. The user's operations will eventually form data into annotation information, and the annotation information will generate annotation content on the transparent layer of the listening end used by the user. While each listening end generates annotation content on its own transparent layer, it will also send the annotation information to the first terminal (speaking end). The first terminal is equivalent to the "headquarters". The annotation information of all terminals, including itself, must be generated or erased by it to become the final annotation, so the first terminal aggregates the annotation information of all terminals together and generates or erases the annotation content. The first terminal can be provided with a transparent layer 21 and a base layer 22 like this end. The operations of the user of the first terminal itself will eventually generate annotation content on the transparent layer 21, and the annotation content of other terminals will also be generated and aggregated on its transparent layer 21. At the same time, the first terminal will output the real-time video stream ( Figure 2 The dotted line part in the image is sent to each listening terminal, and each listening terminal plays the real-time video stream on its own background layer. At this time, the user using the listening terminal can see the annotation content generated by all listening terminals and the main terminal.

[0079] Therefore, in the above two steps, the local end receives the real-time screen video stream of the first terminal and plays the video stream on the background layer of the local end, and this process runs through the beginning and end of the execution of the entire method.

[0080] Step S130 : reading the marking information of the video stream, wherein the marking information is information marked on the video stream by the first terminal based on the received annotation information.

[0081] When a listening terminal generates annotation content on its own transparent layer and the real-time screen video stream received from the first terminal also contains the annotation content, there are actually two layers of overlapping annotation content. At this time, the annotation content on its own transparent layer should be deleted to avoid overlapping display or erasure delays. However, there is no feedback mechanism between the first terminal and the various listening terminals. Each listening terminal cannot know whether the currently playing video stream contains its own annotation content, which results in it not knowing when to delete it. It is precisely to address this situation that the first terminal in the present method marks the video stream based on the received annotation information, allowing each terminal to determine whether the currently playing video stream contains its own annotation content based on the marking information. In this method, after receiving a piece of annotation information sent by one of the listening terminals, the first terminal generates annotation content on its transparent layer 21. While generating the annotation content, it also adds marking information to the image frame of the real-time screen video stream at that time to indicate that the first terminal has generated the annotation content corresponding to the annotation information at that moment.

[0082] It should be noted that, with respect to the time point for adding the marking information to the real-time screen video stream, those skilled in the art may select a reasonable time based on actual circumstances. For example, the marking information may be added upon receiving the annotation information, upon the first terminal completing the generation of the annotation content, or at some time after the first terminal completes the generation of the annotation content.

[0083] It should be pointed out here that the first terminal can send a video stream with the marking information of each terminal added uniformly, and the video stream sent to each terminal is the same; or only add marking information to the video stream of the listening end corresponding to the annotation content, and the video stream sent to each terminal is different.

[0084] By adding a marker to the video stream, the first terminal can clearly and intuitively let the listener know the frame from which the annotation content appears. If other feedback mechanisms are used between the first terminal and the listener, the feedback information may arrive before the video stream. Therefore, additional communication messages are required to let the listener know the frame from which the annotation content appears, and to synchronize the response mechanism with the video stream, which is a very cumbersome process.

[0085] As previously mentioned, in the video stream sent by the first terminal to each listening terminal, some image frames will be added with marking information. In this step, while playing the real-time screen image video stream, the local terminal also checks whether the image frame currently being played carries marking information, and if so, reads the marking information.

[0086] Step S140, matching the annotation content on the transparent layer of this end according to the marking information, and when the marking information matches the annotation content on the current transparent layer of this end, deleting the annotation content on the transparent layer of this end based on preset rules, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by this end.

[0087] Since the video stream may carry tag information that is fed back to other listening terminals, or there may be multiple annotations on this listening terminal that need to be deleted, after reading the tag information, it is necessary to compare the tag information with the annotation content on the transparent layer of this terminal. After the comparison is completed and a matching result is obtained, the target annotation content to be deleted is determined based on the matching result, and then the corresponding annotation content on the transparent layer of this terminal is deleted based on the preset rules. The preset rule can be to delete immediately after the comparison is completed, or to delete after a preset period of time (such as 0.5 seconds) after the comparison is completed.

[0088] It should be pointed out here that if the first terminal sends a video stream uniformly adding tag information of each terminal, the matching results of some listening terminals may be empty. At this time, these listening terminals do not need to delete the annotation content.

[0089] In this method, marking information is added to the video stream sent from the main speaker to the listening end, and the listening end deletes the corresponding annotation content on the transparent layer of the local end according to the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the first terminal and each listening end, and the problem of asynchrony between the video and the response mechanism is avoided. The problem of the timing of deleting repeated handwriting on the listening end is simply and quickly solved.

[0090] In one embodiment, the process of matching the annotation content on the transparent layer of the local end according to the marking information in step S140, and deleting the annotation content on the transparent layer of the local end based on preset rules when the marking information matches the annotation content on the current transparent layer of the local end, includes steps S1411 and S1412.

[0091] Step S1411 , comparing the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0092] Step S1412: when the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted.

[0093] Since the marking information is the information that the first terminal marks on the video stream based on the received annotation information, and the annotation content on the current transparent layer of the local terminal is also generated based on the annotation information, the two can be compared. In addition, since the annotation information corresponding to the annotation content on the current transparent layer of the local terminal indicates which annotations exist on the current transparent layer of the local terminal, and the marking information indicates the annotations newly generated by the first terminal at a certain moment, by comparing the two, it can be determined whether the annotations on the current transparent layer of the local terminal have already been generated on the first terminal.

[0094] When the marking information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, it proves that the annotation on the current transparent layer of the local terminal has been generated on the first terminal, which means that the annotation content can be deleted.

[0095] In one embodiment, the process of matching the annotation content on the transparent layer of the local end according to the marking information in step S140, and deleting the annotation content on the transparent layer of the local end based on preset rules when the marking information matches the annotation content on the current transparent layer of the local end, includes steps S1421 and S1422.

[0096] Step S1421 , comparing the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0097] Step S1422: when the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted after a preset time period.

[0098] This embodiment is essentially the same as the previous one, differing only in the timing of deleting the annotations. To guarantee that the annotations are included in the video stream played on the local background layer when the annotations are deleted, the deletion time can be slightly delayed. A preset delay time (e.g., 0.5-1 second) is used, and the deletion is performed after the preset delay time has elapsed.

[0099] Example 2

[0100] See Figure 3 , Figure 3 This is a flow chart of a method for synchronizing annotations provided in the second embodiment of the present invention, which includes steps S310, S320, S330, and S340. It should be noted that steps S310, S320, S330, and S340 are merely reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 2 The corresponding relationship does not limit the order of the steps in the method in this embodiment.

[0101] Step S310: outputting a real-time screen image video stream to a second terminal; wherein the real-time screen image video stream includes: a base layer and a transparent layer located above the base layer.

[0102] Generally speaking, this method is used on the teacher's end or the main lecturer's end, so the local end in this method is the teacher's end or the main lecturer's end, and the second end is the student's end or the listening end. A main lecturer's end is usually connected to one or more listening ends.

[0103] As described in the first embodiment, the main speaker terminal sends its real-time screen video stream to each listening terminal at all times. The video stream is a composite video stream including a base layer and a transparent layer.

[0104] Step S320: receiving annotation information sent by the second terminal.

[0105] Step S330: Generate annotation content on the transparent layer of the local terminal according to the received annotation information.

[0106] As described in the first embodiment, after receiving the annotation information sent from different second terminals (ie, the listening terminals), the main speaker terminal generates annotation content on its own transparent layer according to the annotation information.

[0107] Step S340: adding marking information to the output real-time screen video stream based on the received annotation information.

[0108] As described in the first embodiment, the main speaker marks the video stream based on the received annotation information, and each terminal determines whether the currently played video stream contains its own annotation content based on the mark information.

[0109] This method adds marking information to the video stream sent by the main speaker to the listening end. The listening end deletes the corresponding annotation content on the transparent layer of the local end according to the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the main speaker and each listening end, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening end.

[0110] In one embodiment, the marking information includes part or all of the information in the received annotation information.

[0111] The speaker directly extracts some or all of the annotation information as markup information. This ensures that the content of the markup information is identical to the attendee's own annotation information, eliminating the need for further processing. This facilitates the attendee's ability to identify and determine whether to delete their own annotations. Annotation information typically includes the element type, location coordinates, color, line thickness, and / or font size, so the markup information can also include the aforementioned annotation information.

[0112] It should be noted here that the speaker can directly add part or all of the information in the annotation information to the mark information without adding the additional information added by this end to the mark information. It can also directly add part or all of the information in the annotation information to the mark information and then add the additional information added by this end.

[0113] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0114] In this embodiment, all the annotation information is directly added as all the contents of the mark information. This is conducive to simplifying the adding process and also conducive to simplifying the comparison process of the listening end.

[0115] Example 3

[0116] like Figure 4 As shown, the present invention also discloses a method for synchronizing annotations in a teaching system, wherein the teaching system includes: a lecture terminal and a lecture terminal; wherein the lecture terminal and the lecture terminal both include a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer; the method for synchronizing annotations includes the steps of:

[0117] Step S410: The speaker outputs a real-time screen video stream to the listener.

[0118] Step S420: the listening terminal plays the real-time screen video stream on the base layer of the listening terminal;

[0119] Step S430: the listening terminal receives the annotation information;

[0120] Step S440: The listening terminal generates annotation content on the transparent layer of the listening terminal according to the received annotation information;

[0121] Step S450: the listening terminal sends the received annotation information to the speaker terminal;

[0122] Step S460: the speaker generates annotation content on the transparent layer of the speaker according to the received annotation information;

[0123] Step S470: the speaker adds marking information to the real-time screen video stream output to the listener based on the received annotation information;

[0124] Step S480: The listening terminal reads the marking information on the real-time screen video stream;

[0125] In step S490, the listening terminal matches the annotation content on the local transparent layer according to the marking information, and deletes the annotation content on the local transparent layer based on preset rules when the marking information matches the annotation content on the current transparent layer of the local terminal.

[0126] A teaching system using this method includes a lecturer and an audience, where there can be one or more audiences. Both the lecturer and audience include a base layer and a transparent layer, with the transparent layer on each end positioned above the base layer. The audience continuously and synchronously receives the lecturer's real-time screen video stream and plays it on its own base layer. Specifically, the audience continuously plays a composite image of the lecturer's transparent layer and the base layer. While playing the composite image on the base layer, the audience also receives annotations from the user operating the local terminal and generates corresponding annotation content on its own transparent layer. Simultaneously, the audience also sends the annotation information to the lecturer, instructing the lecturer to generate the corresponding annotation content on its own transparent layer. After receiving the annotation information, the lecturer provides feedback to the audience by adding marking information to the real-time screen video stream. After reading the marking information, the audience deletes the corresponding annotation content on its own transparent layer based on preset rules, so that duplicate annotation content is no longer displayed on its own terminal.

[0127] This method adds marking information to the video stream sent by the main speaker to the listening end. The listening end deletes the corresponding annotation content on the transparent layer of the local end according to the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the main speaker and each listening end, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening end.

[0128] In one embodiment, the listening terminal matches the annotation content on the local transparent layer according to the marking information, and when the marking information matches the annotation content on the current transparent layer of the local terminal, the process of deleting the annotation content on the local transparent layer based on a preset rule includes the steps of:

[0129] The listening end compares the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the listening end;

[0130] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal.

[0131] In one embodiment, the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, includes the steps of:

[0132] The listening end compares the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the listening end;

[0133] When the marking information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal after a preset time period.

[0134] In one embodiment, the marking information added by the speaker terminal includes part or all of the information in the annotation information received by the speaker terminal.

[0135] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0136] Example 4

[0137] Corresponding to the first embodiment, this embodiment also provides a lecture terminal with synchronized annotations, such as Figure 5 As shown, the annotation synchronization terminal includes: a receiving module 501, a playing module 502, a reading module 503 and a deleting module 504.

[0138] Receiving module 501, configured to receive a real-time screen video stream of a first terminal;

[0139] Playing module 502, used for playing the video stream on the bottom layer of the local terminal;

[0140] a reading module 503, configured to read the marking information of the video stream, wherein the marking information is information marked on the video stream by the first terminal based on the received annotation information;

[0141] The deletion module 504 is used to match the annotation content on the transparent layer of this end according to the marking information. When the marking information matches the annotation content on the current transparent layer of this end, the annotation content on the transparent layer of this end is deleted based on preset rules, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by this end.

[0142] This annotation synchronization device adds marking information to the video stream sent from the main speaker to the listening terminal. The listening terminal deletes the corresponding annotation content on the transparent layer of its own terminal based on the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the first terminal and each listening terminal, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening terminal.

[0143] In one embodiment, the deletion module performs the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, including the steps of:

[0144] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0145] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted.

[0146] In one embodiment, the deletion module performs the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on a preset rule when the marking information matches the annotation content on the current transparent layer of the local end, including the steps of:

[0147] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0148] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted after a preset time period.

[0149] Example 5

[0150] Corresponding to the second embodiment, this embodiment also provides a speaker terminal with synchronous annotations, such as Figure 6 As shown, the annotation synchronization main speaker terminal includes: a video output module 601, a receiving module 602, an annotation generation module 603 and a marking module 604.

[0151] The video output module 601 is configured to output a real-time screen video stream to a second terminal, wherein the real-time screen video stream includes a base layer and a transparent layer above the base layer;

[0152] Receiving module 602, configured to receive annotation information sent by the second terminal;

[0153] Annotation generating module 603, used to generate annotation content on the transparent layer of the local terminal according to the received annotation information;

[0154] The marking module 604 is configured to add marking information to the output real-time screen image video stream based on the received annotation information.

[0155] This method adds marking information to the video stream sent by the main speaker to the listening end. The listening end deletes the corresponding annotation content on the transparent layer of the local end according to the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the main speaker and each listening end, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening end.

[0156] In one embodiment, the marking information includes part or all of the information in the received annotation information.

[0157] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0158] Example 6

[0159] Corresponding to the second embodiment, Figure 7 As shown, the present invention also discloses a teaching system, wherein the teaching system includes: a lecture terminal and a lecture terminal; wherein the lecture terminal and the lecture terminal each include a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer;

[0160] The listening terminal also includes:

[0161] The playback module is used to receive the real-time screen video stream output by the main speaker terminal and play the real-time screen video stream on the background layer of the local terminal;

[0162] The first annotation generation module is used to receive annotation information and generate annotation content on the local transparent layer according to the received annotation information;

[0163] The sending module is used to send the received annotation information to the main speaker;

[0164] A tag reading module, configured to read tag information on the real-time screen video stream;

[0165] An annotation deletion module is used to match the annotation content on the local transparent layer according to the marking information, and when the marking information matches the annotation content on the current transparent layer of the local end, delete the annotation content on the local transparent layer based on a preset rule;

[0166] The main speaker also includes:

[0167] Video output module, used to output real-time screen video stream to the listening end;

[0168] The receiving module is used to receive the annotation information sent by the listening end;

[0169] The second annotation generation module is used to generate annotation content on the transparent layer of the local terminal according to the received annotation information;

[0170] The marking module is used to add marking information to the output real-time screen image video stream based on the received annotation information.

[0171] This teaching system adds marking information to the video stream sent from the lecturer to the audience. The audience deletes the corresponding annotation content on the transparent layer of its own end based on the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the lecturer and each audience, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the audience.

[0172] In one embodiment, the annotation deletion module at the lecture end performs the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on preset rules when the marking information matches the annotation content on the current transparent layer at the local end, including the steps of:

[0173] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0174] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted.

[0175] In one embodiment, the annotation deletion module at the lecture end performs the process of matching the annotation content on the local transparent layer according to the marking information, and deleting the annotation content on the local transparent layer based on preset rules when the marking information matches the annotation content on the current transparent layer at the local end, including the steps of:

[0176] Compare the mark information with the annotation information corresponding to the annotation content on the current transparent layer of the local end;

[0177] When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted after a preset time period.

[0178] In one embodiment, the marking information added by the speaker terminal includes part or all of the information in the annotation information received by the speaker terminal.

[0179] In one implementation, the marking information added by the speaker terminal is annotation information received by the speaker terminal.

[0180] Example 7

[0181] An embodiment of the present invention further provides a storage medium storing computer instructions, which, when executed by a processor, implements the annotation synchronization method of any of the above embodiments or the annotation synchronization method of a teaching system.

[0182] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0183] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0184] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, an encoder, and a computer program stored on the memory and runnable on the encoder, wherein when the encoder executes the program, it implements any one of the annotation synchronization methods or annotation synchronization methods of the teaching system in the above-mentioned embodiments.

[0185] The above-mentioned computer device adds marking information to the video stream sent from the main speaker to the listening end. The listening end deletes the corresponding annotation content on the transparent layer of its own end based on the marking information. Without adding other response mechanisms, a feedback mechanism is formed between the main speaker and each listening end, which simply and quickly solves the problem of the timing of deleting repeated handwriting on the listening end.

[0186] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0187] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for synchronizing annotations, characterized in that: The method is applied to a listening terminal, which is connected to a first terminal capable of connecting to multiple listening terminals, and includes the following steps: Receiving a real-time screen video stream of the first terminal; Play the video stream on the local end's bottom layer; While playing the video stream, checking whether the currently played image frame carries marking information, wherein only some of the image frames in the video stream will be added with the marking information, and the marking information is marked on the image frame corresponding to the time point when the first terminal generates the annotation content, the marking information is based on the received annotation information by the first terminal, and the annotation information is used to generate the annotation content on the transparent layer, and the annotation information includes: the type of annotation element, location coordinates, color, line thickness value and / or text font size, and the first terminal receives the annotation information of all terminals including itself; When it is read that the currently played image frame has marking information, the marking information is read, and the marking information is compared with the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, wherein the transparent layer is above the base layer, and the annotation content is generated based on the annotation information received by the local terminal; When the marking information matches the annotation information on the current transparent layer of the local end, deleting the annotation content on the transparent layer of the local end based on a preset rule; When the mark information does not match the annotation information on the current transparent layer of the local end, the annotation content is not deleted.

2. The method for synchronizing annotations according to claim 1, wherein: The process of deleting the annotation content on the current transparent layer of the local end based on a preset rule when the mark information matches the annotation information on the current transparent layer of the local end includes the steps of: When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted.

3. The method for synchronizing annotations according to claim 1, wherein: The process of deleting the annotation content on the current transparent layer of the local end based on a preset rule when the mark information matches the annotation information on the current transparent layer of the local end includes the steps of: When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local end, the corresponding annotation content on the transparent layer of the local end is deleted after a preset time period.

4. A method for synchronizing annotations, characterized in that: The method is applied to a main speaker terminal, the main speaker terminal is connected to a second terminal, and the main speaker terminal can be connected to multiple second terminals, including the steps of: Outputting a real-time screen image video stream to a second terminal, wherein the real-time screen image video stream includes: a base layer and a transparent layer located above the base layer; receiving annotation information from all terminals including itself, wherein the annotation information is used to generate annotation content on the transparent layer; Generate annotation content on the transparent layer of the local terminal according to the received annotation information, wherein the annotation information includes: the type of annotation element, position coordinates, color, line thickness and / or text font size; Based on the received annotation information, marking information is added only to the image frame of the real-time screen video stream corresponding to the time point of generating the annotation content, wherein the added marking information is the annotation information received by the speaker end.

5. A method for synchronizing annotations in a teaching system, the teaching system comprising: A speaker terminal and a listener terminal, wherein the speaker terminal can be connected to multiple listener terminals, wherein both the speaker terminal and the listener terminal include a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer; characterized in that the annotation synchronization method includes the steps of: The speaker outputs real-time screen video stream to the listener; The listening terminal plays the real-time screen image video stream on the base layer of the local terminal; The listening end receives the annotation information, wherein the annotation information is used to generate annotation content on the transparent layer; The listening end generates annotation content on the transparent layer of the local end according to the received annotation information; The listening end sends the received annotation information to the speaking end; The main speaker receives annotation information from all terminals including itself; The main speaker generates annotation content on the transparent layer of the local end according to the received annotation information; Based on the received annotation information, the speaker adds marking information only to the image frame of the real-time screen video stream corresponding to the time point when the annotation content is generated, wherein the added marking information is the annotation information received by the speaker, and the annotation information includes: the element type, position coordinates, color, line thickness and / or text font size of the annotation; While playing the video stream, the listening terminal also checks whether the currently played image frame carries marking information. If the currently played image frame has marking information, the listening terminal reads the marking information and compares the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the listening terminal. When the marking information matches the annotation information on the current transparent layer of the local terminal, the listening terminal deletes the annotation content on the transparent layer of the local terminal based on the preset rules; When the marking information does not match the annotation information on the current transparent layer of the local terminal, the listening terminal does not delete the annotation content.

6. The annotation synchronization method of the teaching system according to claim 5, characterized in that: The process of deleting the annotation content on the transparent layer of the local terminal based on a preset rule when the mark information matches the annotation information on the current transparent layer of the local terminal includes the following steps: When the mark information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal.

7. The annotation synchronization method of the teaching system according to claim 5, characterized in that: The process of deleting the annotation content on the transparent layer of the local terminal based on a preset rule when the mark information matches the annotation information on the current transparent layer of the local terminal includes the following steps: When the marking information matches the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, the listening terminal deletes the corresponding annotation content on the transparent layer of the local terminal after a preset time period.

8. A lecture terminal with synchronized annotations, characterized in that: The listening terminal is connected to a first terminal, and the first terminal can be connected to multiple listening terminals, including: A receiving module, configured to receive a real-time screen video stream of a first terminal; A playback module, configured to play the video stream on the local end's base layer; a reading module, configured to check whether the currently played image frame carries marking information while playing the video stream; and when it is read that the currently played image frame has marking information, read the marking information, and compare the marking information with the annotation information corresponding to the annotation content on the current transparent layer of the local terminal, wherein the marking information is added to only some of the image frames in the video stream, and the marking information is marked on the image frame corresponding to the time point when the first terminal generates the annotation content. The marking information is based on the received annotation information by the first terminal, and the annotation information is used to generate the annotation content on the transparent layer. The annotation information includes: the type of annotation element, position coordinates, color, line thickness and / or text font size, and the first terminal receives the annotation information of all terminals including itself; a deletion module configured to delete the annotation content on the current transparent layer of the local end based on a preset rule when the marking information matches the annotation information on the current transparent layer of the local end, wherein the transparent layer is above the base layer and the annotation content is generated based on the annotation information received by the local end; The deletion module is further configured to not delete the annotation content when the mark information does not match the annotation content on the current transparent layer of the local end.

9. A speaker terminal with synchronized annotations, characterized in that: The main speaker terminal can be connected to multiple second terminals, including: A video output module, configured to output a real-time screen image video stream to a second terminal, wherein the real-time screen image video stream includes: a base layer and a transparent layer located above the base layer; a receiving module, configured to receive annotation information from all terminals including itself, wherein the annotation information is used to generate annotation content on the transparent layer, and the annotation information includes: annotation element type, position coordinates, color, line thickness and / or text font size; An annotation generation module, used to generate annotation content on the transparent layer of the local terminal according to the received annotation information; The marking module is used to add marking information only to the image frame of the real-time screen video stream corresponding to the time point of generating the annotation content based on the received annotation information, wherein the added marking information is the annotation information received by the speaker end.

10. A teaching system, comprising: A main speaker terminal and a listening terminal, wherein the main speaker terminal can be connected to multiple listening terminals, wherein both the main speaker terminal and the listening terminal include a base layer and a transparent layer, and the transparent layer of each terminal is located above the base layer; characterized in that the listening terminal also includes: The playback module is used to receive the real-time screen video stream output by the main speaker terminal and play the real-time screen video stream on the background layer of the local terminal; a first annotation generation module, configured to receive annotation information and generate annotation content on the local transparent layer based on the received annotation information, wherein the annotation information is used to generate the annotation content on the transparent layer, and the annotation information includes: the type of annotation element, location coordinates, color, line thickness and / or text font size; The sending module is used to send the received annotation information to the main speaker; a tag reading module, configured to check whether the currently played image frame carries tag information while playing the video stream, and if the currently played image frame has tag information, read the tag information and compare the tag information with the annotation information corresponding to the annotation content on the current transparent layer of the local end; An annotation deletion module is used to match the annotation content on the local transparent layer according to the marking information, and when the marking information matches the annotation information on the current transparent layer of the local end, delete the annotation content on the local transparent layer based on a preset rule; The annotation deletion module is further configured to not delete the annotation content when the mark information does not match the annotation information on the current transparent layer of the local end; The main speaker also includes: Video output module, used to output real-time screen video stream to the listening end; A receiving module, used to receive annotation information from all terminals including itself; The second annotation generation module is used to generate annotation content on the transparent layer of the local terminal according to the received annotation information; The marking module is used to add marking information only to the image frame of the real-time screen video stream corresponding to the time point of generating the annotation content based on the received annotation information, wherein the added marking information is the annotation information received by the speaker end.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images