Label content storage method, device and system and computer readable storage medium

Abstract

The invention discloses an annotation content storage method, device and system and a computer readable storage medium, and relates to the technical field of computers. The method comprises the steps that track point coordinates of a tagged background picture and tagged handwriting are obtained, the tagged background picture is generated based on a tagged page in a tagged file, and the tagged handwriting is generated according to tagged content of the tagged page; and according to the annotation background picture, adding the annotation handwriting into the annotated page according to the track point coordinates. According to the method, the annotation handwriting is added into the annotated page according to the track point coordinates, so that the annotation handwriting can be automatically added into the annotated page, and the annotation content can be stored in the annotated page. Compared with a mode of manually adding the annotation handwriting to store the annotation content, the method has the advantage that the efficiency and the accuracy of storing the annotation content are higher.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, system and computer-readable storage medium for storing annotation content. Background Art

[0002] In scenarios such as corporate meetings, electronic whiteboards are gradually being used to replace ordinary whiteboards for content discussion. Electronic whiteboards can provide screen projection and annotation functions. For example, a person participating in a meeting uses the screen projection function to send the file to be shared from the terminal used by the user to the electronic whiteboard, and the electronic whiteboard displays the file; then, any person participating in the meeting uses the annotation function to manually add annotation content to the file on the electronic whiteboard for meeting discussion. Therefore, a method for saving annotation content is needed, which can save the annotation content so that the content discussed in the meeting can be reviewed later.

[0003] In the related art, the electronic whiteboard sends the annotation content to the terminal in the form of a picture, and the user adds the annotation content to the above-mentioned shared file based on the picture, so that the annotation content is saved in the shared file. However, the method of manually adding annotation content by the user to save the annotation content is inefficient, and when there are a lot of annotation contents, the user is prone to make mistakes when adding annotation content, resulting in low accuracy of the saved annotation content. Summary of the invention

[0004] The present application proposes a method, device, system and computer-readable storage medium for storing annotation content, which are used to improve the efficiency and accuracy of storing annotation content in annotated files.

[0005] In a first aspect, a method for saving annotation content is provided, which is applied to an electronic whiteboard, and the method includes: obtaining an annotation background image and trajectory point coordinates of annotation handwriting, wherein the annotation background image is generated based on an annotated page in an annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; and according to the annotation background image, the annotation handwriting is added to the annotated page according to the trajectory point coordinates.

[0006] In this method, the electronic whiteboard can automatically add the annotation handwriting to the annotated page by adding the annotation handwriting to the annotated page according to the coordinates of the track points, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, this method has higher efficiency and accuracy in saving the annotation content.

[0007] In a possible implementation, the marked handwriting includes at least one of graphic handwriting or text handwriting, and both the graphic handwriting and the text handwriting include at least one track point. The method can be applied to the case where the marked handwriting includes multiple types of handwriting, and has a wide range of applications.

[0008] In a possible implementation, the marked handwriting includes multiple strokes, and the method further includes: inputting the multiple strokes included in the marked handwriting into a hybrid image-text classifier, the hybrid image-text classifier is used to classify the strokes; obtaining a classification result output by the hybrid image-text classifier, the classification result includes at least one of a graphic stroke or a text stroke; when the classification result includes a graphic stroke, obtaining a graphic handwriting based on the graphic stroke; when the classification result includes a text stroke, obtaining a text handwriting based on the text stroke. This method can be applied to the case where the marked handwriting includes both graphic strokes and text strokes, and has a wide range of applications.

[0009] In a possible implementation, obtaining graphic handwriting according to graphic strokes includes: performing graphic recognition on the graphic strokes, obtaining graphic handwriting according to the obtained graphic recognition results, and the graphic handwriting includes at least one graphic; obtaining text handwriting according to text strokes includes: performing text recognition on the text strokes, obtaining text handwriting according to the obtained text recognition results, and the text handwriting includes at least one text. By recognizing the graphic strokes and text strokes, the method can add the annotation handwriting to the annotation page in the form of standardized graphics and text, thereby eliminating the need for the user to manually convert the graphic handwriting and text handwriting into standardized graphics and text respectively, and providing a good user experience.

[0010] In a possible implementation, when the annotation handwriting includes graphic handwriting and text handwriting, before adding the annotation handwriting to the annotated page according to the coordinates of the track points based on the annotation background image, it also includes: obtaining the position and size of at least one graphic included in the graphic handwriting, and the position and size of at least one text included in the text handwriting; obtaining the first graphic and the first text based on the position and size of at least one graphic and the position and size of at least one text; adjusting at least one of the position or size of the first graphic to obtain the adjusted first graphic, and / or adjusting at least one of the position or size of the first text to obtain the adjusted first text. By adjusting at least one of the first graphic or the first text, it is possible to avoid the situation where the first graphic and the first text appear to overlap and affect reading, and improve the display effect of the subsequent display of the first graphic and the first text.

[0011] In a possible implementation, when the annotation handwriting includes graphic handwriting and the graphic handwriting includes multiple graphics, before the annotation handwriting is added to the annotated page according to the annotation background image and the coordinates of the track points, it also includes: obtaining the positions and sizes of the multiple graphics included in the graphic handwriting; obtaining the second graphic and the third graphic according to the positions of the multiple graphics; adjusting at least one of the position or size of the second graphic to obtain the adjusted second graphic, and / or adjusting at least one of the position or size of the third graphic to obtain the adjusted third graphic. By adjusting at least one of the second graphic or the third graphic, the display position and display size of the second graphic and the third graphic can be adjusted, so that the display of the second graphic and the third graphic better meets the needs.

[0012] In a possible implementation, according to the annotation background image, the annotation handwriting is added to the annotated page according to the track point coordinates, including: obtaining a reference image corresponding to the annotated page; obtaining a scaling ratio according to the size of the annotation background image and the size of the reference image corresponding to the annotated page; scaling the annotation handwriting according to the scaling ratio to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, obtaining the track point coordinates of the scaled annotation handwriting; according to the track point coordinates of the scaled annotation handwriting, adding the scaled annotation handwriting to the annotated page. By scaling the annotation handwriting, the scaled annotation handwriting can be more compatible with the size of the annotated page, so that when the scaled annotation handwriting is added to the annotated page and the annotation handwriting is displayed, the display effect of the annotation handwriting on the annotated page is better.

[0013] In a possible implementation, adding the annotation handwriting to the annotated page according to the annotation background image and the trajectory point coordinates includes: sending the annotation background image and the trajectory point coordinates of the annotation handwriting to the terminal, so that the terminal adds the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the trajectory point coordinates. In this method, the electronic whiteboard can also add the annotation handwriting to the annotated page by sending the annotation background image and the trajectory point coordinates of the annotation handwriting to the terminal, so that the terminal adds the annotation handwriting to the annotated page. Therefore, the subject of the method for performing the action of adding the annotation handwriting is more flexible and has a wider range of application scenarios.

[0014] In a possible implementation, before adding the annotation handwriting to the annotated page according to the coordinates of the track points based on the annotation background image, the method further includes: generating reference images corresponding to each page according to each page of the annotated file; matching the annotation background image with the reference images corresponding to each page, and obtaining the annotated page according to the matching result. By performing image matching, the method can quickly determine the annotated page from the annotated file.

[0015] In a second aspect, a method for saving annotation content is provided, which is applied to a terminal, and the method includes: receiving an annotation background image and trajectory point coordinates of annotation handwriting sent by an electronic whiteboard, wherein the annotation background image is generated based on an annotated page in an annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; according to the annotation background image, the annotation handwriting is added to the annotated page in the annotated file saved by the terminal according to the trajectory point coordinates.

[0016] In this method, the terminal can automatically add the annotation handwriting to the annotated page by adding the annotation handwriting to the annotated page according to the coordinates of the track points, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, this method has higher efficiency and accuracy in saving the annotation content.

[0017] In a possible implementation, according to the annotation background image, the annotation handwriting is added to the annotated page in the annotated file saved by the terminal according to the track point coordinates, including: obtaining a reference image corresponding to the annotated page; obtaining a scaling ratio according to the size of the annotation background image and the size of the reference image corresponding to the annotated page; scaling the annotation handwriting according to the scaling to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, obtaining the track point coordinates of the scaled annotation handwriting; according to the track point coordinates of the scaled annotation handwriting, adding the scaled annotation handwriting to the annotated page. By scaling the annotation handwriting, the adaptation degree of the scaled annotation handwriting to the size of the annotated page can be higher, so that when the scaled annotation handwriting is added to the annotated page and the annotation handwriting is displayed, the display effect of the annotation handwriting on the annotated page is better.

[0018] In a possible implementation, before adding the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the coordinates of the track points based on the annotation background image, the method further includes: generating reference images corresponding to each page according to each page of the annotated file; matching the annotation background image with the reference images corresponding to each page, and obtaining the annotated page according to the matching result. By performing image matching, the method can quickly determine the annotated page from the annotated file.

[0019] According to a third aspect, a device for storing annotation content is provided, which is applied to an electronic whiteboard, and the device includes: an acquisition module and an adding module; wherein the acquisition module is used to obtain the annotation background image and the coordinates of the trajectory points of the annotation handwriting, the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; the adding module is used to add the annotation handwriting to the annotated page according to the coordinates of the trajectory points based on the annotation background image.

[0020] In a possible implementation, the annotation handwriting includes at least one of graphic handwriting or text handwriting, and both the graphic handwriting and the text handwriting include at least one track point.

[0021] In one possible implementation, the marked handwriting includes multiple strokes, and the acquisition module is further used to input the multiple strokes included in the marked handwriting into a graphic-text hybrid classifier, and the graphic-text hybrid classifier is used to classify the strokes; obtain the classification result output by the graphic-text hybrid classifier, and the classification result includes at least one of graphic strokes or text strokes; when the classification result includes graphic strokes, obtain graphic handwriting based on the graphic strokes; when the classification result includes text strokes, obtain text handwriting based on the text strokes.

[0022] In a possible implementation, the acquisition module is used to perform graphic recognition on graphic strokes, obtain graphic handwriting based on the graphic recognition results, and the graphic handwriting includes at least one graphic; perform text recognition on text strokes, obtain text handwriting based on the text recognition results, and the text handwriting includes at least one text.

[0023] In a possible implementation, when the annotation handwriting includes graphic handwriting and text handwriting, a module is added, which is also used to obtain the position and size of at least one graphic included in the graphic handwriting, and the position and size of at least one text included in the text handwriting; obtain a first graphic and a first text according to the position and size of at least one graphic and the position and size of at least one text; adjust at least one of the position or size of the first graphic to obtain an adjusted first graphic, and / or adjust at least one of the position or size of the first text to obtain an adjusted first text.

[0024] In one possible implementation, when the annotation handwriting includes graphic handwriting and the graphic handwriting includes multiple graphics, a module is added to obtain the positions and sizes of the multiple graphics included in the graphic handwriting; obtain the second graphic and the third graphic according to the positions of the multiple graphics; adjust at least one of the position or size of the second graphic to obtain the adjusted second graphic, and / or adjust at least one of the position or size of the third graphic to obtain the adjusted third graphic.

[0025] In one possible implementation, a module is added to obtain a reference image corresponding to the annotated page; a scaling ratio is obtained based on the size of the annotated background image and the size of the reference image corresponding to the annotated page; the annotation handwriting is scaled according to the scaling ratio to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, the coordinates of the track points of the scaled annotation handwriting are obtained; and the scaled annotation handwriting is added to the annotated page according to the coordinates of the track points of the scaled annotation handwriting.

[0026] In a possible implementation, a module is added to send the annotated background image and the track point coordinates of the annotated handwriting to the terminal, so that the terminal adds the annotated handwriting to the annotated page in the annotated file saved by the terminal according to the annotated background image and the track point coordinates.

[0027] In a possible implementation, the acquisition module is further used to generate reference images corresponding to each page according to each page of the marked file; match the marked background image with the reference image corresponding to each page, and obtain the marked page according to the matching result.

[0028] In a fourth aspect, a device for saving annotation content is provided, which is applied to a terminal, and the device includes: an acquisition module and an adding module; wherein the acquisition module is used to receive an annotation background image and trajectory point coordinates of annotation handwriting sent by an electronic whiteboard, the annotation background image is generated based on an annotated page in an annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; the adding module is used to add the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background image and the trajectory point coordinates.

[0029] In one possible implementation, a module is added to obtain a reference image corresponding to the annotated page; a scaling ratio is obtained based on the size of the annotated background image and the size of the reference image corresponding to the annotated page; the annotation handwriting is scaled according to the scaling to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, the coordinates of the track points of the scaled annotation handwriting are obtained; and the scaled annotation handwriting is added to the annotated page according to the coordinates of the track points of the scaled annotation handwriting.

[0030] In a possible implementation, the acquisition module is further used to generate reference images corresponding to each page according to each page of the marked file; match the marked background image with the reference image corresponding to each page, and obtain the marked page according to the matching result.

[0031] In a fifth aspect, a system for preserving annotation content is provided, the system comprising an electronic whiteboard and a terminal, the electronic whiteboard being used to execute any method for preserving annotation content in the first aspect, and the terminal being used to execute any method for preserving annotation content in the second aspect.

[0032] In a sixth aspect, a computer system is provided, the computer system including a processor, when the processor executes a program instruction or code, the computer system implements any method for saving the annotation content in the first aspect, or any method for saving the annotation content in the second aspect. Exemplarily, the computer system also includes a memory, the memory is used to store the program instruction or code.

[0033] In the seventh aspect, a computer-readable storage medium is provided, which stores at least one program instruction or code. When the program instruction or code is loaded and executed by a processor, the computer implements any method for saving annotation content in the first aspect, or any method for saving annotation content in the second aspect.

[0034] In an eighth aspect, a communication device is provided, the device comprising: a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other through an internal connection path, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to control the transceiver to receive signals and control the transceiver to send signals, and when the processor executes the instructions stored in the memory, the processor executes any method for saving the annotation content in the first aspect, or any method for saving the annotation content in the second aspect.

[0035] Illustratively, there are one or more processors and one or more memories.

[0036] For example, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0037] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips. This application does not limit the type of memory and the setting method of the memory and the processor.

[0038] In the ninth aspect, a computer program or a computer program product is provided, and the computer program or the computer program product includes: a computer program code, when the computer program code is executed by a computer, the computer executes any method for saving the annotation content in the first aspect, or any method for saving the annotation content in the second aspect.

[0039] In the tenth aspect, a chip is provided, comprising a processor for calling and executing instructions stored in a memory from the memory. When the chip is installed in an electronic whiteboard, the electronic whiteboard executes any method for saving annotation content in the first aspect. When the chip is installed in a terminal, the terminal executes any method for saving annotation content in the second aspect.

[0040] Exemplarily, the chip further includes: an input interface, an output interface and the memory, and the input interface, the output interface, the processor and the memory are connected via an internal connection path.

[0041] It should be understood that the beneficial effects achieved by the technical solutions of the third to tenth aspects of the present application and the corresponding possible implementation methods can be referred to the above-mentioned technical effects of the first and second aspects and their corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of a marking content provided in an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of another type of marking content provided in an embodiment of the present application;

[0044] Figure 3 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0045] Figure 4 It is a flowchart of a method for saving annotation content provided in an embodiment of the present application;

[0046] Figure 5 is a schematic diagram for comparing a hand-drawn box and a standardized box provided in an embodiment of the present application;

[0047] Figure 6 It is a schematic diagram of comparison between handwritten text and standardized text provided in an embodiment of the present application;

[0048] Figure 7 It is a schematic diagram of a process of obtaining a digitized result of annotated handwriting provided in an embodiment of the present application;

[0049] Figure 8 It is a schematic diagram of a process of adding annotation handwriting to an annotated page provided by an embodiment of the present application;

[0050] Fig. 9 is a schematic diagram of a saved annotation content provided in an embodiment of the present application;

[0051] Fig.10 is a schematic diagram of another type of annotation content stored provided in an embodiment of the present application;

[0052] Fig.11 is a flowchart of another method for saving annotation content provided in an embodiment of the present application;

[0053] Fig.12 This is a flowchart of another method for saving annotation content provided in an embodiment of the present application;

[0054] Fig.13 It is a structural schematic diagram of a device for storing annotation content provided in an embodiment of the present application;

[0055] Fig.14It is a structural schematic diagram of another device for storing annotation content provided in an embodiment of the present application;

[0056] Fig.15 is a schematic diagram of the structure of a computer system provided in an embodiment of the present application;

[0057] Fig.16 It is a structural diagram of another computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The terms used in the implementation method of this application are only used to explain the embodiments of this application, and are not intended to limit this application. The embodiments of this application are described below in conjunction with the accompanying drawings.

[0059] With the development of computer technology, electronic whiteboards have gradually replaced ordinary whiteboards for use in scenarios such as meetings and education. Electronic whiteboards have functions such as screen projection and annotation. For example, when the electronic whiteboard has a screen projection function, the electronic whiteboard can receive a file sent by a terminal that is connected to the electronic whiteboard for communication, and display the file on the electronic whiteboard. When the electronic whiteboard has an annotation function, the user can write on the electronic whiteboard using a writing tool such as a stylus or a finger. For example, after a page of a file is displayed on the electronic whiteboard, the user uses a stylus to write the annotation content corresponding to the page on the electronic whiteboard. Figure 1 is a schematic diagram of a marking content provided in an embodiment of the present application. The marking content written by the user can be as follows Figure 1 shown.

[0060] The page displayed by the electronic whiteboard can also be an independent page, for example, the electronic whiteboard displays a page similar to a blackboard or a whiteboard. In this case, the annotation content written by the user can be called blackboard writing. Figure 2 is a schematic diagram of another type of annotation content provided in an embodiment of the present application. The annotation content written by the user on an independent page can be as follows Figure 2 shown.

[0061] In the related art, the electronic whiteboard sends the annotation content written by the user to the terminal in the form of a picture or a portable document format (PDF) file. Since the content of the picture or PDF file cannot be edited, when the annotation content needs to be saved in an editable manner, the user needs to manually add the annotation content to the editable file after receiving the picture or PDF file. For example, when the annotation content is used to annotate a certain page of the file, the annotation content is added to the annotated page in the file; when the annotation content is written on an independent page, the annotation content is input into a newly created file in an editable form. However, the method of manually saving the annotation content in an editable file by the user is inefficient, and when there are many annotation contents, the user is prone to errors when manually adding the annotation content, resulting in low accuracy of the saved annotation content.

[0062] The present application embodiment provides a method for saving annotation content, which is used to improve the efficiency and accuracy of saving annotation content. The method can be applied to Figure 3 The implementation environment shown is shown in Figure 3 , the implementation scenario includes an electronic whiteboard 301 and a terminal 302, and the electronic whiteboard 301 and the terminal 302 are connected in communication. The electronic whiteboard 301 and the terminal 302 can be connected in communication via a wired or wireless network, which is not limited in the embodiments of the present application. For example, the electronic whiteboard 301 and the terminal 302 are connected in communication via a peer-to-peer (P2P) technology or an access point wireless fidelity (AP-WIFI).

[0063] The embodiment of the present application does not limit the type of the electronic whiteboard 301. The terminal 302 can be any electronic product that can interact with the user through one or more methods such as a keyboard, a touchpad, a touch screen, a remote control, voice interaction or a handwriting device, for example, a mobile phone, a personal digital assistant (PDA), a wearable device, a pocket personal computer (PPC), a tablet computer, a smart car machine, a smart TV, a smart speaker, a smart voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, a smart watch, etc. Those skilled in the art should understand that the above-mentioned terminal 302 is only an example, and other existing or future terminals that may appear should also be included in the scope of protection of the present application if they are applicable to the present application, and are included here by reference.

[0064] The method for storing the annotation content provided in the embodiment of the present application can be as follows: Figure 4 As shown, next, combined Figure 3 The implementation scenario shown in FIG. Figure 4 As shown, the method can be applied to an electronic whiteboard, and the method includes but is not limited to S401 and S402.

[0065] S401, the electronic whiteboard obtains the coordinates of the track points of the annotation background image and the annotation handwriting, the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page.

[0066] In an embodiment of the present application, the annotation function of the electronic whiteboard can be turned on or off according to the user's instructions. For example, the electronic whiteboard receives an on instruction generated by the user pressing the annotation function switch button on the electronic whiteboard, and turns on the annotation function based on the on instruction. For another example, the electronic whiteboard receives an off instruction generated by the user pressing the annotation function switch button on the electronic whiteboard, and turns off the annotation function based on the off instruction. When the annotation function is turned on, the method provided in the embodiment of the present application can be executed.

[0067] Among them, the marked files include but are not limited to those sent by the terminal to the electronic whiteboard. Exemplarily, in a conference scenario, the participants in the meeting send the file to be discussed to the electronic whiteboard through the terminal used, and the page of the file is displayed on the electronic whiteboard. Thus, the participants in the meeting can mark the page of the file on the electronic whiteboard, for example, the participants in the meeting write the marked content of the page on the electronic whiteboard through writing tools such as fingers or styluses. In this case, the file is a marked file, the page is a marked page, and the handwriting of the user when writing the marked content is the marked handwriting. In an educational scenario, the teacher sends the courseware to the electronic whiteboard through the terminal used, and displays the page of the courseware through the electronic whiteboard. Thus, the teacher can mark the page of the courseware on the electronic whiteboard during the teaching process, for example, the teacher writes the marked content of the page on the electronic whiteboard through fingers or writing tools. In this case, the courseware is a marked file, the page of the courseware is a marked page, and the handwriting of the teacher when writing the marked content is the marked handwriting.

[0068] In an embodiment of the present application, the coordinates of the track points of the marked handwriting are the coordinates of the marked handwriting on the display interface of the electronic whiteboard. For example, a coordinate system is established with a certain point on the display interface of the electronic whiteboard, and the coordinates of the track points of the marked handwriting are the coordinates of the marked handwriting in the coordinate system. The display interface of the electronic whiteboard can be the entire screen or a portion of the screen of the electronic whiteboard, and the embodiment of the present application is not limited to this. The display interface can be used entirely to display the annotated page. When the display interface is used entirely to display the annotated page, the coordinate system established with a certain point on the display interface can also be regarded as a coordinate system established with the position of the point in the annotated background picture. Exemplarily, the marked handwriting includes multiple strokes, and any one of the multiple strokes includes at least one track point, then the coordinates of the track points of the marked handwriting are the coordinates of the track points included in the multiple strokes included in the marked handwriting.

[0069] In one possible implementation, each time the user finishes writing a stroke and lifts the writing tool or finger, the electronic whiteboard saves the coordinates of the trajectory points included in the stroke; or, after the user finishes writing all the annotation content, the electronic whiteboard saves the coordinates of the trajectory points included in each stroke included in the annotation content, so that the electronic whiteboard can obtain the coordinates of the trajectory points of the annotated handwriting. The embodiment of the present application does not limit the method of saving the coordinates of the trajectory points of the annotated handwriting, so that the method of obtaining the coordinates of the trajectory points of the annotated handwriting is more flexible. Exemplarily, the electronic whiteboard can also save the writing time of each trajectory point of the stroke, so that the trajectory points with continuous writing time can be determined as one stroke.

[0070] Exemplarily, the electronic whiteboard obtains the annotated background image by taking a screenshot of the annotated page. For example, each time the electronic whiteboard displays a page, it takes a screenshot of the page, thereby obtaining the annotated background image. For another example, the electronic whiteboard takes a screenshot of the page after the page is annotated, thereby obtaining the annotated background image. In other words, the content displayed on the annotated background image may include the annotated handwriting, or may not include the annotated handwriting, and the embodiment of the present application does not limit this. The timing of obtaining the annotated background image in the embodiment of the present application is relatively flexible.

[0071] S402, the electronic whiteboard adds the annotation handwriting to the annotated page according to the annotation background image and the coordinates of the track points.

[0072] That is, the electronic whiteboard can save the annotated handwriting to the annotated page in the annotated file saved by the electronic whiteboard. In a possible implementation, the electronic whiteboard adds the annotated handwriting to the annotated page according to the track point coordinates based on the annotated background picture, including: sending the annotated background picture and the track point coordinates of the annotated handwriting to the terminal, so that the terminal adds the annotated handwriting to the annotated page in the annotated file saved by the terminal according to the track point coordinates. In other words, the electronic device can also send the annotated background picture and the track point coordinates of the annotated handwriting to the terminal, and the terminal saves the annotated handwriting in the annotated page in the annotated file saved by the terminal, thereby realizing the addition of the annotated handwriting to the annotated page. Therefore, the subject that performs the action of adding the annotated handwriting in the embodiment of the present application is more flexible and has a wider range of application scenarios. The embodiment of the present application does not limit the way in which the electronic whiteboard sends the annotated background picture and the track point coordinates of the annotated handwriting to the terminal. For example, the electronic whiteboard uses the communication method of receiving the annotated file to send the annotated background picture and the track point coordinates of the annotated handwriting to the terminal.

[0073] In a possible implementation, before adding the annotation handwriting to the annotated page according to the annotation background image and the track point coordinates, the electronic whiteboard determines the annotated page from at least one page included in the annotated file according to the annotation background image, so that the annotation handwriting can be added to the annotated page according to the track point coordinates. For example, in the case where the annotated file includes one page, the page is the annotated page. In the case where the annotated file includes multiple pages, each page has a number, and the electronic whiteboard saves the number of each page. The electronic whiteboard can also save the number of the page when obtaining the annotation background image, so that the page corresponding to the number can be determined from the multiple pages included in the annotated file according to the number of the annotation background image, and the page is the annotated page.

[0074] Alternatively, the electronic whiteboard can also obtain the marked page according to the marked background image by means of image matching. For example, the electronic whiteboard generates reference images corresponding to each page according to each page of the marked file; matches the marked background image with the reference image corresponding to each page, and obtains the marked page according to the matching result. In an embodiment of the present application, the electronic whiteboard can generate reference images corresponding to each page by taking screenshots of each page of the marked file. Regarding the method of matching the marked background image with the reference image corresponding to each page, the embodiment of the present application does not limit this, for example, using an image matching algorithm to identify the reference image matching the marked background image in the reference image corresponding to each page. The embodiment of the present application does not limit the type of image matching algorithm, and can always identify the reference image matching the marked background image from the reference image corresponding to each page, so that the marked page can be determined according to the reference image. By performing image matching, the method provided in the embodiment of the present application can determine the marked page from the marked file relatively quickly.

[0075] In an embodiment of the present application, the electronic whiteboard can perform content recognition on the annotated handwriting, and obtain the annotated content corresponding to the annotated handwriting according to the content recognition result. The annotated content can be a standardized font or graphic used by the electronic whiteboard and the terminal, rather than handwriting. Alternatively, the electronic whiteboard can also not perform content recognition on the annotated handwriting, and directly add the annotated handwriting to the annotated page in the form of handwriting. In an embodiment of the present application, a standardized font refers to the font used when the user enters text through the keyboard, rather than the font when the user writes text with a finger or a stylus. A standardized graphic refers to a graphic drawn by a user through a standardized graphic type provided by the software, such as a rectangle obtained by selecting a rectangle, a circle drawn by selecting a circle, a polygon drawn by selecting a line, etc., rather than a graphic drawn by a user with a finger or a stylus. Whether it is a standardized font or graphic or handwriting, the annotated handwriting in the embodiment of the present application includes but is not limited to at least one of a graphic handwriting or a text handwriting. That is, the method provided in the embodiment of the present application can be applied to the case where the annotated handwriting includes multiple types of handwriting, and has a wide range of application. The graphic handwriting and the text handwriting both include at least one track point, so that the track point coordinates of the graphic handwriting include the coordinates of at least one track point included in the graphic handwriting, and the track point coordinates of the text handwriting include the coordinates of at least one track point included in the text handwriting.

[0076] In a possible implementation, when the marked handwriting includes multiple strokes, the types of the multiple strokes are distinguished to determine whether each stroke corresponds to a graphic handwriting or a text handwriting, and then the graphic handwriting is obtained according to the strokes corresponding to the graphic handwriting, and the text handwriting is obtained according to the strokes corresponding to the text handwriting. The operation of distinguishing the types of multiple strokes can be implemented by inputting the multiple strokes included in the marked handwriting into a graphic-text hybrid classifier. That is, the method may also include: inputting the multiple strokes included in the marked handwriting into a graphic-text hybrid classifier, the graphic-text hybrid classifier is used to classify the strokes; obtaining the classification result output by the graphic-text hybrid classifier, the classification result includes at least one of a graphic stroke or a text stroke; when the classification result includes a graphic stroke, obtaining the graphic handwriting according to the graphic stroke, and when the classification result includes a text stroke, obtaining the text handwriting according to the text stroke.

[0077] By using a hybrid image-text classifier, the method provided in the embodiment of the present application can be applied to the situation where the handwriting annotation includes both graphic strokes and text strokes, and has a wide range of applications. The embodiment of the present application does not limit the type of the hybrid image-text classifier, and can distinguish the types of multiple strokes through one classifier. Exemplarily, the hybrid image-text classifier is a hybrid image-text classifier implemented based on a graph attention network (GAT) model. The embodiment of the present application does not limit the type of the hybrid image-text classifier, and can be any classifier that can distinguish graphic strokes from text strokes. Regarding the acquisition method of the hybrid image-text classifier, it includes but is not limited to training the initial GAT model based on training samples. Among them, the training samples can be graphic strokes and text strokes of known stroke types, and the hybrid image-text classifier that meets the requirements is obtained by training the initial GAT model based on the training samples. Exemplarily, meeting the requirements means that the accuracy reaches a threshold, or the number of training times reaches a predetermined number of times, or the model converges, etc. The threshold and the predetermined number of times can be set according to experience or actual needs, and the embodiment of the present application does not limit this.

[0078] Exemplarily, the graphic handwriting is obtained by means of graphic recognition, and the text handwriting is obtained by means of text recognition. That is to say, obtaining the graphic handwriting according to the graphic strokes includes, but is not limited to: performing graphic recognition on the graphic strokes, obtaining the graphic handwriting according to the obtained graphic recognition results, and the graphic handwriting includes at least one graphic; obtaining the text handwriting according to the text strokes includes, but is not limited to: performing text recognition on the text strokes, obtaining the text handwriting according to the obtained text recognition results, and the text handwriting includes at least one text. By recognizing the graphic strokes and the text strokes, the method provided in the embodiment of the present application can add the annotation handwriting to the annotation page in the form of standardized graphics and text, thereby eliminating the need for the user to manually convert the graphic handwriting and the text handwriting into standardized graphics and text, respectively, and providing a better user experience.

[0079] Both graphic recognition and text recognition can be implemented by optical character recognition (OCR) technology, and the OCR technology used to implement graphic recognition and text recognition can be an online OCR technology or an offline OCR technology. The specific method of performing graphic recognition and text recognition by OCR technology can refer to the content in the relevant technology, which will not be described in detail here. In an embodiment of the present application, the graphic recognition result includes the type and size of the graphic, and a standardized graphic used by the terminal and the electronic whiteboard is obtained according to the type and size of the graphic. For example, the graphic recognition result includes: the type is a square box, and the size includes a length of 1 cm and a width of 0.5 cm, then a standardized square box used by the terminal and the electronic whiteboard is obtained according to the type and size of the graphic, and the length of the square box is 1 cm and the width is 0.5 cm. Figure 5 is a schematic diagram comparing a hand-drawn box and a standardized box provided in an embodiment of the present application. Figure 5 The left side of is a schematic diagram of a hand-drawn box. Figure 5 To the right of is a schematic diagram of the standardized box.

[0080] The text recognition result includes but is not limited to the font, font size and content of the text, and the standardized text used by the terminal and the electronic whiteboard is obtained based on the font, font size and content of the text. For example, if the text recognition result includes: the font is Kaiti, the font size is 14, and the content is "modify", then the standardized text used by the terminal and the electronic whiteboard is obtained based on the font, font size and content of the graphic, and the font of the text is Kaiti, the font size is 14, and the content is "modify". Figure 6 is a schematic diagram of comparison between handwritten text and standardized text provided in an embodiment of the present application. Figure 6 The left side of the diagram is a diagram of handwritten text. Figure 6 To the right of is a schematic diagram of standardized text.

[0081] In an embodiment of the present application, the electronic whiteboard can also obtain other information of the marked handwriting. Exemplarily, the electronic whiteboard also obtains the position of the marked handwriting. For example, the electronic whiteboard obtains the position of the marked handwriting based on the coordinates of the trajectory points included in the marked handwriting. The position of the marked handwriting can be the position of the marked handwriting on the marked background image. The electronic whiteboard can also obtain the color of the marked handwriting. For example, the electronic whiteboard also obtains the handwriting color used by the user when writing the marked handwriting, so that when the marked handwriting is subsequently added to the annotated page, the marked handwriting can be added according to the color of the marked handwriting. Similarly, the electronic whiteboard can also send the color of the marked handwriting to the terminal, so that when the terminal adds the marked handwriting to the annotated page in the annotated file saved by the terminal, the marked handwriting can be added according to the color of the marked handwriting.

[0082] In a possible implementation, when the annotation handwriting includes graphic handwriting and text handwriting, the electronic whiteboard also determines the association relationship between the graphics included in the graphic handwriting and the text included in the text handwriting, so that the electronic whiteboard can adjust the graphics and text with an association relationship. In an embodiment of the present application, if a certain text is included in a certain graphic, the text has an association relationship with the graphic; if a certain text is located outside a certain graphic and the distance from the graphic does not exceed a first distance threshold, the text also has an association relationship with the graphic. The first distance threshold can be set according to experience or actual needs, and the embodiment of the present application does not limit this.

[0083] Exemplarily, before adding the annotation handwriting to the annotated page according to the coordinates of the track point according to the annotated background image, the method further includes: obtaining the position and size of at least one graphic included in the graphic handwriting, and the position and size of at least one character included in the character handwriting; obtaining the first graphic and the first character according to the position and size of the at least one graphic and the position and size of the at least one character; adjusting at least one of the position or size of the first graphic to obtain the adjusted first graphic, and / or adjusting at least one of the position or size of the first character to obtain the adjusted first character. That is, the first graphic and the first character are graphics and characters with an associated relationship, so that at least one of the size or position of the first graphic and the first character can be adjusted. By adjusting at least one of the position or size of the first graphic, and / or adjusting at least one of the position or size of the first character, after the adjustment, the position of the first character and the position of the first graphic can not overlap, thereby ensuring the display effect of the first character and the first graphic. Furthermore, in the case where the first character is located in the first graphic, after the adjustment, the position of the first character can be located at the reference position of the first graphic, thereby improving the subsequent display effect of the first character and the first graphic, and the reference position includes but is not limited to any one of the center, the left side or the right side.

[0084] In an embodiment of the present application, when the annotation handwriting includes graphic handwriting and the graphic handwriting includes multiple graphics, the electronic whiteboard can determine the association relationship between the multiple graphics, so that the electronic whiteboard can adjust the graphics with an association relationship. For example, a graphic of a box type and an arrow type can have an association relationship, a graphic of a diamond box type and an arrow type can have an association relationship, and two graphics that have an association relationship with the same type of arrow graphic also have an association relationship. Exemplarily, if the distance between the head or tail of a certain type of arrow graphic and a certain type of box graphic does not exceed the second distance threshold, then the arrow has an association relationship with the graphic. For another example, if the distance between the head or tail of a certain type of arrow graphic and a certain type of diamond box graphic does not exceed the second distance threshold, then the arrow has an association relationship with the graphic. For another example, if a certain type of graphic is an arrow and two types of graphics are associated, then the two types of graphics are also associated; or if a certain type of graphic is an arrow and a type of graphic is a square and a type of graphic is a diamond box, then the type of graphic is a square and the type of graphic is a diamond box. The second distance threshold can be set according to experience or actual needs, and the embodiment of the present application is not limited to this.

[0085] Exemplarily, according to the annotated background image, before the annotation handwriting is added to the annotated page according to the coordinates of the trajectory point, the method further includes: obtaining the positions and sizes of multiple graphics included in the graphic handwriting; obtaining the second graphic and the third graphic according to the positions of the multiple graphics; adjusting at least one of the position or size of the second graphic to obtain the adjusted second graphic, and / or adjusting at least one of the position or size of the third graphic to obtain the adjusted third graphic. That is, the second graphic and the third graphic are graphics with an associated relationship, so that at least one of the size or position of the second graphic and the third graphic can be adjusted. By adjusting at least one of the position or size of the second graphic, and / or adjusting at least one of the position or size of the third graphic, the display effect of the second graphic and the third graphic can be improved, so that the display of the second graphic and the third graphic is more in line with the requirements. For example, after adjustment, when the second graphic is a graphic of the arrow type and the third graphic is a graphic of the box type, the head or tail of the second graphic can be located at the reference position of the third graphic, and the reference position includes but is not limited to any one of the center, the left side or the right side. When the second graphic and the third graphic are both of box type, the second graphic and the third graphic are parallel in the horizontal direction or the vertical direction, and / or the second graphic and the third graphic are the same in size.

[0086] In the embodiment of the present application, whether it is the adjusted first graphic, the adjusted first text, the adjusted second graphic, or the adjusted third graphic, all can be added to the annotated page or sent to the terminal as annotation handwriting. In the embodiment of the present application, the process of adjusting at least one of the first text, the first graphic, the second graphic, or the third graphic is referred to as the process of adjusting the annotation handwriting.

[0087] Figure 7 Schematic diagram of a process for obtaining a digital result of annotated handwriting provided by an embodiment of the present application. Figure 7 As shown, after obtaining the coordinates of the track points of the marked handwriting, it is determined whether the electronic whiteboard has the recognition function turned on. If the recognition function is not turned on, the coordinates of the track points of the marked handwriting are used as the digitized results of the marked handwriting. If the recognition function is turned on, the multiple strokes included in the marked handwriting are input into the graphic-text hybrid classifier, and the graphic handwriting and the text handwriting are obtained according to the classification results; then, the graphic handwriting is subjected to graphic recognition to obtain at least one graphic included in the graphic handwriting, and the text handwriting is subjected to text recognition to obtain at least one text included in the text handwriting. The electronic whiteboard also obtains the position and size of the graphic and the position and size of the text, and the obtained graphic, text, the position and size of the graphic, and the position and size of the text are used as the digitized results of the marked handwriting.

[0088] Since the size of the annotation background image may be different from the size of the annotated page, when adding the annotation handwriting to the annotated page according to the track point coordinates, the annotation handwriting can be adjusted according to the size relationship between the annotation background image and the annotated page to obtain the adjusted annotation handwriting. Then, by obtaining the track point coordinates of the adjusted annotation handwriting based on the adjusted annotation handwriting, the adjusted annotation handwriting is added to the annotated page according to the track point coordinates of the adjusted annotation handwriting, so that the adjusted annotation handwriting is more compatible with the size of the annotated page. When the adjusted annotation handwriting is added to the annotated page and the annotation handwriting is displayed, the display effect of the annotation handwriting on the annotated page is better. Exemplarily, according to the annotation background image, the annotation handwriting is added to the annotated page according to the trajectory point coordinates, including: obtaining a reference image corresponding to the annotated page; obtaining a scaling ratio according to the size of the annotation background image and the size of the reference image corresponding to the annotated page; scaling the annotation handwriting according to the scaling ratio to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, obtaining the trajectory point coordinates of the scaled annotation handwriting; adding the scaled annotation handwriting to the annotated page according to the trajectory point coordinates of the scaled annotation handwriting.

[0089] In a possible implementation, the size of the annotation background image is divided by the size of the reference graphic corresponding to the annotated page, and the obtained quotient is used as the obtained scaling ratio. Since the annotation handwriting is composed of multiple track points included in the annotation handwriting, when the annotation handwriting is scaled, the coordinates of the track points of the annotation handwriting will also change. For any track point among the multiple track points included in the annotation handwriting, the coordinates of any track point before scaling and the coordinates of any track point after scaling are called position offsets, and the value of the position offset can be equal to the value of the coordinates of any track point multiplied by the scaling ratio. That is, based on the scaled annotation handwriting, the coordinates of the track points of the scaled annotation handwriting are obtained, including but not limited to: multiplying the values ​​of the track point coordinates of the annotation handwriting before scaling by the scaling ratio, and obtaining the coordinates of the track points of the scaled annotation handwriting based on the obtained products.

[0090] In an embodiment of the present application, when the size of the annotated background image is different from that of the annotated page, for any track point included in the annotated handwriting, the track point coordinates of any track point before scaling can be the coordinates of any track point in the first plane coordinate system established with the first reference point in the annotated background image. Since the display interface of the electronic whiteboard is used to fully display the annotated background image, the position of the first reference point can be the position of the point in the annotated background image used to establish the coordinate system on the display interface. The track point coordinates of any track point after scaling can be the coordinates of any track point in the second plane coordinate system established with the second reference point in the reference image corresponding to the annotated page, and the embodiment of the present application does not limit the position of the second reference point. In the case where the position of the first reference point in the annotated background image is the same as the position of the second reference point in the reference image, the coordinates of the track point before scaling can be directly multiplied by the scaling ratio, and the product obtained can be used as the track point coordinates after scaling. When the position of the first reference point in the annotated background image is different from the position of the second reference point in the reference image, the coordinate conversion factors of the first plane coordinate system and the second plane coordinate system can be obtained, the coordinates of the trajectory point before scaling are directly multiplied by the scaling ratio, the product is transformed based on the coordinate conversion factor, and the transformed result is used as the coordinates of the trajectory point after scaling.

[0091] Exemplarily, the coordinate conversion factor is obtained by the difference between the coordinates of a certain point in the annotated background image in the first plane coordinate system and the coordinates of the point in the second plane coordinate system. For example, for a certain point in the annotated background image, the coordinates of the point in the first plane coordinate system are (1, 1), and the coordinates of the point in the second plane coordinate system are (1, 1), then the coordinate conversion factor is (1, 1); for any track point included in the annotated track, after multiplying the any track point by the scaling ratio, the horizontal and vertical coordinates are added by 1 respectively, and the coordinates of the any track point after scaling can be obtained.

[0092] In an embodiment of the present application, the process of adding a graphic to an annotated page can be regarded as the process of drawing a graphic in the annotated page, and the process of adding text to an annotated page can be regarded as the process of writing text in the annotated page. Since the graphics and text are added to the annotated page in an editable form in the embodiment of the present application, at least one of the graphics or text can be edited later, such as adjusting one of the position or size of the graphics, and / or adjusting at least one of the position or size of the text. Furthermore, in the case where at least one of the position or size of the text is adjusted, the graphics associated with the text can be automatically adjusted in at least one of the position or size; in the case where at least one of the position or size of the graphics is adjusted, the text or graphics associated with the graphics can be automatically adjusted in at least one of the position or size. In the case where the graphics are curved, adjusting the position of the graphics includes adjusting the degree of curvature of the graphics.

[0093] Figure 8 Schematic diagram of a process of adding annotation handwriting to an annotated page provided by an embodiment of the present application. Figure 8 As shown, according to the annotation background image, the annotated page is obtained from multiple pages included in the annotated file; then, the reference image corresponding to the annotated page is obtained, and the scaling ratio is obtained according to the size of the annotation background image and the size of the reference image corresponding to the annotated page; then, the annotation handwriting is scaled according to the scaling ratio to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, the track point coordinates of the scaled annotation handwriting are obtained; and the scaled annotation handwriting is added to the annotated page according to the track point coordinates of the scaled annotation handwriting. Fig. 9 is a schematic diagram of a saved annotation content provided in an embodiment of the present application, such as Fig. 9 As shown, the annotation handwriting saved in the annotated page is the unrecognized annotation handwriting, that is, Fig. 9 The annotation handwriting shown is handwritten. Fig.10 is a schematic diagram of another type of annotation content stored provided in an embodiment of the present application, such as Fig.10 As shown, the annotation handwriting saved in the annotated page is the standardized text and graphics obtained after recognition, that is, Fig.10 The annotation handwriting shown includes standardized text and graphics.

[0094] In the method provided in the embodiment of the present application, the electronic whiteboard can automatically add the annotation handwriting to the annotated page by adding the annotation handwriting to the annotated page according to the coordinates of the track points, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, this method has higher efficiency and accuracy in saving the annotation content. Furthermore, after adding the annotation content to the annotated page, if the user needs to edit the annotation content, the user can edit the annotation content by modifying the annotation handwriting in the annotated page, which provides a better user experience.

[0095] The above describes the method for saving the annotation content provided by the embodiment of the present application by taking the electronic whiteboard side as an example. Next, the method for saving the annotation content by taking the terminal side as an example is described. Fig.11 As shown, the method includes but is not limited to S1101 and S1102.

[0096] S1101, the terminal receives an annotation background image and trajectory point coordinates of annotation handwriting sent by the electronic whiteboard, the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page.

[0097] The embodiment of the present application does not limit the way in which the terminal receives the coordinates of the trajectory points of the annotated background image and the annotated handwriting, and can correspond to the way in which the electronic whiteboard sends the annotated background image and the annotated handwriting to the terminal. The annotated handwriting includes at least one of graphic handwriting or text handwriting. The situation of graphic handwriting and text handwriting can refer to the relevant content of graphic handwriting and text handwriting obtained on the electronic whiteboard side, which will not be repeated here.

[0098] S1102, the terminal adds the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background image and the coordinates of the track points.

[0099] Exemplarily, the way the terminal executes S1102 is the same as the way the electronic whiteboard adds the annotation handwriting to the annotated page according to the coordinates of the track points based on the annotated background image. Please refer to the content in S402 above, which will not be repeated here.

[0100] In a possible implementation, according to the annotation background image, the annotation handwriting is added to the annotated page in the annotated file saved by the terminal according to the coordinates of the track points, including: obtaining a reference image corresponding to the annotated page; obtaining a scaling ratio according to the size of the annotation background image and the size of the reference image corresponding to the annotated page; scaling the annotation handwriting according to the scaling to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, obtaining the coordinates of the track points of the scaled annotation handwriting; according to the coordinates of the track points of the scaled annotation handwriting, adding the scaled annotation handwriting to the annotated page. The scaling of the annotation handwriting by the terminal and adding the scaled annotation handwriting to the content of the annotated page are the same as the scaling of the annotation handwriting by the electronic whiteboard and adding the scaled annotation handwriting to the content of the annotated page, which will not be repeated here. By scaling the annotation handwriting, the adaptation degree of the scaled annotation handwriting to the size of the annotated page can be higher, so that when the scaled annotation handwriting is added to the annotated page and the annotation handwriting is displayed, the display effect of the annotation handwriting on the annotated page is better.

[0101] Exemplarily, before the terminal adds the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background image and the coordinates of the trajectory points, the method further includes: generating reference images corresponding to each page according to each page of the annotated file; matching the annotation background image with the reference images corresponding to each page, and obtaining the annotated page according to the matching result. The process of the terminal obtaining the annotated page by image matching is the same as the process of the electronic whiteboard obtaining the annotated page by image matching, which will not be repeated here. By performing image matching, the method provided in the embodiment of the present application can determine the annotated page from the annotated file relatively quickly.

[0102] In the method provided in the embodiment of the present application, the terminal can automatically add the annotation handwriting to the annotated page by adding the annotation handwriting to the annotated page according to the coordinates of the trajectory points, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, this method has higher efficiency and accuracy in saving the annotation content. Furthermore, after adding the annotation content to the annotated page, if the user needs to edit the annotation content, the user can edit the annotation content by modifying the annotation handwriting in the annotated page, which provides a better user experience.

[0103] Next, the method for saving the marked content is described by taking the example that the saving of the marked content is realized by the electronic whiteboard and the terminal together. Fig.12 is a flowchart of another method for saving annotation content provided in an embodiment of the present application, such as Fig.12 As shown, the method includes but is not limited to S1201 to S1205.

[0104] S1201, when the annotation function is turned on and the annotated page of the annotated file is annotated, the electronic whiteboard obtains the annotation background image and the coordinates of the trajectory points of the annotation handwriting.

[0105] The content of S1201 is the same as that of S401 and will not be repeated here.

[0106] S1202, the electronic whiteboard obtains the graphics, position and size of the graphic handwriting in the marked handwriting, and the text, position and size of the text handwriting in the marked handwriting.

[0107] S1202 content and Figure 4 In the illustrated method embodiment, the principles of obtaining the graphics, position and size of graphic handwriting and obtaining the text, position and size of text handwriting are the same and will not be repeated here.

[0108] S1203, the electronic whiteboard sends the annotated background image, the graphic, position and size of the graphic handwriting, and the text, position and size of the text handwriting to the terminal.

[0109] The content of S1203 is the same as that of S402 and will not be repeated here.

[0110] S1204: The terminal determines the marked page from a plurality of pages included in the stored marked file according to the received marked background image.

[0111] The content of S1204 is the same as the principle of the terminal obtaining the content of the marked page in a way of image matching, which will not be repeated here.

[0112] S1205, the terminal adds the graphic handwriting and the text handwriting to the annotated page according to the annotated background image, the graphic handwriting, the position and size, and the text handwriting, the position and size.

[0113] Since the graphics, position and size of the graphic handwriting, as well as the text, position and size of the text handwriting can be represented by the track point coordinates, the content of S1205 is the same as Fig.11 The content and principle of the terminal executing S1102 in the illustrated method embodiment are the same and will not be repeated here.

[0114] The embodiment of the present application also provides a device for storing annotation content. Fig.13 Schematic diagram of a device for storing annotation content provided in an embodiment of the present application. The device is applied to an electronic whiteboard, that is, based on Fig.13 The device can perform all or part of the operations performed by the electronic whiteboard. It should be understood that the device may include more additional modules than the modules shown or omit some of the modules shown, and the embodiments of the present application are not limited to this. Fig.13As shown, the device includes an acquisition module 1301 and an adding module 1302 .

[0115] The acquisition module 1301 is used to obtain the coordinates of the trajectory points of the annotation background image and the annotation handwriting. The annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; the adding module 1302 is used to add the annotation handwriting to the annotated page according to the annotation background image and the coordinates of the trajectory points.

[0116] In a possible implementation, the annotation handwriting includes at least one of graphic handwriting or text handwriting, and both the graphic handwriting and the text handwriting include at least one track point.

[0117] In one possible implementation, the marked handwriting includes multiple strokes, and the acquisition module 1301 is also used to input the multiple strokes included in the marked handwriting into a graphic-text hybrid classifier, and the graphic-text hybrid classifier is used to classify the strokes; obtain the classification result output by the graphic-text hybrid classifier, and the classification result includes at least one of graphic strokes or text strokes; when the classification result includes graphic strokes, obtain graphic handwriting based on the graphic strokes; when the classification result includes text strokes, obtain text handwriting based on the text strokes.

[0118] In a possible implementation, the acquisition module 1301 is used to perform graphic recognition on graphic strokes, obtain graphic handwriting based on the graphic recognition results, and the graphic handwriting includes at least one graphic; perform text recognition on text strokes, obtain text handwriting based on the text recognition results, and the text handwriting includes at least one text.

[0119] In a possible implementation, when the annotation handwriting includes graphic handwriting and text handwriting, the adding module 1302 is also used to obtain the position and size of at least one graphic included in the graphic handwriting, and the position and size of at least one text included in the text handwriting; obtain a first graphic and a first text according to the position and size of at least one graphic and the position and size of at least one text; adjust at least one of the position or size of the first graphic to obtain an adjusted first graphic, and / or adjust at least one of the position or size of the first text to obtain an adjusted first text.

[0120] In one possible implementation, when the annotation handwriting includes graphic handwriting and the graphic handwriting includes multiple graphics, add module 1302, which is also used to obtain the positions and sizes of the multiple graphics included in the graphic handwriting; obtain the second graphic and the third graphic according to the positions of the multiple graphics; adjust at least one of the position or size of the second graphic to obtain the adjusted second graphic, and / or adjust at least one of the position or size of the third graphic to obtain the adjusted third graphic.

[0121] In one possible implementation, a module 1302 is added to obtain a reference image corresponding to the annotated page; a scaling ratio is obtained according to the size of the annotated background image and the size of the reference image corresponding to the annotated page; the annotation handwriting is scaled according to the scaling ratio to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, the track point coordinates of the scaled annotation handwriting are obtained; and the scaled annotation handwriting is added to the annotated page according to the track point coordinates of the scaled annotation handwriting.

[0122] In a possible implementation, adding module 1302 is used to send the annotated background image and the track point coordinates of the annotated handwriting to the terminal, so that the terminal adds the annotated handwriting to the annotated page in the annotated file saved by the terminal according to the annotated background image and the track point coordinates.

[0123] In a possible implementation, the acquisition module 1301 is further used to generate reference images corresponding to each page according to each page of the marked file; match the marked background image with the reference image corresponding to each page, and obtain the marked page according to the matching result.

[0124] In the device provided in the embodiment of the present application, by adding the annotation handwriting to the annotated page according to the coordinates of the trajectory point, the annotation handwriting can be automatically added to the annotated page, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, the efficiency and accuracy of the device in saving the annotation content are higher. Furthermore, after adding the annotation content to the annotated page, if the user needs to edit the annotation content, the user can edit the annotation content by modifying the annotation handwriting in the annotated page, which provides a better user experience.

[0125] Fig.14 Schematic diagram of another storage device for annotation content provided in an embodiment of the present application. The device is applied to a terminal, that is, based on Fig.14 The device can perform all or part of the operations performed by the terminal. It should be understood that the device may include more additional modules than the modules shown or omit some of the modules shown, and the embodiments of the present application are not limited to this. Fig.14 As shown, the device includes an acquisition module 1401 and an adding module 1402.

[0126] The acquisition module 1401 is used to receive the annotation background image and the track point coordinates of the annotation handwriting sent by the electronic whiteboard. The annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; the adding module 1402 is used to add the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background image and the track point coordinates.

[0127] In one possible implementation, a module 1402 is added to obtain a reference image corresponding to the annotated page; obtain a scaling ratio based on the size of the annotated background image and the size of the reference image corresponding to the annotated page; scale the annotation handwriting based on the scaling to obtain the scaled annotation handwriting; based on the scaled annotation handwriting, obtain the track point coordinates of the scaled annotation handwriting; and add the scaled annotation handwriting to the annotated page according to the track point coordinates of the scaled annotation handwriting.

[0128] In a possible implementation, the acquisition module 1401 is further used to generate reference images corresponding to each page according to each page of the marked file; match the marked background image with the reference image corresponding to each page, and obtain the marked page according to the matching result.

[0129] In the device provided in the embodiment of the present application, by adding the annotation handwriting to the annotated page according to the coordinates of the trajectory point, the annotation handwriting can be automatically added to the annotated page, thereby saving the annotation content to the annotated page. Compared with the method of manually adding annotation handwriting to save the annotation content, the efficiency and accuracy of the device in saving the annotation content are higher. Furthermore, after adding the annotation content to the annotated page, if the user needs to edit the annotation content, the user can edit the annotation content by modifying the annotation handwriting in the annotated page, which provides a better user experience.

[0130] It should be understood that the above Figure 13-14 When the device provided realizes its functions, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0131] See also Fig.15 , Fig.15 The following is a schematic diagram of the structure of a computer system provided in an embodiment of the present application. Fig.15 As shown, the computer system is a computer system 2000. The computer system 2000 may be a network device, a routing device or a switching device. Fig.15The computer system 2000 shown is used to execute the above Figure 4 and Fig.12 The operations involved in the electronic whiteboard in the method for saving the annotation content shown in the figure, or executing the above Fig.11 and Fig.12 The operations involved in the terminal in the method for saving the annotation content shown. The computer system 2000 is, for example, a server, etc. The computer system 2000 can be implemented by a general bus architecture.

[0132] like Fig.15 As shown, the computer system 2000 includes at least one processor 2001 , a memory 2003 , and at least one communication interface 2004 .

[0133] Processor 2001 is, for example, a central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the solution of the present application. For example, processor 2001 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can implement or execute various logic blocks, modules, and circuits described in conjunction with the disclosure of the embodiments of the present application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0134] Optionally, the computer system 2000 further includes a bus. The bus is used to transmit information between the components of the computer system 2000. The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0135] The memory 2003 is, for example, a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2003 is, for example, independent and connected to the processor 2001 via a bus. The memory 2003 can also be integrated with the processor 2001.

[0136] The communication interface 2004 uses any transceiver-like device to communicate with other devices or communication networks, and the communication network can be Ethernet, radio access network (RAN) or wireless local area network (WLAN), etc. The communication interface 2004 can include a wired communication interface and can also include a wireless communication interface. Specifically, the communication interface 2004 can be an Ethernet interface, a fast Ethernet (FE) interface, a gigabit Ethernet (GE) interface, an asynchronous transfer mode (ATM) interface, a WLAN interface, a cellular network communication interface or a combination thereof. The Ethernet interface can be an optical interface, an electrical interface or a combination thereof. In an embodiment of the present application, the communication interface 2004 can be used for the computer system 2000 to communicate with other devices.

[0137] In a specific implementation, as an embodiment, the processor 2001 may include one or more CPUs, such as Fig.15 0 and CPU1 shown in FIG. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0138] In a specific implementation, as an embodiment, the computer system 2000 may include multiple processors, such as Fig.15 2001 and processor 2005 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0139] In a specific implementation, as an embodiment, the computer system 2000 may further include an output device and an input device. The output device communicates with the processor 2001 and may display information in a variety of ways. For example, the output device may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor 2001 and may receive user input in a variety of ways. For example, the input device may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0140] In some embodiments, the memory 2003 is used to store the program code 2010 for executing the solution of the present application, and the processor 2001 can execute the program code 2010 stored in the memory 2003. The program code 2010 may include one or more software modules. Optionally, the processor 2001 itself may also store the program code or instructions for executing the solution of the present application.

[0141] In a specific embodiment, the computer system 2000 of the embodiment of the present application may be the electronic whiteboard in each of the above-mentioned method embodiments, and the processor 2001 in the computer system 2000 reads the program code 2010 in the memory 2003 or the program code or instruction stored in the processor 2001 itself, so that Fig.15 The computer system 2000 shown is capable of executing all or part of the operations executed by the electronic whiteboard.

[0142] In a specific embodiment, the computer system 2000 of the embodiment of the present application may be a terminal in each of the above-mentioned method embodiments, and the processor 2001 in the computer system 2000 reads the program code 2010 in the memory 2003 or the program code or instruction stored in the processor 2001 itself, so that Fig.15 The computer system 2000 shown can perform all or part of the operations performed by the terminal.

[0143] The computer system 2000 may also correspond to the above Fig.13 , 14 The device shown, Fig.13 , 14 Each functional module in the device shown is implemented using software of the computer system 2000. In other words, Fig.13 , 14 The functional modules included in the device shown are generated after the processor 2001 of the computer system 2000 reads the program code 2010 stored in the memory 2003 .

[0144] in, Figure 4 , Fig.11 and Fig.12 Each step of the method for preserving the annotation content shown is completed by the hardware integrated logic circuit or software instructions in the processor of the computer system 2000. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0145] Fig.16 is a schematic diagram of the structure of another computer system provided in an embodiment of the present application, the computer system is used to execute the above Figure 4 and Fig.12 The operations involved in the electronic whiteboard in the method for saving the annotation content shown in the figure, or executing the above Fig.11 and Fig.12 Operations involved in the terminal in the method for saving the annotated content shown. Exemplarily, the computer system is a server, and the server may have relatively large differences due to different configurations or performances. The computer system may include one or more processors 1601 and one or more memories 1602, wherein at least one computer program is stored in one or more memories 1602, and at least one computer program is loaded and executed by one or more processors 1601. Exemplarily, processor 1601 is a CPU. Of course, the computer system may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The computer system may also include other components for realizing device functions, which will not be repeated here.

[0146] The embodiment of the present application also provides a computer system, the computer system includes a processor, when the processor executes the program instruction or code, the computer system executes the method for saving the annotation content executed by the electronic whiteboard, or the method for saving the annotation content executed by the terminal. Exemplarily, the computer system also includes a memory, the memory is used to store the program instruction or code.

[0147] In a possible implementation, the computer system further includes: an input interface and an output interface, and the input interface, the output interface, the processor, and the memory are connected via an internal connection path.

[0148] The embodiment of the present application also provides a system for saving the annotation content, which includes an electronic whiteboard and a terminal. The electronic whiteboard is used to execute Figure 4 and Fig.12The method executed by the electronic whiteboard shown in the figure, the terminal is used to execute Fig.11 and Fig.12 The functions of the electronic whiteboard and the terminal of the system can be referred to above. Figure 4 , Fig.11 and Fig.12 The relevant descriptions shown will not be repeated here one by one.

[0149] The embodiment of the present application also provides a communication device, which includes: a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other through an internal connection path, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to control the transceiver to receive signals and control the transceiver to send signals, and when the processor executes the instructions stored in the memory, the processor executes the method for saving the annotation content executed by the electronic whiteboard in the above method embodiment, or executes the method for saving the annotation content executed by the terminal in the above method embodiment.

[0150] It should be understood that the processor may be a CPU, or other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting an advanced RISC machines (ARM) architecture.

[0151] Further, in an optional embodiment, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. The memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0152] The memory may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory. Among them, the nonvolatile memory may be a ROM, a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM) or a flash memory. The volatile memory may be a RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (static RAM, SRAM), dynamic random access memory (dynamic random access memory, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).

[0153] An embodiment of the present application also provides a computer-readable storage medium, in which at least one program instruction or code is stored. When the program instruction or code is loaded and executed by a processor, the computer implements the method for saving the annotation content executed by the electronic whiteboard in the method embodiment, or the method for saving the annotation content executed by the terminal.

[0154] The embodiment of the present application also provides a computer program or a computer program product, which includes: computer program code, when the computer program code is run by a computer, it enables the computer to execute the method for saving the annotation content executed by the electronic whiteboard in the method embodiment, or the method for saving the annotation content executed by the terminal.

[0155] An embodiment of the present application also provides a chip, including a processor, for calling and executing instructions stored in a memory from a memory. When the chip is installed in an electronic whiteboard, the electronic whiteboard executes a method for saving the annotation content executed by the electronic whiteboard in an embodiment of the method. When the chip is installed in a terminal, the terminal executes a method for saving the annotation content executed by the terminal in an embodiment of the method.

[0156] Exemplarily, the chip further includes: an input interface, an output interface and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the memory contains the above-mentioned program instructions or codes.

[0157] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program or a computer program product. A computer program or a computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, a process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, a computer instruction can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0158] In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] The computer program code for realizing the method for the embodiment of the present application can be written in one or more programming languages. These computer program codes can be provided to the processor of the storage device of general-purpose computer, special-purpose computer or other programmable annotation content, so that the program code, when being executed by the storage device of computer or other programmable annotation content, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on the remote computer or completely on the remote computer or server.

[0160] In the context of the embodiments of the present application, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, or it can be an electrical, mechanical or other form of connection.

[0163] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0164] In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0165] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is the quantity and execution order limited. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, the first figure can be referred to as the second figure, and similarly, the second figure can be referred to as the first figure.

[0166] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0167] The term "at least one" in this application means one or more, and the term "multiple" in this application means two or more, for example, multiple range locks means two or more range locks. The terms "system" and "network" are often used interchangeably herein.

[0168] It should be understood that the terms used in the description of various examples herein are only for describing specific examples and are not intended to be limiting. As used in the description of various examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0169] It should also be understood that the term “comprise” (also known as “includes,” “including,” “comprises” and / or “comprising”) when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0170] It should also be understood that, depending on the context, the phrase “if it is determined that…” or “if [stated condition or event] is detected” may be interpreted to mean “upon determining that…” or “in response to determining that…” or “upon detecting [stated condition or event]” or “in response to detecting [stated condition or event]”.

[0171] It should be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and / or other information.

[0172] It should also be understood that the references to "one embodiment", "an embodiment", or "a possible implementation" throughout the specification mean that specific features, structures, or characteristics related to the embodiment or implementation are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment", or "a possible implementation" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0173] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for storing annotation content, characterized in that: The method is applied to an electronic whiteboard, and the method comprises: Acquire the coordinates of the track points of the annotation background image and the annotation handwriting, wherein the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; According to the annotated background image, the annotated handwriting is added to the annotated page according to the track point coordinates.

2. The method according to claim 1, characterized in that The marking handwriting includes at least one of graphic handwriting and text handwriting, and both the graphic handwriting and the text handwriting include at least one track point.

3. The method according to claim 2, characterized in that The marking handwriting includes a plurality of strokes, and the method further includes: Inputting a plurality of strokes included in the annotated handwriting into a graphic-text hybrid classifier, wherein the graphic-text hybrid classifier is used to classify the strokes; Acquire a classification result output by the graphic-text hybrid classifier, wherein the classification result includes at least one of graphic strokes or text strokes; In the case where the classification result includes the graphic strokes, obtaining graphic handwriting according to the graphic strokes; In the case that the classification result includes the character strokes, the character handwriting is obtained according to the character strokes.

4. The method according to claim 3, characterized in that The step of obtaining graphic handwriting according to the graphic strokes comprises: Performing graphic recognition on the graphic strokes, and obtaining graphic handwriting according to the obtained graphic recognition result, wherein the graphic handwriting includes at least one graphic; The step of obtaining text handwriting according to the text strokes includes: The character strokes are subjected to character recognition, and character handwriting is obtained according to the character recognition result, wherein the character handwriting includes at least one character.

5. The method according to any one of claims 2 to 4, characterized in that: In the case where the annotated handwriting includes the graphic handwriting and the text handwriting, before adding the annotated handwriting to the annotated page according to the annotated background image and the track point coordinates, the method further includes: Acquire the position and size of at least one graphic included in the graphic handwriting, and the position and size of at least one character included in the character handwriting; Acquire a first graphic and a first text according to the position and size of the at least one graphic and the position and size of the at least one text; At least one of the position or size of the first graphic is adjusted to obtain the adjusted first graphic, and / or at least one of the position or size of the first text is adjusted to obtain the adjusted first text.

6. The method according to any one of claims 2 to 4, characterized in that: In the case where the annotation handwriting includes the graphic handwriting and the graphic handwriting includes a plurality of graphics, before adding the annotation handwriting to the annotated page according to the annotation background picture and the track point coordinates, the method further includes: Acquire the positions and sizes of multiple graphics included in the graphic handwriting; According to the positions of the plurality of graphics, acquiring a second graphic and a third graphic; At least one of the position or size of the second graphic is adjusted to obtain the adjusted second graphic, and / or at least one of the position or size of the third graphic is adjusted to obtain the adjusted third graphic.

7. The method according to any one of claims 1 to 6, characterized in that: The adding the annotation handwriting to the annotated page according to the annotation background picture and the coordinates of the track points includes: Obtaining a reference image corresponding to the annotated page; Obtaining a scaling ratio according to the size of the annotated background image and the size of the reference image corresponding to the annotated page; Scaling the annotation handwriting according to the scaling ratio to obtain the scaled annotation handwriting; Based on the scaled annotation handwriting, obtaining the coordinates of the trajectory points of the scaled annotation handwriting; According to the coordinates of the track points of the scaled annotation handwriting, the scaled annotation handwriting is added to the annotated page.

8. The method according to any one of claims 1 to 6, characterized in that: The adding the annotation handwriting to the annotated page according to the annotation background picture and the coordinates of the track points includes: The annotated background image and the track point coordinates of the annotated handwriting are sent to the terminal, so that the terminal adds the annotated handwriting to the annotated page in the annotated file saved by the terminal according to the annotated background image and the track point coordinates.

9. The method according to any one of claims 1 to 8, characterized in that: Before adding the annotation handwriting to the annotated page according to the annotation background picture and the track point coordinates, the method further includes: Generate reference images corresponding to the pages according to the pages of the marked file; The annotated background image is matched with reference images corresponding to each of the pages, and the annotated pages are obtained according to the matching results.

10. A method for storing annotation content, characterized in that: The method is applied to a terminal, and the method includes: Receiving a marked background image and coordinates of a track point of a marked handwriting sent by an electronic whiteboard, wherein the marked background image is generated based on a marked page in a marked file, and the marked handwriting is generated according to the marked content of the marked page; According to the annotated background image, the annotated handwriting is added to the annotated page in the annotated file saved by the terminal according to the coordinates of the track points.

11. The method according to claim 10, characterized in that The adding the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotated background picture and the coordinates of the track points includes: Obtaining a reference image corresponding to the annotated page; Obtaining a scaling ratio according to the size of the annotated background image and the size of the reference image corresponding to the annotated page; Scaling the annotation handwriting according to the scaling to obtain the scaled annotation handwriting; Based on the scaled annotation handwriting, obtaining the coordinates of the trajectory points of the scaled annotation handwriting; According to the coordinates of the track points of the scaled annotation handwriting, the scaled annotation handwriting is added to the annotated page.

12. The method according to claim 10 or 11, characterized in that: Before adding the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background picture and the coordinates of the track points, the method further includes: Generate reference images corresponding to the pages according to the pages of the marked file; The annotated background image is matched with reference images corresponding to each of the pages, and the annotated pages are obtained according to the matching results.

13. A device for storing annotation content, characterized in that: The device is applied to an electronic whiteboard, and comprises: An acquisition module, used to acquire the coordinates of the track points of the annotation background image and the annotation handwriting, wherein the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; The adding module is used to add the annotation handwriting to the annotated page according to the annotation background image and the coordinates of the track points.

14. A device for storing annotation content, characterized in that: The device is applied to a terminal, and the device includes: An acquisition module, used for receiving the annotation background image and the coordinates of the trajectory points of the annotation handwriting sent by the electronic whiteboard, wherein the annotation background image is generated based on the annotated page in the annotated file, and the annotation handwriting is generated according to the annotation content of the annotated page; The adding module is used to add the annotation handwriting to the annotated page in the annotated file saved by the terminal according to the annotation background picture and the coordinates of the track points.

15. A system for preserving annotation content, characterized in that: The system comprises an electronic whiteboard and a terminal, wherein the electronic whiteboard is used to execute the method according to any one of claims 1 to 9, and the terminal is used to execute the method according to any one of claims 10 to 12.

16. A computer system, characterized in that: The computer system comprises: a processor, and when the processor executes program instructions or codes, the computer system implements the method according to any one of claims 1 to 12.

17. The computer system according to claim 16, characterized in that: The computer system further comprises: a memory, wherein the memory is used to store the program instructions or codes.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program instruction or code, and when the program instruction or code is loaded and executed by the processor, the computer implements the method according to any one of claims 1 to 12.

19. A chip, characterized in that: The chip includes a processor, which is used to call and execute instructions stored in the memory from the memory. When the chip is installed in an electronic whiteboard, the electronic whiteboard executes the method described in any one of claims 1 to 9. When the chip is installed in a terminal, the terminal executes the method described in any one of claims 10 to 12.

Images