Whiteboard handwriting recognition method, device and electronic equipment

Abstract

The present invention discloses a handwriting recognition method, device and electronic device for a whiteboard. The whiteboard contains at least one object, which is a table or a mind map. The method comprises: receiving a first operation to select a target object in the whiteboard; displaying a toolbar, wherein the toolbar includes a handwriting recognition control; receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting in each unit area in the target object, and obtaining a first recognition text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node; replacing the handwriting displayed in the unit area with the first recognition text corresponding to the unit area, and binding the unit area to the corresponding first recognition text. The present invention solves the technical problem of poor text recognition effect of handwriting in tables or mind maps in the related art.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method, device and electronic equipment for whiteboard handwriting recognition. Background Art

[0002] In a whiteboard application, you can write and recognize handwriting, and you can also draw tables or mind maps in the whiteboard application. However, when writing in a table or mind map, the handwriting exists independently on the whiteboard, resulting in poor overall recognition results.

[0003] Currently, no effective solution has been proposed to the problem that the text recognition effect of handwriting in tables or mind maps in related technologies is poor. Summary of the Invention

[0004] The embodiments of the present invention provide a whiteboard handwriting recognition method, device, and electronic device to at least solve the technical problem in the related art of poor text recognition of handwriting in tables or mind maps.

[0005] According to one aspect of an embodiment of the present invention, a method for handwriting recognition on a whiteboard is provided, wherein the whiteboard contains at least one object, which is a table or a mind map. The method comprises: receiving a first operation to select a target object in the whiteboard; displaying a toolbar, wherein the toolbar includes a handwriting recognition control; receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting in each unit area in the target object, and obtaining a first recognition text corresponding to the unit area, wherein the unit area of ​​the table is a cell, and the unit area of ​​the mind map is a node; replacing the handwriting displayed in the unit area with the first recognition text corresponding to the unit area, and binding the unit area to the corresponding first recognition text.

[0006] According to another aspect of an embodiment of the present invention, a handwriting recognition device for a whiteboard is also provided. The whiteboard contains at least one object, which is a table or a mind map. The handwriting recognition device for the whiteboard includes: a first receiving module for receiving a first operation to select a target object in the whiteboard; a display module for displaying a toolbar, wherein the toolbar includes a handwriting recognition control; a second receiving module for receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting in each unit area in the target object, and obtaining a first recognition text corresponding to the unit area, wherein the unit area of ​​the table is a cell, and the unit area of ​​the mind map is a node; a binding module for replacing the handwriting displayed in the unit area with the first recognition text corresponding to the unit area, and binding the unit area to the corresponding first recognition text.

[0007] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the above method steps.

[0008] According to another aspect of an embodiment of the present invention, an electronic device is provided, including: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0009] Although the related art can perform text recognition and generate tables or mind maps, the recognized text exists independently in the whiteboard and is not related to the table or mind map in which it is located. As a result, it is easy for the recognition result to be incompatible with the table or mind map. For example, the recognized text cannot be accurately displayed in the corresponding cell or node, which is contrary to the original intention of the user to write content in the table or mind map. As a result, the related art has a technical problem of poor text recognition effect of handwriting in tables or mind maps, which in turn limits the application scenarios of text recognition. The above embodiment of the present application receives a first operation to select a target object in a whiteboard; displays a toolbar, wherein the toolbar includes a handwriting recognition control; receives a trigger instruction for the handwriting recognition control, independently recognizes the handwriting of each unit area in the target object, and obtains a first recognized text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node; replaces the handwriting displayed in the unit area with the first recognized text corresponding to the unit area, and binds the unit area to the corresponding first recognized text. The above solution performs text partition recognition based on the cells of a table or the nodes of a mind map, and binds the recognized text to the cell or node to which it belongs, so as to combine the intelligent recognition function with the table and the mind map, solving the technical problem of poor text recognition effect of handwriting in tables or mind maps in related technologies, thereby expanding the application scenarios of intelligent recognition and facilitating the organization of handwriting. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0011] Figure 1 is a flow chart of a whiteboard handwriting recognition method according to an embodiment of the present application;

[0012] Figure 2 This is a schematic diagram of displaying a table and a mind map on a whiteboard application interface according to an embodiment of the present application;

[0013] Figure 3is a schematic diagram of recognizing handwriting in a target object according to an embodiment of the present application;

[0014] Figure 4 This is a schematic diagram of enabling a real-time text recognition function according to an embodiment of the present application;

[0015] Figure 5 is a schematic diagram of an expanded cell according to an embodiment of the present application;

[0016] Figure 6 Yes Figure 5 A schematic diagram of deleting the content in a unit area in ;

[0017] Figure 7 Yes Figure 5 A schematic diagram of moving the content in a unit area in ;

[0018] Figure 8 Yes Figure 5 A schematic diagram of moving and deleting the contents of a unit area in ;

[0019] Figure 9 is a schematic diagram of a handwriting recognition device for a whiteboard according to an embodiment of the present invention;

[0020] Figure 10 2 is a schematic diagram of an intelligent interactive tablet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Example 1

[0024] According to an embodiment of the present invention, an embodiment of a method for handwriting recognition on a whiteboard is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Figure 1 This is a flowchart of a handwriting recognition method for a whiteboard according to an embodiment of the present application, wherein the whiteboard contains at least one object, which is a table or a mind map, such as Figure 1 As shown, the method includes the following steps:

[0026] Step S102: receiving a first operation to select a target object in the whiteboard.

[0027] Specifically, the first operation can be a preset selection operation, such as a box selection operation, a click selection operation, etc. In the above solution, at least one object already exists in the whiteboard application interface, and the user can select the target object by performing the first operation. The target object is the object whose handwriting needs to be converted.

[0028] Figure 2 This is a schematic diagram of displaying a table and a mind map in a whiteboard application interface according to an embodiment of the present application. In this example, the whiteboard application interface includes two objects, a table on the left and a mind map on the right. Both the table and the mind map include generated handwriting. Figure 2 In the first operation, you can select any object as the target object. For example, you can select a table or a mind map as the target object by selecting it in a box, or select both a table and a mind map as the target object.

[0029] Step S104: displaying a toolbar, wherein the toolbar includes a handwriting recognition control.

[0030] After selecting a target object, the toolbar can be displayed directly or through other triggering methods. The toolbar can be displayed at a specified location within the application interface or adaptively based on the location of the target object, for example, below the target object. The toolbar can display multiple controls, including at least a handwriting recognition control. When handwriting recognition is required for the target object, this handwriting recognition control can be triggered and selected.

[0031] Specifically, the handwriting recognition control is used to trigger the start of a handwriting recognition function, which is used to recognize handwriting as text or graphics.

[0032] In an optional embodiment, still combined with Figure 2 As shown in the example, a table in the whiteboard is selected, and a toolbar is displayed below the table. The toolbar includes: edit controls, move controls, handwriting recognition controls, and delete controls. The user can select controls in the toolbar.

[0033] Step S106, receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting of each unit area in the target object, and obtaining a first recognized text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node.

[0034] In the above steps, the handwriting recognition control is selected to trigger the handwriting recognition function. In this embodiment, when performing handwriting recognition, the unit area of ​​the object is independently recognized, that is, the handwriting of each unit area in the target object is recognized as a set of handwriting sets.

[0035] In an optional embodiment, combined with Figure 2 As shown, taking the table therein as an example, the table has nine cells, that is, nine unit areas, of which five cells have handwriting, so the handwriting in the five cells with handwriting is recognized separately, thereby obtaining five groups of first recognition texts corresponding to the five cells.

[0036] In an optional embodiment, combined with Figure 2 As shown, taking the mind map as an example, the mind map has four nodes, namely four unit areas, so the handwriting in the four nodes with handwriting is recognized respectively, thereby obtaining four groups of first recognition texts corresponding to the four nodes.

[0037] It should be noted that handwriting recognition can be achieved through the functions carried by the whiteboard application, or the whiteboard application can package the handwriting and send it to a remote server, which recognizes it and returns the recognition result to the whiteboard application.

[0038] In step S108 , the handwriting displayed in the unit area is replaced with the first recognized text corresponding to the unit area, and the unit area is bound to the corresponding first recognized text.

[0039] After the first recognized text corresponding to the unit area containing the handwriting is identified and obtained, the handwriting in the unit area is replaced with the corresponding first recognized text, so that the target object displays content in text form.

[0040] Figure 3 This is a schematic diagram of identifying handwriting in a target object according to an embodiment of the present application. In an optional embodiment, Figure 3As shown in the example, in this example, a table and a mind map are selected as target objects simultaneously or one after the other. The handwriting in the table and the mind map is recognized on a per-area basis through the above steps. Taking the table as an example, the handwriting in the five cells containing handwriting is recognized as first recognized text, displayed in the corresponding cells, and bound to the cells to which it belongs. Similarly, in the mind map, the handwriting in the four nodes is recognized as first recognized text, displayed in the corresponding nodes, and bound to the nodes to which it belongs.

[0041] Binding the unit area to the corresponding first recognized text means that a certain mapping relationship is established between the unit area and its corresponding first recognized text through binding. Such a mapping relationship can ensure that the first recognized text therein will adapt to certain operations performed on the target object. For example, when the target object is moved, the first recognized text will also move with it, ensuring that the relative position of the first recognized text and its unit area is fixed.

[0042] It should be noted that the first recognized text displayed in the target object is editable, and the user can edit the first recognized text. For example, when the first recognized text is not recognized accurately, the user can edit the first recognized text to correct it.

[0043] It should also be noted that after the first recognized text is displayed in the unit display area, you can continue writing in the target object, so that the recognized text and handwriting can coexist in the target object. After writing again, text recognition can be performed again.

[0044] Although the related art can perform text recognition and generate tables or mind maps, the recognized text exists independently in the whiteboard and is not related to the table or mind map in which it is located. As a result, it is easy for the recognition result to be incompatible with the table or mind map. For example, the recognized text cannot be accurately displayed in the corresponding cell or node, which is contrary to the original intention of the user to write content in the table or mind map. As a result, the related art has a technical problem of poor text recognition effect of handwriting in tables or mind maps, which in turn limits the application scenarios of text recognition. The above embodiment of the present application receives a first operation to select a target object in a whiteboard; displays a toolbar, wherein the toolbar includes a handwriting recognition control; receives a trigger instruction for the handwriting recognition control, independently recognizes the handwriting of each unit area in the target object, and obtains a first recognized text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node; replaces the handwriting displayed in the unit area with the first recognized text corresponding to the unit area, and binds the unit area to the corresponding first recognized text. The above solution performs text partition recognition based on the cells of a table or the nodes of a mind map, and binds the recognized text to the cell or node to which it belongs, so as to combine the intelligent recognition function with the table and the mind map, solving the technical problem of poor text recognition effect of handwriting in tables or mind maps in related technologies, thereby expanding the application scenarios of intelligent recognition and facilitating the organization of handwriting.

[0045] As an optional embodiment, a trigger instruction for a handwriting recognition control is received, and the handwriting of each unit area in the target object is independently recognized to obtain a first recognition text corresponding to the unit area, including: determining that the handwriting belonging to the same unit area in the target object is a group of handwriting sets; recognizing each handwriting set as a whole to obtain a second recognition text corresponding to each handwriting set; obtaining the unit area where the first handwriting point in the handwriting set is located, and determining that the second recognition text corresponding to the handwriting set is the first recognition text corresponding to the unit area where the first handwriting point is located.

[0046] In the above scheme, all handwritings in the target object are divided into handwriting sets according to the unit areas to which they belong. Each handwriting set includes the handwriting data within a unit area. Each handwriting set is uploaded to the recognition module for text recognition. During the recognition process, each handwriting set is treated as a whole, and the corresponding unit area is determined based on the unit area where the first handwriting point in the handwriting set is located.

[0047] In an optional embodiment, the handwriting data of all handwritings in the target object are divided into handwriting sets according to the unit areas to which they belong. The handwriting sets can be handwriting point sequences composed of handwriting data. Then, the identifier of the unit area where the first handwriting point in each handwriting set is located is obtained, and a data packet consisting of the identifier and the handwriting point sequence is uploaded to a remote or local handwriting recognition module. After the handwriting recognition module identifies the second recognition text corresponding to the handwriting point sequence, the unit area corresponding to the second recognition text is found based on the identifier of the unit area carried in the data packet, thereby binding and displaying the two.

[0048] As an optional embodiment, the handwriting displayed in the unit area is replaced with the first recognized text corresponding to the unit area, including one or more of the following: the first recognized text is displayed in the center of the corresponding unit area; when the target object is a table, the text attributes of the first recognized text corresponding to each cell are determined to be preset text attributes; when the target object is a mind map, the text attributes of the first recognized text corresponding to the node are determined according to the level of the node in the mind map.

[0049] In the above solution, the first recognized text can be displayed in the center of its corresponding unit area, still combined with Figure 3 As shown, the first recognized text in the table is displayed in the center of the cell to which it belongs. It should be noted that in the case of non-first recognition, in order to prevent the overlap of the first recognized texts recognized multiple times, the first recognized text will no longer be set to be centered, but the display position will be determined based on the center of the handwriting.

[0050] The above text attributes may include the size, color and font of the text. In the case where the target object is a table, the font, color and size of the first recognized text in each unit area are the same, such as Figure 3 As shown in the table in . As for the mind map, since there is a parent-child relationship between the nodes in the mind map, in order to reflect this relationship, it can be determined according to the level of the node in the mind map structure. For example, the font of the root node is the largest, and its child nodes and grandchild nodes are reduced in sequence. For another example, the first recognized text of the root node is bolded based on the preset font, and the first recognized text of other nodes is not bolded, etc. It can also be as follows Figure 3 As shown, set the text attributes corresponding to each level of nodes to be the same.

[0051] The above solution sets the display mode of the recognition results when performing text recognition in a table or mind map, so that the overall recognition effect is neater and more beautiful, meeting the subsequent handwriting organization needs and improving the user's writing experience.

[0052] As an optional embodiment, the toolbar also includes a recognition undo control. After replacing the handwriting displayed in the unit area with the first recognition text corresponding to the unit area and binding the unit area with the corresponding first recognition text, the above method also includes: receiving a second operation, wherein the second operation is used to trigger the recognition undo control; and restoring the handwriting in the target object.

[0053] Specifically, the above-mentioned second operation can be performed after selecting the target object. When it is necessary to restore the handwriting in the target object, the target object can be selected. At this time, the corresponding toolbar will pop up. This toolbar is different from the toolbar in step S104. The toolbar here includes tools for processing the target object after text recognition. The toolbar includes at least an identification undo control. After the identification undo control is selected through the second operation, the target object can restore the original handwriting.

[0054] In an optional embodiment, still combined with Figure 2 and Figure 3 As shown, after text recognition, Figure 2 The handwriting in the table and mind map is converted into Figure 3 After the text in Figure 3 Select the table and mind map again to bring up the toolbar and trigger the recognition undo control in the toolbar. Figure 3 The tables and mind maps in the file can be restored as follows Figure 2 shown.

[0055] The whiteboard in this embodiment provides a recognition undo control so that the content of the object on the whiteboard can be freely switched between the original handwriting and the recognized text, thereby meeting various needs of users.

[0056] As an optional embodiment, the above method also includes: receiving a third operation, wherein the third operation is used to turn on the real-time text recognition function of the whiteboard; receiving handwriting generated in the target object; performing handwriting recognition on the handwriting generated in the target object to obtain a third recognized text; and displaying the third recognized text in the unit area where the handwriting is located.

[0057] Specifically, the above-mentioned real-time text recognition function is used to realize the function of recognizing text while the user is writing. Figure 4 This is a schematic diagram of enabling the real-time text recognition function according to an embodiment of the present application, combined with Figure 4 As shown, bring up the whiteboard recognition menu, including: Writing Settings, Writing Snapshot, Scan Settings and About; Select Writing Settings to bring up the secondary menu, in which you can turn the real-time text recognition function on or off. Figure 4 In the example, the real-time text recognition function is turned on.

[0058] It should be noted that Figure 4 The method of enabling the real-time text recognition function is only used as an example. The control of the real-time text recognition function can be set in the first-level menu or other levels of menus as needed, or it can be set as a separate shortcut control on the whiteboard application interface. There is no limitation here.

[0059] It should also be noted that before receiving the third operation, if no object is selected, it can be determined to enable the real-time text recognition function for the entire whiteboard, that is, real-time recognition of handwriting written at any position on the whiteboard; before receiving the third operation, a selection operation for any object can also be received to confirm the target object. If the target object is determined based on the selection operation before receiving the third operation, the real-time text recognition function can be enabled only for the selected target object.

[0060] It should be noted that after obtaining the third recognized text, the third recognized text can still be bound to its corresponding unit area to achieve a mapping relationship between the unit area and the third recognized text. The text attributes of the third recognized text can be pre-set attributes or determined based on the attributes of the handwriting. For example, the color of the third recognized text can be the same as the handwriting, the font of the third recognized text can be a preset font, and the size of the third recognized text can be similar to the size of the handwriting.

[0061] After the third operation is received and the text real-time recognition function is turned on, the above solution can recognize the handwriting in real time while receiving the handwriting, without waiting for the user to trigger the handwriting recognition control in step S104 for unified recognition.

[0062] As an optional embodiment, handwriting recognition is performed on the handwriting generated in the target object to obtain a third recognized text, including: starting a timer when it is detected that the writing stops to obtain the stop time; when the stop time reaches a preset duration, determining that the current text recognition time has arrived; and recognizing the handwriting generated between the previous text recognition time and the current text recognition time to obtain the third recognized text.

[0063] When performing real-time text recognition, it is necessary to determine the timing of recognition. When a user writes, there is usually a short pause after writing a character or a word, and the writing between a character or a phrase is relatively coherent. Based on this assumption, the above scheme determines the timing of recognition based on the detected writing stop time. Specifically, the timing is started every time the writing stop is detected. When the stop time reaches the preset time, it is considered that the user has completed a stage of writing, and the writing of this stage can be uniformly recognized as a handwriting set.

[0064] In an optional embodiment, while a user is writing on a whiteboard, the whiteboard continuously detects handwriting data. When the first "down" signal is detected, it is determined that the writing medium has started writing, and this time is determined to be the first text recognition moment. When the "up" signal is detected, it is determined that the writing medium has left the whiteboard and writing has stopped, and a timer is started to determine the stop time. If the stop time has not reached a preset time length and the "down" signal is detected again, text recognition is not performed and handwriting point detection continues. If the stop time reaches the preset time length, it is determined that the second text recognition moment has arrived, and text recognition is performed on the handwriting data generated between the first and second text recognition moments, and so on.

[0065] As an optional embodiment, displaying the third recognized text in the unit area where the handwriting is located includes: obtaining the center of the handwriting; and displaying the third recognized text in the unit area where the handwriting is located with the center of the handwriting as the center.

[0066] In the scenario of real-time text recognition, if the third recognized text recognized each time is displayed in the middle of the unit area, it is easy to cause the handwriting and the previous third recognized text to overlap when writing again. Therefore, in the scenario of real-time text recognition, the display position of the third recognized text can be determined according to the display position of the handwriting. More specifically, the third recognized text of the handwriting can be displayed with the center of the handwriting as the center.

[0067] As an optional embodiment, after displaying the third recognized text in the unit area where the handwriting is located, the above method also includes: detecting that the third recognized text exceeds the designated unit area where the third recognized text is located; and expanding the designated unit area where the third recognized text is located.

[0068] In the above solution, since the third recognized text exceeds the designated unit area where it is located, that is, the designated unit area is smaller than the minimum circumscribed rectangle of the third recognized text, the designated unit area is expanded.

[0069] In an optional embodiment, taking the designated unit area as a cell in a table as an example, when the cell is small and there are many third recognized texts, the cell can be expanded horizontally. During the expansion, the cells in the same column as the cell are also expanded horizontally so that the cells in this column are always aligned; when the cell is expanded horizontally to a preset specified width, if it is still smaller than the minimum enclosing rectangle of the third recognized text, the cell can be expanded vertically. Similarly, during the expansion, the cells in the same row as the cell are also expanded vertically so that the cells in this row are always aligned.

[0070] In another optional embodiment, taking the designated unit area as a node in a mind map as an example, when the node is small and contains a lot of third-recognized text, the node can be expanded horizontally. During the expansion, the node's sibling nodes are also expanded horizontally, so that the sibling nodes at this level are always aligned; and the child nodes at this level are all moved in the direction of expansion. When the node is small and contains a lot of third-recognized text, the node can also be expanded vertically. During the expansion, the node's sibling nodes are moved in the direction of expansion, so that a certain distance is maintained between the designated node and its sibling nodes.

[0071] In the above solution, when the real-time text recognition function is turned on, the text is recognized while writing in the table or mind map, and expansion and adaptation are performed at the same time. For example, if a piece of handwriting spans multiple table areas and is recognized as a whole paragraph of third-recognized text, the third-recognized text belongs to the cell where the first handwriting point is located, and it is expanded and adapted in the cell where the first handwriting point is located. After deleting the third-recognized text, the table or mind map can also be adapted.

[0072] As an optional embodiment, the above method also includes: receiving a writing operation, and detecting whether the handwriting generated by the writing operation exceeds the unit area to which the handwriting belongs; if the handwriting exceeds the unit area to which the handwriting belongs, expanding the unit area to which the handwriting belongs.

[0073] The above-mentioned writing operation can be a writing operation generated in the target object at any time. In the case where the real-time text recognition function is not enabled, when the handwriting exceeds the unit area, the unit area can also be expanded using a method similar to the above method so that the handwriting is completely displayed within the unit area to which it belongs.

[0074] Figure 5 This is a schematic diagram of an expanded cell according to an embodiment of the present application, combined with Figure 5 As shown in the example, the cells in the top two rows of the table are of default size. Since the handwriting in the lower right cell exceeds the cell, the cell is expanded downward to fully display the handwriting within the cell to which it belongs. It is important to note that when the first handwriting point belongs to a unit area, the handwriting is confirmed to belong to that unit area.

[0075] As an optional embodiment, when the handwriting exceeds the unit area to which the handwriting belongs, after expanding the unit area to which the handwriting belongs, the above method also includes: receiving a fourth operation, wherein the fourth operation is used to move or delete the initial content in the unit area to which the handwriting belongs to obtain the adjusted content; when the minimum enclosing rectangle of the adjusted content is the same size as the minimum enclosing rectangle of the initial content, it is prohibited to adjust the size of the unit area to which the handwriting belongs; when the minimum enclosing rectangle of the adjusted content is different in size from the minimum enclosing rectangle of the initial content, the target object is rearranged according to the minimum enclosing rectangle of the adjusted content.

[0076] The fourth operation is used to move or delete the content in the target object, where the content may include handwriting in the target object or the first recognized text in the target object. After the content of a unit area in the target object is moved or deleted, if the unit area can still match the adjusted content, that is, the minimum bounding rectangle of the adjusted content is the same as the minimum bounding rectangle of the initial content, then there is no need to adjust the size of the unit area. However, if the size of the unit area cannot match the adjusted content, in this case, it is necessary to adjust the size of the unit area and adjust the sizes of other unit areas accordingly, that is, to rearrange the target object.

[0077] Specifically, the above-mentioned rearrangement is used to indicate adjusting the size of at least one unit area in the target object so that the unit area is adapted to the adjusted content therein.

[0078] Figure 6 Yes Figure 5 A schematic diagram of deleting the contents of a unit area in Figure 6 As shown, the first recognized text in the lower right corner cell is deleted. At this time, the size of the minimum bounding rectangle of the cell does not change, so the size of the cell will not be changed, and thus the entire target object will not be rearranged.

[0079] Figure 7 Yes Figure 5 A schematic diagram of moving the content in a unit area in the Figure 7 As shown, the first recognized text in the lower right corner cell is moved. At this time, the size of the minimum bounding rectangle of the cell changes, so the size of the cell is adaptively changed, and the entire target object is adaptively rearranged.

[0080] Figure 8 Yes Figure 5 A schematic diagram of moving and deleting the contents of a unit area in Figure 8As shown, the first recognized text in the lower right corner cell is deleted, and one of the handwritings is moved upward. At this time, the minimum enclosing rectangle of the cell is reduced, so the size of the cell is adaptively reduced, and the entire target object is adaptively rearranged.

[0081] As an optional embodiment, when receiving the third operation to turn on the real-time text recognition function of the whiteboard, the above method also includes: receiving the fifth operation, wherein the fifth operation is used to turn on the real-time chart recognition function of the whiteboard; before performing handwriting recognition on the handwriting generated in the target object to obtain the third recognized text, the method also includes: performing chart recognition on the handwriting in the whiteboard; when the handwriting is recognized as a preset graphic, the preset graphic is displayed on the whiteboard; when the handwriting is not recognized as a preset graphic, text recognition is performed on the handwriting generated on the whiteboard.

[0082] Specifically, the method of performing graphic recognition on handwriting can be to obtain the similarity between the handwriting and the preset image. When it is determined that the similarity between the handwriting and the preset image is greater than a preset value, the handwriting is determined to be recognized as a preset graphic, otherwise the step of performing text recognition on the handwriting is entered.

[0083] In the above scheme, when the real-time text recognition function and the real-time chart recognition function of the whiteboard are both turned on, the handwriting is prioritized for chart recognition, that is, first analyze whether the handwriting is used to draw a preset graphic. If the handwriting is recognized as a preset image, the handwriting is replaced with its corresponding preset image. If it is not recognized as a preset graphic, text recognition is performed on it.

[0084] In an optional embodiment, combined with Figure 4 As shown, the secondary menu also includes a chart drawing control. When the text recognition control and the chart drawing control are turned on at the same time, when performing handwriting recognition, the recognition of the preset graphics is given priority. When it is determined that the handwriting is not a preset image, text recognition is performed.

[0085] Example 2

[0086] According to an embodiment of the present invention, an embodiment of a handwriting recognition device for a whiteboard is provided. Figure 9 Schematic diagram of a handwriting recognition device for a whiteboard according to an embodiment of the present invention, wherein the whiteboard includes at least one object, which is a table or a mind map, such as Figure 9 As shown, the device includes:

[0087] A first receiving module 90 is configured to receive a target object selected on the whiteboard by a first operation;

[0088] A display module 92 is used to display a toolbar, wherein the toolbar includes a handwriting recognition control;

[0089] The second receiving module 94 is configured to receive a trigger instruction for the handwriting recognition control, independently recognize the handwriting of each unit area in the target object, and obtain a first recognized text corresponding to the unit area, wherein the unit area of ​​a table is a cell and the unit area of ​​a mind map is a node;

[0090] The binding module 96 is configured to replace the handwriting displayed in the unit area with the first recognized text corresponding to the unit area, and bind the unit area to the corresponding first recognized text.

[0091] As an optional embodiment, the second receiving module includes: a first determination submodule, used to determine that the handwriting belonging to the same unit area within the target object is a group of handwriting sets; a first recognition submodule, used to recognize each handwriting set as a whole to obtain the second recognition text corresponding to each handwriting set; a second determination submodule, used to obtain the unit area where the first handwriting point in the handwriting set is located, and determine that the second recognition text corresponding to the handwriting set is the first recognition text corresponding to the unit area where the first handwriting point is located.

[0092] As an optional embodiment, the binding module is used to perform one or more of the following: centering the first recognized text in the corresponding unit area; when the target object is a table, determining that the text attributes of the first recognized text corresponding to each cell are preset text attributes; when the target object is a mind map, determining the text attributes of the first recognized text corresponding to the node according to the level of the node in the mind map.

[0093] As an optional embodiment, the toolbar also includes an identification undo control, and the above-mentioned device also includes: a third receiving module, used to receive a second operation after replacing the handwriting displayed in the unit area with the first recognition text corresponding to the unit area and binding the unit area with the corresponding first recognition text, wherein the second operation is used to trigger the identification undo control; a recovery module, used to restore the handwriting in the displayed target object.

[0094] As an optional embodiment, the above-mentioned device also includes: a fourth receiving module for receiving a third operation, wherein the third operation is used to turn on the real-time text recognition function of the whiteboard; a fifth receiving module for receiving handwriting generated in the target object; a real-time recognition module for performing handwriting recognition on the handwriting generated in the target object to obtain a third recognized text; and a second display module for displaying the third recognized text in the unit area where the handwriting is located.

[0095] As an optional embodiment, the real-time recognition module includes: a timing submodule, which is used to start timing when writing stops and obtain the stop time; a third determination submodule, which is used to determine that the current text recognition time has arrived when the stop time reaches a preset duration; and a second recognition submodule, which is used to recognize the handwriting generated between the previous text recognition time and the current text recognition time and obtain a third recognized text.

[0096] As an optional embodiment, the second display module includes: an acquisition submodule for acquiring the center of the handwriting; and a display submodule for displaying the third recognized text in the unit area where the handwriting is located, with the center of the handwriting as the center.

[0097] As an optional embodiment, the above-mentioned device also includes: a detection module, which is used to detect that the third recognized text exceeds the designated unit area where the third recognized text is located after the third recognized text is displayed in the unit area where the handwriting is located; and a first expansion module, which is used to expand the designated unit area where the third recognized text is located.

[0098] As an optional embodiment, the above-mentioned device also includes: a sixth receiving module, used to receive a writing operation and detect whether the handwriting generated by the writing operation exceeds the unit area to which the handwriting belongs; a second expansion module, used to expand the unit area to which the handwriting belongs when the handwriting exceeds the unit area to which the handwriting belongs.

[0099] As an optional embodiment, the above-mentioned device also includes: a seventh receiving module, which is used to receive a fourth operation after expanding the unit area to which the handwriting belongs when the handwriting exceeds the unit area to which the handwriting belongs, wherein the fourth operation is used to move or delete the initial content in the unit area to which the handwriting belongs to obtain the adjusted content; a prohibition module, which is used to prohibit adjusting the size of the unit area to which the handwriting belongs when the minimum enclosing rectangle of the adjusted content is the same as the minimum enclosing rectangle of the initial content; and a layout module, which is used to rearrange the target object according to the minimum enclosing rectangle of the adjusted content when the minimum enclosing rectangle of the adjusted content is different from the minimum enclosing rectangle of the initial content.

[0100] As an optional embodiment, the above-mentioned device also includes: an eighth receiving module, which is used to receive the fifth operation when receiving the third operation to turn on the real-time text recognition function of the whiteboard, wherein the fifth operation is used to turn on the real-time chart recognition function of the whiteboard; a third recognition module, which is used to perform handwriting recognition on the handwriting generated in the target object and perform chart recognition on the handwriting in the whiteboard before obtaining the third recognized text; a third display module, which is used to display the preset graphic on the whiteboard when the handwriting is recognized as a preset graphic; and a third recognition module, which is used to perform text recognition on the handwriting generated on the whiteboard when the handwriting is not recognized as a preset graphic.

[0101] Example 3

[0102] According to an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps described in Example 1.

[0103] Example 4

[0104] According to an embodiment of the present invention, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps described in Example 1.

[0105] The electronic device can be a smart interactive tablet. Figure 10 This is a schematic diagram of a smart interactive tablet provided by an embodiment of the present application, which includes the above-mentioned interactive device body and touch frame, combined with Figure 10 As shown, the smart interactive tablet 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0106] The communication bus 1002 is used to implement the connection and communication between these components.

[0107] The user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0108] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0109] The processor 1001 may include one or more processing cores. The processor 1001 utilizes various interfaces and circuits to connect the various components within the smart interactive tablet 1000. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by accessing data stored in the memory 1005, the processor 1001 executes various functions and processes data for the smart interactive tablet 1000. Optionally, the processor 1001 may be implemented using at least one hardware form factor selected from the group consisting of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 1001 and may be implemented as a separate chip.

[0110] Among them, the memory 1005 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 10 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an operating application of the smart interactive tablet.

[0111] exist Figure 10In the smart interactive tablet 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call the operating application of the smart interactive tablet stored in the memory 1005, and specifically perform any one of the operations in Example 1.

[0112] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0116] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0118] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A handwriting recognition method for a whiteboard, characterized in that: The whiteboard includes at least one object, which is a table or a mind map. The handwriting recognition method of the whiteboard includes: receiving a first operation to select a target object in the whiteboard; Displaying a toolbar, wherein the toolbar includes a handwriting recognition control; receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting of each unit area in the target object, and obtaining a first recognized text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node; replacing the handwriting displayed in the unit area with the first recognized text corresponding to the unit area, and binding the unit area to the corresponding first recognized text; receiving a writing operation and detecting whether handwriting generated by the writing operation exceeds a unit area to which the handwriting belongs; and expanding the unit area to which the handwriting belongs if the handwriting exceeds the unit area to which the handwriting belongs; Receive a fourth operation, wherein the fourth operation is used to move or delete the initial content in the unit area to which the handwriting belongs to obtain adjusted content; when the minimum enclosing rectangle of the adjusted content is the same size as the minimum enclosing rectangle of the initial content, prohibit adjusting the size of the unit area to which the handwriting belongs; when the minimum enclosing rectangle of the adjusted content is different from the minimum enclosing rectangle of the initial content, rearrange the target object according to the minimum enclosing rectangle of the adjusted content.

2. The method according to claim 1, characterized in that Receiving a trigger instruction for the handwriting recognition control, independently recognizing the handwriting of each unit area in the target object, and obtaining a first recognized text corresponding to the unit area, including: Determining handwritings belonging to the same unit area within the target object as a group of handwriting sets; Recognizing each of the handwriting sets as a whole to obtain a second recognized text corresponding to each of the handwriting sets; The unit area where the first handwriting point in the handwriting set is located is obtained, and the second recognized text corresponding to the handwriting set is determined to be the first recognized text corresponding to the unit area where the first handwriting point is located.

3. The method according to claim 1, characterized in that Replacing the handwriting displayed in the unit area with the first recognized text corresponding to the unit area includes one or more of the following: Displaying the first recognized text in the center of the corresponding unit area; In a case where the target object is the table, determining that the text attributes of the first recognized text corresponding to each cell are all preset text attributes; In a case where the target object is the mind map, the text attribute of the first recognized text corresponding to the node is determined according to the level of the node in the mind map.

4. The method according to claim 1, wherein The toolbar further includes a recognition undo control, which replaces the handwriting displayed in the unit area with the first recognized text corresponding to the unit area. After binding the unit area with the corresponding first recognized text, the method further includes: receiving a second operation, wherein the second operation is used to trigger the recognition revocation control; The handwriting in the target object is restored and displayed.

5. The method according to claim 1, wherein The method further comprises: receiving a third operation, wherein the third operation is used to enable a real-time text recognition function of the whiteboard; receiving handwriting generated in the target object; performing handwriting recognition on the handwriting generated in the target object to obtain a third recognized text; The third recognized text is displayed in the unit area where the handwriting is located.

6. The method according to claim 5, characterized in that Performing handwriting recognition on the handwriting generated in the target object to obtain a third recognized text includes: When the writing stops, the timing is started to obtain the stop time; When the stop time reaches a preset time, determining that the current text recognition time has arrived; The handwriting generated between the last text recognition moment and the current text recognition moment is recognized to obtain the third recognized text.

7. The method according to claim 5, characterized in that Displaying the third recognized text in the unit area where the handwriting is located includes: Obtaining the center of the handwriting; The third recognized text is displayed in the unit area where the handwriting is located, with the center of the handwriting as the center.

8. The method according to claim 5, characterized in that After displaying the third recognized text in the unit area where the handwriting is located, the method further includes: detecting that the third recognized text exceeds the designated unit area where the third recognized text is located; The designated unit area where the third recognized text is located is expanded.

9. The method according to claim 5, characterized in that In the case where the third operation is received to enable the real-time text recognition function of the whiteboard, the method further includes: receiving a fifth operation, wherein the fifth operation is used to enable the real-time chart recognition function of the whiteboard; Before performing handwriting recognition on the handwriting generated in the target object to obtain the third recognized text, the method further includes: Performing chart recognition on the handwriting on the whiteboard; In the case where the handwriting is recognized as a preset graphic, displaying the preset graphic on the whiteboard; In the case that the handwriting is not recognized as the preset graphic, text recognition is performed on the handwriting generated on the whiteboard.

10. A handwriting recognition device for a whiteboard, characterized in that: The whiteboard includes at least one object, which is a table or a mind map, wherein the handwriting recognition device of the whiteboard includes: A first receiving module is configured to receive a first operation to select a target object on the whiteboard; A display module, configured to display a toolbar, wherein the toolbar includes a handwriting recognition control; a second receiving module, configured to receive a trigger instruction for the handwriting recognition control, independently recognize the handwriting in each unit area in the target object, and obtain a first recognized text corresponding to the unit area, wherein the unit area of ​​the table is a cell and the unit area of ​​the mind map is a node; A binding module, configured to replace the handwriting displayed in the unit area with the first recognized text corresponding to the unit area, and bind the unit area to the corresponding first recognized text; The device further includes: a sixth receiving module and a second expansion module, wherein the sixth receiving module is configured to receive a writing operation and detect whether handwriting generated by the writing operation exceeds the unit area to which the handwriting belongs; and the second expansion module is configured to expand the unit area to which the handwriting belongs if the handwriting exceeds the unit area to which the handwriting belongs; The device also includes: a seventh receiving module, a prohibition module and a layout module, the seventh receiving module is used to receive a fourth operation after expanding the unit area to which the handwriting belongs when the handwriting exceeds the unit area to which the handwriting belongs, wherein the fourth operation is used to move or delete the initial content in the unit area to which the handwriting belongs to obtain adjusted content; the prohibition module is used to prohibit adjusting the size of the unit area to which the handwriting belongs when the minimum bounding rectangle of the adjusted content is the same as the minimum bounding rectangle of the initial content; the layout module is used to rearrange the target object according to the minimum bounding rectangle of the adjusted content when the minimum bounding rectangle of the adjusted content is different from the minimum bounding rectangle of the initial content.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, the computer program being adapted to be loaded by the processor and executing the method steps according to any one of claims 1 to 9.

Images