Display device and graph display method thereof

Abstract

The present invention discloses a display device and a graph display method thereof. The display device includes a display screen and a control circuit, the control circuit includes a processor and a memory, the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and execute the following steps: recognize handwriting trajectory information in the display area of ​​the display screen, obtain a data recognition result, and in response to a user's graph drawing command, determine a graph type corresponding to the graph drawing command, and in response to the user's graph drawing command, draw a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and display the drawn graph in the display area. According to the present invention, table data written by a user is recognized, and the table data is drawn into a graph, which is then displayed on an electronic whiteboard, thereby providing an automatic graph drawing method and accurately presenting the rules of change in a data set to improve the interactive experience of smart meetings.

Description

Technical Field

[0004] ,

[0003] , , , , , ,

[0005] ,

[0001] [Cross - reference to Related Applications] This application claims priority based on a Chinese patent application filed with the Chinese Patent Office on July 20, 2021, with an application number of 202110819109.8 and an invention title of "Electronic Whiteboard and Its Graph Display Method", and incorporates all of its disclosures herein.

[0002] The present invention relates to the field of man - machine interaction technology, and particularly to a display device and its graph display method.

Background Art

[0003] Meetings are one of the most common forms in work. However, with the development of the times and the progress of science and technology, the traditional meeting model no longer suits the needs of modern affairs, and the importance of meeting efficiency and interactive experience is gradually emerging in corporate meetings. Therefore, smart meetings with various functions such as interactive smart whiteboards, which integrate various devices such as electronic whiteboards, computers, and remote meeting terminals, have been proposed.

[0004] With the progress of high - precision positioning technologies such as infrared, electromagnetic, and capacitive touch, electronic whiteboards can now support touch note - taking. However, when taking notes using an electronic whiteboard, it is necessary to present the data set on the electronic whiteboard in graph form. Currently, one method is the hand - drawn graph method, which is not only inefficient but also difficult to accurately present the change rules of the data set, giving unnecessary variations to the later - stage analysis. Another method is to project the graph created by a computer onto the electronic whiteboard, which is cumbersome to operate and not suitable for meeting interaction, resulting in a poor meeting interactive experience.

Summary of the Invention

Problems to be Solved by the Invention

[0005] This invention provides a display device and a method for displaying the same graph, thereby recognizing table data written by a user, graphing the table data and displaying it on the display device, providing an automatic graph drawing method, and accurately presenting the laws of change in the dataset to improve the interactive experience of smart meetings. [Means for solving the problem]

[0006] In a first embodiment, the display device provided by the embodiment of the present invention comprises a display screen and a control circuit, The aforementioned display screen is configured to display content, The control circuit comprises a processor and memory, the memory is configured to store a program that the processor can execute, and the processor is configured to read the program from the memory and execute the next step. The handwriting trajectory information in the display area of ​​the aforementioned display screen is recognized, and the data recognition result is obtained. In response to a user's graph drawing command, the system determines the graph type corresponding to the graph drawing command. In response to a user's graph drawing command, the system draws a graph of the graph type corresponding to the graph drawing command based on the data recognition results, and displays the drawn graph in the display area.

[0007] In a second embodiment, the graph display method provided by the embodiment of the present invention is: The steps include: recognizing the handwriting trajectory information in the display area of ​​the aforementioned display screen and obtaining the data recognition result; The steps include: determining the graph type corresponding to the graph drawing command in response to the user's graph drawing command; The system includes the steps of: responding to a user's graph drawing command, drawing a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and displaying the drawn graph in the display area.

[0008] In a third embodiment, the graph display device further provided by the embodiments of the present invention is: A recognition unit configured to recognize handwriting trajectory information in the display area of ​​the aforementioned display screen and acquire data recognition results, A determination unit configured to determine the graph type corresponding to a graph drawing command in response to a user's graph drawing command, The system includes a drawing unit configured to respond to a user's graph drawing command, draw a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and display the drawn graph in the display area.

[0009] In a fourth embodiment, a text recognition model further provided by embodiments of the present invention comprises multiple RNN network layers, wherein at least two outputs are coupled to the inputs of any RNN network layer except the first two RNN network layers. The at least two outputs include outputs from any two RNN network layers prior to any of the aforementioned RNN network layers.

[0010] In a fifth embodiment, the automatic grid projection method further provided by embodiments of the present invention is: The steps include acquiring handwriting trajectory information for the display area of ​​the display screen, The steps include determining each row separator and each column separator based on the first projection histogram on the first coordinate axis of the handwriting trajectory information and the second projection histogram on the second coordinate axis of the handwriting trajectory information, The method includes the step of determining each cell based on each row separator and each column separator.

[0011] In a sixth embodiment, an electronic device further provided by an embodiment of the present invention comprises a processor and memory, the memory being configured to store a program executable by the processor, and the processor being configured to read the program from the memory and perform the steps of the method according to the first, fourth, or fifth embodiment.

[0012] In a seventh embodiment, an embodiment of the present invention further provides a computer storage medium for storing a computer program, which is configured to perform steps of the method according to the first, fourth, or fifth embodiment when the program is executed by a processor.

[0013] These or other embodiments of the present invention will become clearer in the following description of the embodiments. [Brief explanation of the drawing]

[0014] To more clearly illustrate the technical aspects of the embodiments of the present invention, the drawings that will be used in the description of the embodiments are briefly described below. The drawings in the following description represent only a few embodiments of the present invention, and it will be apparent to those skilled in the art that other drawings can be obtained without any creative effort. [Figure 1] This is a configuration diagram of a display device provided by an embodiment of the present invention. [Figure 2A] This figure shows the display area of ​​a first type of electronic whiteboard provided by an embodiment of the invention. [Figure 2B] This figure shows the display area of ​​a second type of electronic whiteboard provided by an embodiment of the invention. [Figure 2C] This figure shows the display area of ​​a third type of electronic whiteboard provided by an embodiment of the invention. [Figure 2D] This figure shows the display area of ​​a fourth type of electronic whiteboard provided by an embodiment of the invention. [Figure 3] This is a schematic diagram of the handwriting trajectory recognition provided by an embodiment of the invention. [Figure 4] This is a schematic diagram for displaying a line graph provided by an embodiment of the invention. [Figure 5] This is a schematic diagram for displaying a histogram provided by an embodiment of the invention. [Figure 6] This is a schematic diagram for displaying a pie chart provided by an embodiment of the invention. [Figure 7A] It is a schematic diagram for displaying the first type of note area and graph area provided by an embodiment of the invention. [Figure 7B] It is a schematic diagram for displaying the second type of note area and graph area provided by an embodiment of the invention. [Figure 7C] It is a schematic diagram for displaying the third type of note area and graph area provided by an embodiment of the invention. [Figure 7D] It is a schematic diagram for displaying the fourth type of note area and graph area provided by an embodiment of the invention. [Figure 7E] It is a schematic diagram for displaying the fifth type of note area and graph area provided by an embodiment of the invention. [Figure 8A] It is a diagram showing a comparative example of the change of the first type of note content provided by an embodiment of the invention. [Figure 8B] It is a diagram showing a comparative example of the change of the second type of note content provided by an embodiment of the invention. [Figure 9A] It is a schematic diagram of a line graph comparison display provided by an embodiment of the invention. [Figure 9B] It is a schematic diagram of a histogram comparison display provided by an embodiment of the invention. [Figure 9C] It is a schematic diagram of a pie chart comparison display provided by an embodiment of the invention. [Figure 10] It is a flowchart of the implementation of the graph drawing method provided by an embodiment of the invention. [Figure 11] It is a flowchart of the implementation for drawing a line graph provided by an embodiment of the invention. [Figure 12] It is a flowchart of the implementation for drawing a histogram provided by an embodiment of the invention. [Figure 13] It is a flowchart of the implementation for drawing a pie chart provided by an embodiment of the invention. [Figure 14]This is a flowchart illustrating a method for displaying a graph in a writing scene having table lines, as provided by an embodiment of the invention. [Figure 15A] This is a schematic diagram of a guide template for displaying a bar graph provided by an embodiment of the invention. [Figure 15B] This figure shows an editing interface for displaying a bar graph provided by an embodiment of the invention. [Figure 15C] This figure shows a writing interface for displaying a bar graph, provided by an embodiment of the present invention. [Figure 15D] This figure shows a full-screen interface for displaying a bar graph provided by an embodiment of the present invention. [Figure 15E] This figure shows a full-screen interface for displaying a bar graph provided by an embodiment of the invention. [Figure 16A] This figure shows a guide template for displaying a radar map provided by an embodiment of the invention. [Figure 16B] This figure shows an editing interface for displaying a radar map provided by an embodiment of the invention. [Figure 16C] This figure shows a writing interface for displaying a radar map, provided by an embodiment of the invention. [Figure 16D] This figure shows a writing interface for displaying a radar map, provided by an embodiment of the invention. [Figure 16E] This figure shows a full-screen interface for displaying a radar map provided by an embodiment of the invention. [Figure 17A] This figure shows a guide template for displaying a pie chart provided by an embodiment of the invention. [Figure 17B] This figure shows an editing interface for displaying a pie chart provided by an embodiment of the invention. [Figure 17C] This figure shows a writing interface for displaying a pie chart, provided by an embodiment of the invention. [Figure 17D]This figure shows an update interface for displaying a pie chart provided by an embodiment of the invention. [Figure 17E] This figure shows a full-screen interface for displaying a pie chart, provided by an embodiment of the present invention. [Figure 18A] This figure shows a guide template for a line graph display provided by an embodiment of the present invention. [Figure 18B] This figure shows the editing interface for a line graph display provided by an embodiment of the present invention. [Figure 18C] This figure shows the writing interface for a display line graph provided by an embodiment of the present invention. [Figure 18D] This figure shows a full-screen interface for a line graph display provided by an embodiment of the present invention. [Figure 19] This is an implementation flowchart of the graph display method in a writing scene that displays a title prompt, as provided by an embodiment of the present invention. [Figure 20] This is a schematic diagram of writing a line graph in a writing scene displaying a title prompt, as provided by an embodiment of the present invention. [Figure 21] This is a schematic diagram of the matrix partitioning policy provided by embodiments of the present invention. [Figure 21A] This is a projection histogram of the X-axis provided by an embodiment of the present invention. [Figure 21B] This is a projected X-axis histogram after reduction mapping, provided by an embodiment of the present invention. [Figure 21C] This is a projection histogram of the Y-axis provided by an embodiment of the present invention. [Figure 21D] This is a projected histogram of the Y-axis after reduction mapping, provided by an embodiment of the present invention. [Figure 21E] This is a projection histogram of the Y-axis provided by an embodiment of the present invention. [Figure 21F] This is a projected histogram of the Y-axis after reduction mapping, provided by an embodiment of the present invention. [Figure 21G] This is a schematic diagram of matrix partitioning provided by an embodiment of the present invention. [Figure 21H] This is a schematic diagram of matrix partitioning provided by an embodiment of the present invention. [Figure 22] This is a flowchart for implementing a first type of matrix partitioning policy provided by an embodiment of the present invention. [Figure 23] This is a flowchart for implementing a second type of matrix partitioning policy provided by an embodiment of the present invention. [Figure 24] This is a schematic diagram of a text recognition model provided by an embodiment of the present invention. [Figure 25] This is an implementation flowchart of the graph display method provided by an embodiment of the present invention. [Figure 26] This is a schematic diagram of a graph display device provided by an embodiment of the present invention. [Figure 27] This is an electronic device provided by an embodiment of the present invention. [Modes for carrying out the invention]

[0015] Hereinafter, the technical solutions of the embodiments of the present invention will be clearly and completely described with reference to the drawings of the embodiments, in order to further clarify the object, technical solutions and advantages of the embodiments of the present invention. It is clear that the embodiments described are part of the present invention and not all embodiments. Furthermore, embodiments and features in embodiments of the present invention may be combined without conflict. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without requiring creative work are within the scope of the protection of the present invention.

[0016] In embodiments of the present invention, the term "and / or" describes the relationship between related objects. For example, "and / or" can represent three cases: A existing alone, both A and B existing, or B existing alone. The letter " / " generally indicates that the preceding and succeeding related objects are in an "or" relationship.

[0017] The application scenarios described in the embodiments of the present invention are provided to more clearly illustrate the technical solutions of the embodiments of the present invention and are not limited to the technical solutions provided by the present invention. Those skilled in the art will understand that as new application scenarios arise, the embodiments provided by the embodiments of this disclosure are equally applicable to similar technical problems. Herein, in the description of the present invention, unless otherwise specified, “multiple” means two or more.

[0018] Smart conferencing is an office product that integrates various equipment such as electronic whiteboards, computers, and remote conferencing terminals, and is equipped with diverse functions such as interactive smart whiteboards. With the rise of smart conferencing, the importance of meeting efficiency and interactive experiences in corporate meetings has become apparent. How to enhance the efficiency and interactive experience of smart conferencing has also become a matter of interest. With advances in high-precision positioning technologies such as infrared, electromagnetic, and capacitive touch, electronic whiteboards can now be used for touch writing. However, when using electronic whiteboards, it is often necessary to present the dataset on the electronic whiteboard in graph format. Currently, this requires hand-drawn graphs, which are not only inefficient but also make it difficult to accurately present the rules of change in the dataset, introducing unnecessary variability in later analysis. On the other hand, the method of drawing graphs on a computer and projecting them onto an electronic whiteboard is cumbersome to operate and unsuitable for interactive use in meetings.

[0019] This embodiment provides a display device and a method for displaying the same graph, which can automatically recognize the trajectory information written by the user, convert the written content into a graph, and display it on an electronic whiteboard. By automatically generating and displaying a graph based on the content written by the user, the efficiency of meetings can be improved and the user's interactive experience can be enhanced.

[0020] In some embodiments, the display device in this embodiment includes, but is not limited to, an electronic whiteboard. The electronic whiteboard includes a touch component. The touch component is configured to acquire handwriting trajectory information. Optionally, the electronic whiteboard further includes components such as a camera or radar for acquiring handwriting trajectory information.

[0021] In this embodiment, the term "electronic whiteboard" refers not to a conventional electronic whiteboard written on with marker pens or chalk, but to a large-sized (generally 50 inches or larger) smart interactive display device with touch handwriting functionality. This smart interactive display device implements a writing program, such as an electronic whiteboard, that allows users to write with their fingers, active pens, or passive pens, and may include a display module such as a liquid crystal display (LCD), an organic electroluminescence display (OLED), and electronic ink. The handwriting can be displayed in real time.

[0022] This embodiment recognizes the acquired writing trajectory information upon receiving a graph drawing command from the user, and then uses the data recognition result to draw and display a graph. This allows for the automatic visualization and display of the user's handwritten content, which is effective in improving the interactive experience. The electronic whiteboard provided in this embodiment is capable of receiving the user's writing trajectory information, and in this embodiment, the configured touch component can receive both the writing trajectory information and the user's touch commands. In this embodiment, the specific method for acquiring the writing trajectory information is not particularly limited.

[0023] In some embodiments, as shown in Figure 1, the display device provided in this embodiment comprises a display screen 100 and a control circuit 101.

[0024] The display screen 100 is configured to display content.

[0025] The control circuit 101 comprises a processor and memory, the memory is configured to store a program that the processor can execute, and the processor is configured to read the program from the memory and execute the next step. The handwriting trajectory information in the display area of ​​the aforementioned display screen is recognized, and the data recognition result is obtained. In response to a user's graph drawing command, the system determines the graph type corresponding to the graph drawing command. In response to a user's graph drawing command, the system draws a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and displays the drawn graph in the display area.

[0026] In some embodiments, the display device in this embodiment includes, but is not limited to, an electronic whiteboard. The electronic whiteboard includes a touch component. The touch component is configured to acquire handwriting trajectory information. The electronic whiteboard according to this embodiment further includes components such as a camera or radar for acquiring handwriting trajectory information. The control circuit according to this embodiment further includes at least one of a drive control circuit, a clock module, and a rendering engine. The memory according to this embodiment may be used to store information that has storage needs, such as instructions and data.

[0027] In some embodiments, this embodiment includes, but is not limited to, at least one of a touch component, an imaging component, and a radar component in the electronic whiteboard configuration. Alternatively, the user can acquire writing trajectory information written on a terminal via a terminal connected to the electronic whiteboard in real time. The writing scene according to this embodiment is suitable for writing with a finger or a stylus, and the writing performed by the user via a terminal connected to the electronic whiteboard is also applicable to writing performed by the user in a non-contact (air writing) manner on a display screen, and this embodiment is not particularly limited in how writing trajectory information is specifically acquired.

[0028] In this embodiment, there is no limited priority relationship between the process of recognizing handwriting trajectory information and the response to the user's graph drawing command. In this embodiment, the handwriting trajectory information may be recognized and the data recognition result obtained, and then the system may respond to the user's graph drawing command, or the handwriting trajectory information may be recognized and the data recognition result obtained after responding to the user's graph drawing command.

[0029] In this embodiment, the position of the user's handwriting trajectory information in the display area and the position of the corresponding graph in the display area do not overlap, allowing the user to view both the written content and the drawn graph simultaneously. By automatically drawing the graph without interfering with normal content interaction, the efficiency of meeting interactions and the user's interactive experience are improved.

[0030] In some embodiments, in an intelligent drawing graph mode for a known graph type, the graph drawing is triggered after it is determined that the user has stopped writing and is waiting for a predetermined time. In this embodiment, in an intelligent drawing graph mode for a known graph type, in response to a command triggered after the user has stopped writing and is waiting for a predetermined time, the system recognizes the writing trajectory information in the display area of ​​the display screen and obtains data recognition results. Based on the data recognition results, the system draws the graph and displays it. In this embodiment, when the user stops writing in the display area and waits for a predetermined time, the system recognizes the writing trajectory information in the display area, obtains data recognition results, and draws the graph based on the data recognition results, and displays the drawn graph in the display area. In this embodiment, depending on the timing of receiving the graph drawing command, the system may recognize the writing trajectory information and, based on the data recognition results, decide to draw and display the graph timing.

[0031] In some embodiments, the timing for recognizing the writing trajectory information in this embodiment is such that when the user stops writing and it is determined that no writing has been done after a predetermined time has passed, the writing trajectory information in the display area of ​​the display screen is recognized and the data recognition result is obtained.

[0032] In some embodiments, in an intelligent drawing graph mode for a known graph type, the drawing of a graph is triggered when it is determined that a user gesture command has been received. In this embodiment, in an intelligent drawing graph mode for a known graph type, the handwriting trajectory information of the display area of ​​the display screen is recognized in response to the user gesture command, and data recognition results are obtained. Based on the data recognition results, a graph is drawn and the graph is displayed.

[0033] In some embodiments, the gesture instructions include, but are not limited to, gesture instructions such as arrows that indicate direction. The arrow gesture instructions include, but are not limited to, at least one of a unidirectional gesture instruction and a bidirectional arrow gesture instruction.

[0034] Selectively, determine the table data corresponding to the X-axis based on the arrow direction in the received gesture instruction, or determine the table data corresponding to the Y-axis based on the arrow direction in the received gesture instruction.

[0035] Selectively, if the gesture instruction is a single-arrow gesture instruction, the system determines either the table data corresponding to the X-axis or the table data corresponding to the Y-axis based on the direction of the single arrow. In this embodiment, for example, the data content in the first column of the table data is identified as the table data corresponding to the X-axis, and the data content in the second column of the table data is identified as the table data corresponding to the Y-axis.

[0036] Selectively, if the gesture command is a bidirectional arrow gesture command, the bidirectional arrow gesture command includes a first arrow command and a second arrow command drawn sequentially. Here, the reception time of the first arrow command is earlier than the reception time of the second arrow command. Also, the second direction of the second arrow command is different from the direction of the first arrow command. In this case, the table data for the X and Y axes is set as follows.

[0037] Based on the first direction of the first arrow command, the table data corresponding to the X-axis is determined, and based on the second direction of the second arrow command, the table data corresponding to the Y-axis is determined. Or, Based on the first direction of the first arrow command, the table data corresponding to the Y axis is determined, and based on the second direction of the second arrow command, the table data corresponding to the X axis is determined.

[0038] The data recognition results in this embodiment include, but are not limited to, data indicating that the data can be plotted as a graph, such as table data or dataset data.

[0039] Furthermore, the above-mentioned table data may be data that has row information or column information, or data that has both row information and column information simultaneously, and may not even be data that does not need to completely include the dividing lines between rows or columns. Any computer data that can represent the row or column information of the data may be referred to as table data.

[0040] The table data in this embodiment may include multiple cells, for example, cells arranged in a 3x2 grid (uniformly arranged), or it may include the user's handwriting trajectory information in the cells, or a graph may be drawn using the data content in each cell after recognizing the handwriting trajectory information in the cells and obtaining the data content of the cells.

[0041] In this embodiment, it is possible to identify a dataset from data that has column information or row information in table data. For example, the data content of each cell in the first column of the table data may be the content of the first dataset, and the data content of each cell in the second column of the table data may be the content of the second dataset. Here, the order of the contents in the dataset may be determined by the order of the cells, or the data content of each cell in the first row of the table data may be the content of the first dataset, and the data content of each cell in the second row may be the content of the second dataset. For example, if the data content of each cell in the first column of the table data is 10, 20, and 30, then the content of the first dataset will be [10, 20, 30].

[0042] This embodiment schematically shows the display area of ​​the display screen of an electronic whiteboard. As shown in Figures 2A, 2B, 2C, and 2D, after the user writes on one side of the display area of ​​the electronic whiteboard, a graph drawing command is triggered, and the corresponding graph is displayed on the other side of the display area. In this embodiment, after the user selects a graph type, in the intelligent drawing graph mode of a known graph type, if the user stops writing and waits for a predetermined time, the corresponding graph may be triggered to be drawn and displayed on the other side of the display area.

[0043] In some examples, methods for displaying graphs include, but are not limited to, methods that display both the coordinate axes and table data, methods that display the coordinate axes first and then the table data, or methods that display the table data first and then the coordinate axes. However, for line graphs, the table data is understood as each coordinate point on the line; for histograms, as each rectangle in the histogram; and for pie charts, as blocks of different sizes. In some examples, this embodiment may or may not display titles for each coordinate axis, but this can be specifically determined according to actual needs and is not particularly limited here.

[0044] In some examples, this embodiment acquires handwriting trajectory information in the display area upon receiving a graph drawing command from the user, or acquires handwriting trajectory information in the display area after determining that the user has finished writing. In this embodiment, the timing of acquiring handwriting trajectory information is not particularly limited. Specifically, the user determines that they have finished writing if they have not received any handwriting trajectory information within a predetermined time after receiving the last handwriting trajectory information, or if they receive a completion button command triggered by the user. Note that the buttons or keys in this embodiment include virtual buttons or entity buttons, and are not particularly limited here.

[0045] In some embodiments, the methods for identifying the writing trajectory information provided by this embodiment can be broadly divided into three types based on different writing scenes.

[0046] Type 1: A writing scene with table lines.

[0047] This writing scene primarily recognizes writing trajectory information and performs recognition based on the data content in the writing presentation table displayed in the display area. In this scene, the position (area size) of each cell in the writing presentation table is known, and even if the writing presentation table can be dynamically expanded while the user is writing, the position (area size) of each cell in the expanded writing presentation table is also known. The writing presentation table may be fixed and displayed in the display area before the user starts writing, or the reference position of the writing presentation table may be dynamically determined according to the position where the user first writes. The user only needs to fill in the content corresponding to each cell in the writing presentation table, and in the recognition process, only writing trajectory information is recognized. Since the position of each cell in each writing presentation table is known, the data recognition result is divided into each cell of the writing presentation table to obtain a table containing the content entered by the user, and a graph is generated and displayed from the data content in the table.

[0048] In some embodiments, the processor is configured to perform the following actions specifically before recognizing the handwriting trajectory information in the display area of ​​the display screen and obtaining the data recognition result.

[0049] The processor displays a writing presentation table in the display area, and the writing presentation table presents the user to write in each cell of the writing presentation table. The system receives the handwriting trajectory information entered by the user on the handwriting display table.

[0050] In some embodiments, the processor is configured to perform the following actions:

[0051] The processor determines the reference position of the writing presentation table based on the position of the writing trajectory information that the user first wrote on the display area. A writing display table with a fixed number of rows and columns is displayed in the display area according to the reference position of the writing display table, and the writing trajectory information written by the user for the first time is located within a cell of the writing display table.

[0052] For example, the position of the content first written by the user in the display area is set as the position of the cell in the first row and first column of the writing presentation table, or as the position of the cell in the second row and second column of the writing presentation table, and a writing presentation table with a fixed number of rows and columns is displayed using that position as the reference position of the writing presentation table.

[0053] In some embodiments, after displaying the writing presentation table, the processor is specifically configured to perform one of the methods of increasing the number of rows and columns in the writing presentation table.

[0054] Method 1) If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of rows in the writing presentation table is increased according to the position of the writing trajectory information that the user continues to write.

[0055] Method 2) If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of columns in the writing presentation table is increased according to the position of the writing trajectory information that the user continues to write.

[0056] Method 3) If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of rows and columns in the writing presentation table is increased according to the position of the writing trajectory information that the user continues to write.

[0057] Method 4) If the writing trajectory information that the user continues to write is in the critical row of the writing presentation table, increase the number of rows in the writing presentation table.

[0058] Method 5) If the writing trajectory information that the user continues to write is in the critical column of the writing presentation table, increase the number of columns in the writing presentation table.

[0059] Method 6) If the writing trajectory information that the user continues to write is in a cell in the critical row and critical column of the writing presentation table, increase the number of rows and columns of the writing presentation table.

[0060] Here, the critical row according to this embodiment indicates that the last row of the writing presentation table is about to be reached, meaning, for example, the second to last row of the writing presentation table, or the last row of the writing presentation table.

[0061] In this embodiment, the writing display table can dynamically determine its reference position according to the position where the user first writes. Furthermore, immediately before the user fills each cell in the writing display table, or if the written content exceeds the range included in the writing display table, the writing display table may be automatically and dynamically expanded to ensure that the user's written content remains within the range of the writing display table.

[0062] In some embodiments, the processor is configured to perform the following actions:

[0063] The processor, in response to a user's edit command in a smart graph mode of a known graph type, displays a writing suggestion table in the display area, wherein the number of rows and columns in the writing suggestion table is fixed or determined based on parameters entered by the user.

[0064] In this embodiment, the timing of the displayed handwriting presentation table may be after the handwriting presentation table is displayed in response to a user edit command in smart graph mode of a known graph type, or the user may set the rows and columns of the handwriting presentation table in response to further entered row and column parameters. For example, if the user inputs a row parameter of 3 and a column parameter of 5, a 3-row, 5-column handwriting presentation table is generated and displayed.

[0065] In some embodiments, the cell size of the writing display table according to this embodiment may be predetermined, determined based on cell parameters entered by the user, determined based on the first writing trajectory information written by the user, or determined, for example, by the position of each trajectory point in the first writing trajectory information written by the user, and is not particularly limited.

[0066] In some embodiments, in a writing scene having a table line, the data recognition result includes a plurality of datasets, each dataset containing the content written by the user in any column of the writing presentation table, or the content written by the user in any row of the writing presentation table.

[0067] In this embodiment, multiple datasets are determined as follows.

[0068] When the handwriting display table is displayed in the aforementioned display area, the system either obtains multiple datasets by dividing the content entered by the user in the handwriting display table into columns or rows, or If the handwriting presentation table is not displayed in the aforementioned display area, after determining each cell, the data content of each cell is divided by column or row to obtain multiple datasets.

[0069] Each element in any dataset corresponds to the content of any column, and the order of elements in a dataset is the same as the order of cells in that column. For example, if a user writes content in 4 rows and 3 columns in a writing table, the writing table can be divided into 3 columns, each corresponding to one of three datasets. The first dataset contains the data content of the 4 cells in the first column, meaning each dataset contains 4 elements.

[0070] In some embodiments, in a writing scene having table lines, the processor of this embodiment determines the data recognition result in one of the following ways:

[0071] Method 1: After determining the trajectory points of the cells, the trajectory points of each cell are recognized.

[0072] Based on the positional relationship between each trajectory point in the aforementioned writing trajectory information and the cells in the aforementioned writing presentation table, the writing trajectory information contained in each cell is determined.

[0073] The system recognizes the handwriting trajectory information contained in each cell and retrieves the data content of each cell.

[0074] Method 2: The data content within the cell is determined after recognizing the handwriting trajectory information.

[0075] The system recognizes the aforementioned writing trajectory information and obtains the content written by the user and its position within the display area.

[0076] Based on the location of the content and the location of the cell in the handwriting presentation table, the data content of each cell is determined.

[0077] In a writing scene with table lines, the position and size of each cell in the writing presentation table are known. When recognizing the cells, the system determines which cell each trajectory point is located in based on the positional relationship between each trajectory point in the writing trajectory information and the cell. This identifies the trajectory points (i.e., writing trajectory information) contained in each cell, recognizes the trajectory points in each cell, and obtains the data content of each cell. The data content of each cell is then divided into columns to obtain multiple datasets, or the data content of each cell is divided into rows to obtain multiple datasets. A graph is then drawn and displayed for each dataset.

[0078] Type 2, a writing scene without table lines.

[0079] This writing scene requires recognition not only of the writing trajectory information but also of the content written by the user using a clustering algorithm. In this scene, the writing presentation table is not displayed in the display area; the user only needs to fill in the data content in the table. The recognition process requires not only a handwriting recognition algorithm but also a clustering algorithm without table lines to divide the writing trajectory information into different cells, recognize the writing trajectory information in each cell to obtain the data content in each cell, and finally generate and display a graph from the data content in each cell.

[0080] In some examples, the data recognition result in this embodiment includes multiple cells in this scene, and the data recognition result includes the position of each cell and the data content within each cell. In this embodiment, the content actually written by the user is obtained by recognizing the writing trajectory information, and based on the user's drawing request, the content written by the user is treated as individual data content, and a graph of the graph type corresponding to the graph drawing command is drawn from each data content in the table, and the drawn graph is displayed in the display area.

[0081] In this scene, in some embodiments, the embodiment determines the writing trajectory information that needs to be displayed in the graph after receiving a graph drawing command from the user. The specific determination method is one of the following, but is not limited to these.

[0082] Method 1) Identify all writing trajectory information within the display area as writing trajectory information that requires graph display.

[0083] In a specific embodiment, this method configures the electronic whiteboard's display screen to receive handwriting trajectory information and a display area for the graph when the user needs to draw a graph while using the electronic whiteboard. The display screen is configured to hide any previously written content by the user.

[0084] Method 2) Identify all handwriting trajectory information within the preset boxes in the display area as handwriting trajectory information that requires graph display.

[0085] This method allows users to request a graph display without hiding existing user-written content on the display screen. When a user requests a graph, a preset box is displayed in the display area of ​​the screen, and the user can write the content they need for the graph within the preset box, allowing the graph to be displayed in the other areas of the display area.

[0086] When a graph drawing command is received from the user using method 2), all handwriting trajectory information in the display area is recognized, and all table data corresponding to the handwriting trajectory information is obtained. Here, the handwriting trajectory information includes, but is not limited to, the writing order and coordinates of each trajectory point. That is, the handwriting trajectory information includes trajectory points with sequential writing order. Since the handwriting trajectory information includes not only the coordinates of the trajectory points but also the writing order of the trajectory points, the content written by the user can be recognized more accurately based on the relationship between the writing order and coordinates of each trajectory point.

[0087] In some embodiments, the writing scene includes a data recognition result comprising multiple cells, the location of each cell and the data content within each cell.

[0088] In some examples, the algorithm for performing handwriting trajectory recognition in this embodiment includes, but is not limited to, a clustering algorithm and a handwriting recognition algorithm. Here, the clustering algorithm divides all received handwriting trajectory information into cells by clustering them according to the similarity between each trajectory point, with each cell containing only one type of trajectory point, which can be understood as the data content within a single cell written by the user. The handwriting recognition algorithm is used to recognize the trajectory points in each cell and to obtain the final data content. Based on the location of each cell and the data content within the cell, it is ultimately possible to identify each data content in the table written by the user.

[0089] In this embodiment, after acquiring handwriting trajectory information, the acquired handwriting trajectory information is identified to obtain the position information of each cell in the table and the data content corresponding to each cell. For example, in a 2x4 table, the position of the first cell is (1,1) and the data content of the first cell is "income", the position of the second cell is (1,2) and the data content of the second cell is "100".

[0090] In some examples, this embodiment acquires table data by recognizing the handwriting trajectory information of the display area of ​​the display screen through the following steps.

[0091] Step 1: Determine each cell and divide the handwriting trajectory information into each cell.

[0092] Here, each of the cells is used to correspond to each cell in the table, and the handwriting trajectory information contained in each cell is used to correspond to the data content in each cell.

[0093] In this embodiment, each cell is determined based on the handwriting trajectory information according to a clustering algorithm, and then the handwriting trajectory information is divided into cells, with each cell containing data content.

[0094] In some cases, the clustering algorithm in this embodiment can employ an adaptive class clustering algorithm such as K-Means (K-means clustering algorithm). By grouping trajectory points with matching writing order and close coordinate distances into one class and dividing the trajectory information of the same class into the same cell, all writing trajectory information is divided into individual cells.

[0095] In this embodiment, each cell may be determined by a clustering algorithm, or the user's handwriting trajectory information may be divided into each cell.

[0096] In this embodiment, it is possible to pre-set the number of clusters in the K-Means algorithm, i.e., the total number of cells that were last masked. However, an improved K-Means algorithm that automatically determines the number of clusters can also automatically determine the final number of clusters. That is, the K-Means algorithm can automatically determine the total number of cells that were last masked. Here, the dataset for the K-Means algorithm contains N sample points, each sample point consisting of two features: the writing order of the trajectory points and the coordinates of the trajectory points. The design philosophy is to calculate the distance between sample points within each category through a pre-set loss function such that the error between sample points in different clusters becomes large, while the distance between sample points within each cluster becomes small. The K-Means algorithm first selects L mean points, clusters the sample points based on these L mean points, removes points with small samples within a cluster, randomly selects one mean point from the remaining L1 sample points after sample removal, searches for the value furthest from that mean point, and repeats this process until K mean points have been selected. Here, L, L1, and K are all positive integers, and L > L1.

[0097] The specific steps of the K-Means algorithm are as follows:

[0098] Step 1: Initialize and select K average scores.

[0099] Step 2: Classify the sample points into the cluster with the mean score closest to your own, and determine the clustering result.

[0100] By calculating the minimum value of the loss function, the optimal clustering result can be obtained.

[0101] Step 3: Update the average score based on the clustering results.

[0102] Step 4: Repeat Steps 2 and 3 until the average score no longer changes.

[0103] Step 5: Output the clustering results.

[0104] In some examples, this embodiment provides two methods for dividing handwriting trajectory information into individual cells.

[0105] The implementation steps for Method 1 are as follows:

[0106] Step 1a: Obtain the writing order and coordinates of each trajectory point in the writing trajectory information.

[0107] Step 1b: Based on the writing order and coordinates of each trajectory point, the degree of similarity between each trajectory point is determined.

[0108] Step 1c: The trajectory points whose similarity satisfies a predetermined threshold are divided into individual cells.

[0109] In this embodiment, after obtaining the writing order and coordinates of each trajectory point in the writing trajectory information, each trajectory point may be clustered according to a clustering algorithm, trajectory points whose similarity satisfies a predetermined threshold may be divided into one class, and cells corresponding to the trajectory points in each cluster may be generated.

[0110] The implementation steps for Method 2 are as follows:

[0111] Step 2a: The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke.

[0112] In some embodiments, the writing trajectory information is obtained after performing a resampling operation on the trajectory points in the writing trajectory information. The resampling is configured to increase the number of trajectory points in the corresponding stroke according to the stroke length in the writing trajectory information. In this embodiment, the writing trajectory information may be resampled to increase the number of points in the stroke according to the stroke length, and each cell may be determined using the resampled writing trajectory information.

[0113] Step 2b: The stroke is divided into cells based on the positional relationship between the center of gravity of the stroke and each cell.

[0114] In some embodiments, each cell is determined by one or more of the following methods:

[0115] Method 1) Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined, and based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information contained in any of the columns, row separators for any of the columns are determined, and each cell is determined based on the column separators and row separators.

[0116] In this embodiment, the first coordinate axis and the second coordinate axis are perpendicular to each other.

[0117] In this embodiment, the column separators are determined as follows.

[0118] The column separators are determined based on the valley locations in the first projection histogram where the valley width is greater than the threshold.

[0119] In this embodiment, the line separator is determined as follows.

[0120] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0121] In this embodiment, determining each cell based on column separators and row separators is, specifically, The handwriting trajectory information is divided into multiple columns based on column dividers, the handwriting trajectory information in each column is divided into multiple rows based on row dividers, and finally, the table borders are determined from the minimum and maximum values ​​of the first and second coordinate axes of the trajectory points in all the handwriting trajectory information. The table and the data content within the table are then determined based on the column dividers, row dividers, borders, and handwriting trajectory information. In this embodiment, the cell determination method is performed by dividing the columns first and then dividing the rows for each column, which solves the problem of the user's handwriting being illegible and the contents of different columns in each row not being aligned. Even if the contents belonging to different columns in each row of the user's table are not on the same horizontal line, the method in this embodiment can accurately divide the data content of each cell and accurately realize the division of rows and columns. By using this embodiment in a vertically written table, the problem of handwritten columns in a vertically written table being misaligned and displayed in the wrong row, affecting recognition, is solved.

[0122] Method 2) Based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined. Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information contained in any row, column separators for any row are determined, and each cell is determined based on the row separators and column separators.

[0123] In this embodiment, the column separators are determined as follows.

[0124] The column separators are determined based on the valley locations in the first projection histogram where the valley width is greater than the threshold.

[0125] The line separators are determined as follows:

[0126] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0127] In this embodiment, determining each cell based on column separators and row separators is, specifically, The handwriting trajectory information is divided into multiple rows based on row dividers, and the handwriting trajectory information of each row is divided into multiple columns based on column dividers. Finally, the table borders are determined from the minimum and maximum values ​​of the first and second coordinate axes of the trajectory points in all the handwriting trajectory information. The table and the data content within the table are then determined based on the column dividers, row dividers, borders, and handwriting trajectory information. In the cell determination method of this embodiment, row division is achieved first, and then column division is performed for each row, which solves the problem of the user's handwriting being unsuitable and resulting in blank rows in the table. However, in the method according to this embodiment, blank rows can be identified and the data content within each cell can be accurately divided, thereby accurately achieving row and column division.

[0128] Method 3) Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; and each cell is determined based on the row separators and column separators.

[0129] In this embodiment, the column separators are determined as follows.

[0130] The column separators are determined based on the valley locations in the first projection histogram where the valley width is greater than the threshold.

[0131] The line separators are determined as follows:

[0132] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0133] In this embodiment, determining each cell based on column separators and row separators is, specifically, The handwriting trajectory information is divided into multiple columns based on column dividers, and into multiple rows based on row dividers. Finally, the table borders are determined from the minimum and maximum values ​​of the first and second coordinate axes of the trajectory points in all the handwriting trajectory information. The table and the data content within the table are then determined based on the column dividers, row dividers, borders, and handwriting trajectory information. In this embodiment, the cell determination method solves the problem of empty cells appearing in the table due to unsuitable user handwriting by simultaneously achieving matrix division. According to the method in this embodiment, empty cells, i.e., cells in which no trajectory information has been written, can be identified, and the data content within each cell can be accurately separated to accurately achieve row and column division.

[0134] In some embodiments, the first projected histogram is a projected histogram obtained by projecting the writing trajectory information onto a first coordinate axis, and is obtained by reduction mapping. The reduction mapping reduces the unit pitch of the first coordinate axis of the projected histogram to a first predetermined pitch, sums the number of trajectory points projected within the first predetermined pitch, and takes the sum as the number of trajectory points corresponding to a second predetermined pitch. Here, taking the example that the first coordinate axis is the X-axis, the X-axis coordinate of the projected histogram represents the position of the trajectory points in the writing trajectory information in the X direction, and the Y-axis coordinate represents the number of trajectory points projected to different X-axis positions. For example, the number of trajectory points in the 0-100 interval of the histogram projected onto the X-axis is reduced and mapped to a new 0-10 interval. Specifically, the sum of the number of locus points in the 0-10 sub-interval of the 0-100 interval is used as the number of locus points corresponding to the 0-1 sub-interval of the new 0-10 interval, and the sum of the number of locus points in the 10-20 sub-interval of the 0-100 interval is used as the number of locus points corresponding to the 1-2 sub-interval of the new 0-10 interval. This is done by analogy sequentially. Furthermore, the number of locus points in each sub-interval of the 0-10 interval is normalized, and taking the 0-1 sub-interval of the 0-10 interval as an example, the sum of the number of locus points corresponding to the 0-1 sub-interval is taken as the number of locus points corresponding to the 0-1 sub-interval. Here, the 0-1 sub-interval corresponds to the number of locus points of one, that is, the 0-1 sub-interval corresponds to one number.

[0135] The second projection histogram is obtained by projecting the writing trajectory information onto a second coordinate axis, and is obtained by reduction mapping. The reduction mapping reduces the unit pitch of the second coordinate axis of the projection histogram to a second predetermined pitch, sums the number of trajectory points projected within the second predetermined pitch, and takes the sum as the number of trajectory points corresponding to the second predetermined pitch. Here, with the second coordinate axis as the Y axis, the Y-axis coordinate of the projection histogram indicates the position of the trajectory points in the writing trajectory information in the Y direction, and the X-axis coordinate indicates the number of trajectory points mapped to different Y-axis positions. For example, the projection histogram reduces the number of trajectory points in the 0-100 interval of the Y axis to a new 0-10 interval through reduction mapping. The specific implementation principle can be found in the explanation of the first projection histogram described above and is omitted here.

[0136] Here, the projection histogram according to this embodiment is a statistical diagram representing the number of trajectory points on which handwriting trajectory information is projected onto a predetermined coordinate axis.

[0137] Step 2: Recognize the handwriting trajectory information in each cell and retrieve the data content in each cell.

[0138] Here, based on a handwriting recognition algorithm, the handwriting trajectory information is recognized and the data content in each cell is obtained. Selectively, the handwriting recognition algorithm according to this embodiment includes, but is not limited to, at least one of a character recognition algorithm and a digit recognition algorithm.

[0139] Step 3: Determine the data recognition result based on the position of each cell and the data content within each cell.

[0140] As shown in Figure 3, this embodiment illustrates the recognition of handwriting trajectories. The user's handwriting trajectory information, "product, profit, A, B, C, 10000, 20000, 30000," is divided into 8 cells. All handwriting trajectory information, including the data content in each cell, is recognized by a handwriting recognition algorithm, and the data content in each cell is acquired to determine the data recognition result.

[0141] In this embodiment, the table data consists of individual cells, and the data content within the table consists of the data content within each cell. By recognizing the position of each cell and the data content (data content within each cell) obtained, the table data can be determined. Either the data content within each column cell is treated as a single array, or the data content within each row cell is treated as a single array, and a graph is drawn and displayed based on each array.

[0142] Type 3: A writing scene that displays a title prompt.

[0143] In the writing scene, the displayed title prompt includes, but is not limited to, horizontal or vertical lines. Selectively, this includes identifying writing trajectory information above the horizontal line, recognizing the writing trajectory information below the horizontal line using a clustering algorithm and a handwriting recognition algorithm, or recognizing the writing trajectory information to the left of the vertical line, and recognizing the writing trajectory information to the right of the vertical line using a clustering algorithm and a handwriting recognition algorithm. In the writing scene, only horizontal or vertical lines are displayed in the display area. At least one horizontal or vertical line is displayed, which may be a long horizontal line or multiple short horizontal lines, or a long vertical line or multiple short vertical lines. This embodiment does not limit the style or number of horizontal or vertical lines displayed. For example, the user may write a title for each coordinate axis above the horizontal line, or one or more titles, or write the content below the title in association with the horizontal line.

[0144] In some embodiments, in the writing scene, the processor is configured to perform the following actions specifically before recognizing the writing trajectory information of the display area of ​​the display screen and obtaining the data recognition result.

[0145] The processor is configured to display a title prompt in the display area, and the title prompt is configured to indicate the writing position of the user table title. Here, when drawing a line graph, the table title represents the title of the coordinate axis; when drawing a bar graph, the table title represents the title of each category or cylinder on a predetermined coordinate axis; when drawing a radar map, the table title represents the title of each vertex or the polar coordinate name; and when drawing a pie chart, the table title represents the title of each partition.

[0146] In the first region corresponding to the title prompt, first writing trajectory information is acquired, and in the second region corresponding to the title prompt, second writing trajectory information is acquired.

[0147] The title prompt according to this embodiment not only presents the writing position to the user but also allows the user to write according to the title prompt. As a result, when the user writes different content, the spacing between the contents becomes narrower, which can lead to problems in accurately performing automatic grid recognition when using clustering algorithms. This makes it easy to mistakenly divide some or all of the trajectory points within the same content into other cells, resulting in recognition errors.

[0148] In some embodiments, the title prompt is used to separate the title of each coordinate axis written by the user (i.e., the table title) and the content corresponding to the title. In some embodiments, the first area and the second area are separated by the title prompt. In some embodiments, the title prompt includes at least one of a horizontal line, a vertical line, or a symbol.

[0149] In this embodiment, two horizontal lines are displayed, and each horizontal line corresponds to one column in the table. For example, first handwriting trajectory information representing title 1 is acquired in the area above the first horizontal line, and the first handwriting trajectory information is recognized as title 1 on the X coordinate axis. Second handwriting trajectory information representing title 2 is acquired in the area above the second horizontal line, and the first handwriting trajectory information is recognized as title 2 on the Y coordinate axis. Second handwriting trajectory information representing content is acquired in the area below the first horizontal line, and the second handwriting trajectory information is recognized as content corresponding to title 1. Second handwriting trajectory information representing content is acquired in the area below the second horizontal line, and the second handwriting trajectory information is recognized as content corresponding to title 2.

[0150] In a writing scene that displays a title prompt, the data recognition result includes the title of each coordinate axis and multiple cells, and the data recognition result includes the position of each cell and the data content within each cell.

[0151] In some embodiments, the data identification result is determined by the following steps.

[0152] Step 3a) Recognize the writing trajectory information of the first region and obtain the title of each coordinate axis.

[0153] Step 3b) The handwriting trajectory information of the second region is divided into individual cells, the handwriting trajectory information in each cell is recognized, and the data content in each cell of the second region is obtained.

[0154] Here, the writing trajectory is divided into cells in one of the following ways:

[0155] Method 1: The writing trajectory information is converted into a stroke, the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke, and the stroke is divided into cells based on the positional relationship between the centroid of the stroke and each cell.

[0156] Method 2: Obtain the writing order and coordinates of each trajectory point in the writing trajectory information, determine the similarity between each trajectory point based on the writing order and coordinates of each trajectory point, and divide the trajectory points whose similarity satisfies a predetermined threshold into cells.

[0157] Each cell is determined in one of the following ways:

[0158] Method 1: Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information contained in any of the columns, row separators for any of the columns are determined; and each cell is determined based on the column separators and row separators.

[0159] Method 2: Based on the valley position in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; based on the valley position in the first projection histogram on the first coordinate axis of the handwriting trajectory information contained in any row, column separators for any row are determined; and each cell is determined based on the row separators and column separators.

[0160] Method 3: Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; and each cell is determined based on the row separators and column separators.

[0161] Here, the column separators are determined as follows:

[0162] Based on the valley locations where the valley width in the first projection histogram is greater than a threshold, the column separator lines are determined, and / or, The line separators are determined as follows:

[0163] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0164] Here, the handwriting trajectory information in methods 1 to 3 is obtained by resampling the trajectory points within the handwriting trajectory information written by the user. The resampling is configured to increase the number of trajectory points in the corresponding stroke according to the length of each stroke in the handwriting trajectory information written by the user.

[0165] The first projected histogram is a projected histogram obtained by projecting the writing trajectory information onto a first coordinate axis, and is obtained by reduction mapping. The reduction mapping reduces the unit pitch of the first coordinate axis of the projected histogram to a first predetermined pitch and sums the number of trajectory points projected onto the first predetermined pitch.

[0166] The second projection histogram is obtained by projecting the writing trajectory information onto a second coordinate axis, and is obtained by reduction mapping. The reduction mapping reduces the unit pitch of the second coordinate axis of the projection histogram to a second predetermined pitch and sums the number of trajectory points projected onto the second predetermined pitch.

[0167] Step 3c) The data recognition result is determined based on the title, the data content in each cell of the second area, and the positional relationship between the title and each cell.

[0168] In this writing scene, in addition to recognizing the writing trajectory information in the first area corresponding to the title prompt, the recognition method in the second area can be recognized using the recognition method for the second type of writing scene without table lines. In this scene, the user writes the title in the first area and the content in the second area, so by identifying only the first area, the writing trajectory information in the second area can be identified using a clustering algorithm and a handwriting recognition algorithm. First, the writing trajectory information is divided into individual cells, and then the writing trajectory information in each cell is recognized to determine the data content within each cell. Finally, the data content in the cells in the column corresponding to each title is determined, or the data content in the cells in the row corresponding to each title is determined.

[0169] Either treat the data content in each column cell (excluding the title) as a single array, or treat the data content in each row cell (excluding the title) as a single array, and then draw and display a graph based on each array.

[0170] In this embodiment, the method for recognizing the three different scenes and obtaining the recognition results is slightly different; that is, the recognition method for scenes with tables is different from the recognition method for scenes without tables. In this embodiment, based on the three different scenes, it is possible to identify the contents of tables written by the user in each different scene and to identify the contents of any column or any row written by the user.

[0171] In some embodiments, multiple datasets are determined by dividing the user's handwritten content into columns, or by dividing the user's handwritten content into rows. The content of any column written by the user is determined as a single dataset, or the content of any row written by the user is determined as a single dataset.

[0172] In some embodiments, the graph type is determined according to a graph drawing command, and a graph corresponding to the graph type is drawn based on each dataset corresponding to the content written by the user.

[0173] In some embodiments, the contents of each column written by the user are determined as a data set based on the pre-set correspondence between each row and the coordinate axes, or the contents of each row written by the user are determined as a data set based on the pre-set correspondence between each column and the coordinate axes.

[0174] In some embodiments, the content written by the user that is located in a predetermined row is determined as the title of each coordinate axis, and the data content in the other cells is determined as the data content, or the content written by the user that is located in a predetermined column is determined as the title of each coordinate axis, and the data content in the other cells is determined as the data content.

[0175] In some embodiments, the data content within each cell is determined as the data content.

[0176] In some embodiments, the graph type is determined according to the graph plotting command. Based on the data sets corresponding to the content written by the user, a graph corresponding to the graph type is drawn.

[0177] In this embodiment, whether in table mode or table-less mode, the content written by the user during graph plotting is divided into separate datasets, and a graph corresponding to the graph type is plotted based on each dataset corresponding to the content written by the user.

[0178] In this mode with a table, the contents of each column written by the user in the handwriting presentation table are treated as a single dataset, or the contents of each row written by the user in the handwriting presentation table are treated as a single dataset.

[0179] In table-less mode, the user's handwritten content is divided into individual cells, and then either the data in the cells of a column is combined into a single dataset, or the data in the cells of each row is combined into a single dataset.

[0180] In some embodiments, if any of the datasets contain the contents of any of the columns written by the user, different types of graphs can be drawn as follows:

[0181] If the graph type is determined to be a bar graph, the classification titles on the predetermined coordinate axes are determined based on the dataset corresponding to the predetermined column, and the column names are determined based on the first content corresponding to the predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column. Based on the content excluding the first content in the dataset corresponding to the remaining columns, the column heights corresponding to each classification title and column name are determined, and the bar graph is drawn based on each classification title, column name, and column height, or If the graph type is determined to be a radar map, the title of each vertex is determined based on the dataset corresponding to a predetermined column, the polar coordinate name is determined based on the second content corresponding to a predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column, the polar coordinates corresponding to each title and polar coordinate name are determined based on the content excluding the second content in the dataset corresponding to the remaining columns, and the radar map is drawn based on each title, polar coordinate name and polar coordinate, or If the graph type is determined to be a line graph, the title of the coordinate axis is determined based on the content corresponding to a predetermined index value in the dataset. The coordinate points are determined based on the content corresponding to the same index value other than the predetermined index value in different datasets. Based on the title of the coordinate axis and the coordinate points, a line graph is drawn, or If the graph type is determined to be a pie chart, the title of each partition is determined based on the dataset corresponding to the designated column. The occupancy rate of each partition is determined based on the dataset corresponding to the columns other than the designated column, and the pie chart is drawn based on the title and occupancy rate of each partition.

[0182] Here, when plotting a bar graph, the dataset according to this embodiment is understood as a single sequence. The predetermined index value according to this embodiment is configured to represent a specific element in the array. For example, if the predetermined index value is 1, then the first element corresponding to the predetermined index value 1 is the first element in the array.

[0183] Here, when drawing the radar map, the polar coordinate names according to this embodiment are used to distinguish between different examples. For example, if the corresponding datasets in the first column are mathematics, English, and language, the polar coordinate name corresponding to a predetermined index value 1 in the corresponding dataset in the second column is called "set one," and the polar coordinate name corresponding to a predetermined index value 1 in the corresponding dataset in the third column is called "set two," thus distinguishing between "set one" and "set two" with different polar coordinate names.

[0184] Here, when plotting a line graph, for example, the first content corresponding to a predetermined index value of 1 in the first array is given the X-axis title "Month," the first content corresponding to a predetermined index value of 1 in the second array is given the Y-axis title "Income," and the coordinate point (12, 100) is determined based on the second content 12 corresponding to a predetermined index value of 2 in the first array and the second content 100 corresponding to a predetermined index value of 2 in the second array, indicating that the income for December is 100. The line graph for each coordinate point is determined and plotted sequentially.

[0185] In this embodiment, the method for drawing the different types of graphs described above was explained using the scenario in which any dataset contains the content of any column written by the user. However, in this embodiment, rows and columns are interchangeable, and the above embodiment can similarly be applied to a scenario in which any dataset contains the content of any row written by the user. In this case, it is only necessary to replace the columns with rows in the above embodiment, and based on the implementation principle described above, the implementation principle is the same whether the content of rows is used as a dataset or the content of columns is used as a dataset, and since this falls within the scope of protection of the present invention, an explanation is omitted.

[0186] In some embodiments, the data recognition results determined by different graph types are shown below, for example, when one of the datasets contains content written by the user in one of the columns of the handwriting presentation table.

[0187] Case 1: Determine that the graph type is a bar graph.

[0188] Based on the dataset corresponding to a predetermined column, the classification title for each predetermined coordinate axis is determined. Based on the dataset corresponding to columns other than the predetermined column, the cylinder height and cylinder name corresponding to each classification title are determined. A bar graph is drawn based on each classification title and the cylinder height and cylinder name corresponding to each classification title.

[0189] In some embodiments, if the graph type is determined to be a bar graph, the classification titles for each predetermined coordinate axis are determined based on the dataset corresponding to the predetermined column. The column names are determined based on the first content corresponding to predetermined index values ​​in the dataset corresponding to the remaining columns excluding the predetermined column. The column heights corresponding to each classification title and column name are determined based on the content excluding the first content in the dataset corresponding to the remaining columns, and the bar graph is drawn based on each classification title, column name, and column height.

[0190] In this embodiment, as shown in the handwriting display table 1, in scenarios where the graph type is known, the user writes content in the handwriting display table 1, the content written by the user is recognized, the data content in each cell in the handwriting display table is retrieved, and a bar graph is drawn using the data content in those cells. Specifically, it is as follows.

[0191] [Table 1] Here, the handwriting presentation table may contain empty cells.

[0192] For example, each item in the dataset corresponding to the first column is determined as a classification title on the X-axis. That is, from "Language, Mathematics, English" in the dataset in the first column, the classification titles on the X-axis are identified as "Language, Mathematics, English," respectively. From the first item in the dataset corresponding to the first element in the second column, the cylinder name is identified as "Set 1." From the first item in the dataset corresponding to the first element in the third column, the cylinder name is identified as "Set 2." From the items in the dataset corresponding to the remaining elements (90, 95, 94) in the second column, the item corresponding to Language and Set 1 is set to 90, the item corresponding to Mathematics and Set 1 is set to 95, and the item corresponding to English and Set 1 is set to 94. That is, the cylinder name is identified as "Set 1," the cylinder height with the title "Japanese Language" is set to 90, and the cylinder name is called "Set 1." The cylinder height with the title "Mathematics" is set to 95, and the cylinder name is called "Set 1." The cylinder height with the title "English" is set to 94. Similarly, based on the content corresponding to the remaining elements (89, 99, 92) in the corresponding dataset in the third column, the cylinder name "Two Sets" and title "Japanese Language" will have a height of 89, the cylinder name "Two Sets" and title "Mathematics" will have a height of 99, and the cylinder name "Two Sets" and title "English" will have a height of 92. Based on these determined classification titles, cylinder names (One Set, Two Sets), and cylinder heights, a bar graph will be drawn.

[0193] Case 2: Determine that the graph type is a radar map.

[0194] Based on the dataset corresponding to a predetermined column, the title of each vertex is determined, and based on the dataset corresponding to columns other than the predetermined column, the polar coordinate sum and polar coordinate name corresponding to each title are determined. Based on the title of each vertex and the polar coordinate and polar coordinate name corresponding to each title, a radar map is drawn.

[0195] In some embodiments, if the graph type is determined to be a radar map, the title of each vertex is determined based on a dataset corresponding to a predetermined column, the polar coordinate name is determined based on a second content corresponding to a predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column, the polar coordinates corresponding to each title and polar coordinate name are determined based on the content excluding the second content in the dataset corresponding to the remaining columns, and the radar map is drawn based on each title, polar coordinate name and polar coordinate.

[0196] In this embodiment, as shown in the handwriting display table 1, in scenarios where the graph type is known, the user writes content in the handwriting display table 1, the content written by the user is recognized, the data content in each cell in the handwriting display table is retrieved, and the radar map is drawn using the data content in that cell. Specifically, For example, each item in the dataset corresponding to the first column is determined to be the title of each vertex on the radar map. That is, from the "Language, Mathematics, English" included in the dataset in the first column, it is determined that the title of each vertex on the radar map is "Language, Mathematics, English". Based on the first item in the dataset corresponding to the second column, the polar coordinate name is set to "Set 1". Based on the first item in the dataset corresponding to the third column, the polar coordinate name is set to "Set 2". Based on the items in the dataset corresponding to the second column (90, 95, 94), the item corresponding to "Language" and "Set 1" is set to 90, the item corresponding to "Mathematics" and "Set 1" is set to 95, and the item corresponding to "English" and "Set 1" is set to 94. That is, the polar coordinate name is set to "Set 1", the title to "Japanese", and the polar coordinate is set to 90; the polar coordinate name is set to "Set 1", the title to "Mathematics", and the polar coordinate is set to 95; the polar coordinate name is set to "Set 1", the title to "English", and the polar coordinate is set to 94. Similarly, based on the content corresponding to the remaining elements (89, 99, 92) in the corresponding dataset in the third column, the polar coordinate name is set to "Two Sets" and the title to "Japanese Language" is set to 89, the polar coordinate name is set to "Two Sets" and the title to "Mathematics" is set to 99, and the polar coordinate name is set to "Two Sets" and the title to "English" is set to 92. Based on this, a radar map is drawn based on each determined title, polar coordinate name (One Set, Two Sets), and each polar coordinate.

[0197] Case 3: Determine that the graph type is a line graph.

[0198] The title of the coordinate axis is determined based on the content corresponding to a predetermined index value in the dataset. The coordinate points are determined based on the content corresponding to the same index value other than the predetermined index value in different datasets. A line graph is drawn based on the title of the coordinate axis and the coordinate points.

[0199] In this embodiment, as shown in the writing display table 2, in scenarios where the graph type is known, the user writes content in the writing display table 2, the content written by the user is recognized, the data content in each cell in the writing display table is retrieved, and a line graph is drawn using the data content in that cell. Specifically, it is as follows.

[0200] [Table 2]

[0201] For example, the content corresponding to the first element in the dataset corresponding to each column is determined as the title of the coordinate axis. Specifically, the first element "Month" in the dataset corresponding to the first column is determined as the title of the X coordinate axis, the first element "Income" in the dataset corresponding to the second column is determined as the title of the Y coordinate axis, and the coordinate point (3,32) is determined based on the second element in the dataset corresponding to the first column and the second element in the dataset corresponding to the second column. Similarly, the coordinate point (4,44) is determined based on the third element in the dataset corresponding to the first column and the third element in the dataset corresponding to the second column. The coordinate point (5,57) is determined based on the fourth element in the dataset corresponding to the first column and the fourth element in the dataset corresponding to the second column. The coordinate point (6,14) is determined based on the fifth element in the dataset corresponding to the first column and the fifth element in the dataset corresponding to the second column. A line graph is drawn based on the coordinate axis titles and each coordinate point.

[0202] Case 4: Determine that the graph type is a pie chart.

[0203] Based on the dataset corresponding to a designated column, the title of each partition is determined. Based on the dataset corresponding to columns other than the designated column, the proportion of each partition is determined, and a pie chart is drawn based on the title and proportion of each partition.

[0204] In this embodiment, as shown in the handwriting display table 3, in scenarios where the graph type is known, the user writes content in the handwriting display table 3, the content written by the user is recognized, the data content in each cell of the handwriting display table is obtained, and a pie chart is drawn using the data content in those cells. Specifically, it is as follows.

[0205] [Table 3]

[0206] For example, each item in the dataset corresponding to the first column becomes the title of each partition. That is, the titles of each partition are February, March, April, and May, respectively, and the percentage of each partition corresponding to its title is determined based on the corresponding dataset in the second column. That is, the partition with the title February has a 10% occupancy rate, the partition with the title March has a 20% occupancy rate, the partition with the title April has a 30% occupancy rate, and the partition with the title April has a 40% occupancy rate. A pie chart is then drawn based on the title and occupancy rate of each partition.

[0207] In some embodiments, the graph type can also be determined by one of the following methods:

[0208] Method (1): The graph drawing command includes a graph selection command, and in response to the user's intelligent drawing graph mode, the graph type corresponding to the graph selection command is determined.

[0209] In this embodiment, the intelligent drawing graph mode may include a writing application that allows the user to write by some operation and draws the written content as a chart, or the intelligent drawing graph mode according to this embodiment may include an intelligent drawing graph mode that is activated in other applications such as a mode that has a graph drawing function in an electronic whiteboard handwriting application or an electronic interface that has a graph drawing function.

[0210] In this embodiment, when the user enters intelligent drawing graph mode on the electronic whiteboard, multiple graph type keys are displayed on the screen, and the user forms graph selection commands for different graph types according to the different keys. For example, when entering intelligent drawing graph mode, a bar graph, radar graph, line graph, and pie chart are displayed in the lower left corner of the display area, and when the user clicks on a bar graph, the display interface enters the intelligent drawing graph editing interface corresponding to the bar graph.

[0211] In some embodiments, in response to a graph selection command from the user in intelligent drawing graph mode, a writing suggestion table is displayed in the display area, and writing trajectory information entered by the user in the writing suggestion table is received. After the user stops writing and a predetermined time is waited, a graph of the graph type corresponding to the graph selection command is drawn and displayed.

[0212] Method (2), the graph drawing command includes a first touch gesture in which the user responds to a first touch gesture in intelligent drawing graph mode, recognizes a first figure generated from the first touch gesture, and determines a graph type based on the first figure.

[0213] In some embodiments, the graph drawing command includes at least one of the following: an arrow drawing command, a circle drawing command, a polygon drawing command, a cylindrical drawing command, and a rectangle drawing command, where a polygon includes a first figure having at least three sides. The arrow drawing command includes a single arrow drawing command, a bidirectional arrow drawing command, a command to draw two or more arrows, and so on. The user can use a touch component to draw the first figure on the display interface, and the processor can recognize the first figure type and determine different types of graphs based on different first figure types.

[0214] In some embodiments, after identifying a first shape corresponding to a graph drawing command, the generation of a selection menu is also included before determining the graph type. For example, after identifying a first shape, a candidate menu is generated that includes a selection icon for at least one graph type or a selection icon for one graph type and a selection icon for at least one other operation object, and the graph type is determined based on a selection operation (e.g., a click) by the user on the selection icon.

[0215] In some embodiments, before the user responds to a graph drawing command in intelligent drawing graph mode, the system receives handwriting trajectory information written by the user, recognizes the handwriting trajectory information, obtains data recognition results, and then, in response to the user's graph drawing command in intelligent drawing graph mode, draws and displays a graph of the graph type corresponding to the graph drawing command based on the data recognition results.

[0216] Selectively, the first shape generated based on the first touch gesture includes, but is not limited to, at least one of the following: an arrow shape, a circular shape, a rectangular shape, a cylindrical shape, or a polygonal shape.

[0217] If the first shape is an arrow shape, the graph type is determined to be a line graph. If the first shape is a circle shape, the graph type is determined to be a pie chart. If the first shape is a rectangle shape, the graph type is determined to be a histogram. If the first shape is a bar graph, the graph type is determined to be a bar graph. If the first shape is a polygon shape, the graph type is determined to be a radar map.

[0218] In this way, this embodiment can determine the graph type using one or more of the following methods.

[0219] Method (2a): If the graph drawing command includes an arrow drawing command, the graph type is determined to be a line graph. In this embodiment, when the user draws a bidirectional arrow, the system determines that the graph type is a line graph and triggers the generation of a line graph.

[0220] In some examples, the arrow drawing command in this embodiment may be, for example, a drawing command in which the user draws two vertical arrows, or a drawing command in which the user draws one double arrow, or for example, a single bidirectional arrow "⇔".

[0221] In some examples, if the command to draw a bidirectional arrow includes a first arrow command "→" and a second arrow command "↑", the X-axis direction of the drawn graph is determined based on the first direction pointed to by the first arrow command, and the Y-axis direction of the graph is determined based on the second direction pointed to by the second arrow command. Alternatively, if the command to draw a bidirectional arrow includes a first arrow command "↑" and a second arrow command "→", the X-axis direction of the drawn graph is determined based on the first direction pointed to by the first arrow command. The Y-axis direction of the graph is determined based on the second direction pointed to by the second arrow command. It should be noted that the command to draw an arrow may include not only commands that are understood as user-drawn arrows, but also commands that indicate that user-drawn figures are recognized as arrows, that is, that user-drawn line segments are recognized as arrows using the product pre-configuration procedure in this embodiment (i.e., user-drawn line segments that do not contain arrows correspond to arrows).

[0222] In this embodiment, it is also possible to create a graph by determining the direction of the X and Y axes in the graph by combining the correspondence between the X and Y axes and the cells in each column. For example, if the cells in the left column correspond to the X axis, the data content in the cells in the left column is data on the X axis. Similarly, if the cells in the right column correspond to the X axis, the data content in the cells in the right column is data on the Y axis. The displayed graph will differ depending on the direction of the X and Y axes. In this embodiment, a first arrow command that sets the X axis direction to horizontal may be received first, or a first arrow command that sets the X axis direction to vertical may be received first. That is, the X axis direction is determined based on the direction pointed to by the received first arrow, and then the Y axis direction is determined, thereby adjusting the coordinate axis direction of the graph so that the user can obtain more drawing needs.

[0223] Method (2b): If the graph drawing command includes a circular drawing command, the graph type is determined to be a pie chart. In this embodiment, if the user draws a circle, or a double-headed arrow and a circle, the graph type is determined to be a pie chart, and the generation of a pie chart is triggered.

[0224] Method (2c): If the graph drawing command includes a rectangle drawing command, the graph type is determined to be a histogram. In this embodiment, if a rectangle, or a bidirectional arrow and rectangle, is used to draw a rectangle, the graph type is determined to be a histogram, and the generation of a histogram is triggered.

[0225] Method (2d): If the graph drawing command includes a cylindrical drawing command, the graph type is determined to be a bar graph. In this embodiment, when the user draws a cylinder, the graph type is determined to be a bar graph, and the generation of a bar graph is triggered.

[0226] Method (2e): If the graph drawing command includes a polygon drawing command, the graph type is determined to be a radar map. In this embodiment, when the user draws a polygon, the graph type is determined to be a radar map, and the generation of a radar map is triggered.

[0227] The dataset according to this embodiment may consist of a single data item, or it may consist of multiple data items determined based on the contents of a table written by the user in the writing area.

[0228] In some examples, if a dataset has two data contents, the data contents in the same column may be considered the same dataset, or the data contents in the same row may be considered the same dataset, as determined according to actual needs, and are not limited to this embodiment. For example, if the plotted graph type is a line graph, the dataset contains two data, where one data represents the X-axis coordinate value and the other data represents the Y-axis coordinate value. If the plotted graph type is a pie chart, the dataset contains two data, where one data represents the type name and the other data represents the data corresponding to the type name. If the plotted graph type is a histogram, the dataset contains two data, where one data represents the X-axis coordinate value and the other data represents the Y-axis coordinate value.

[0229] In this embodiment, the dataset can also be determined by any of the following embodiments.

[0230] 2-1. Based on the pre-defined correspondence between each row and the coordinate axes, the data content of each column is determined as each dataset.

[0231] In this embodiment, when a user writes two rows of data, the number of rows represents the XY coordinate axes, and the number of columns represents the number of datasets. Of these, the top row corresponds to the X axis, and the bottom row corresponds to the Y axis, and the data content of each column determines each dataset (X, Y) from top to bottom. Alternatively, the top row corresponds to the Y axis, and the bottom row corresponds to the X axis, and the data content of each column determines each dataset (X, Y) from bottom to top.

[0232] 2-2. Based on the pre-defined correspondence between each column and the coordinate axes, the data content of each row is determined to form each dataset.

[0233] In this embodiment, when the user writes data in two columns, the number of rows represents the number of datasets, and the number of columns represents the X and Y coordinate axes. The leftmost column corresponds to the X axis, and the rightmost column corresponds to the Y axis, and the data content of each row is used to determine each dataset (X, Y) from left to right. Alternatively, the leftmost column corresponds to the Y axis, and the rightmost column corresponds to the X axis, and the data content of each row is used to determine each dataset (X, Y) from right to left.

[0234] In some examples, the titles for each coordinate axis are determined as follows:

[0235] Based on the pre-defined correspondence between each column and the coordinate axes, the data content of each row cell is determined as each dataset. Here, the content written by the user that is located in a predetermined row is determined as the title of each coordinate axis, and the data content in the other row cells is determined as each dataset.

[0236] Alternatively, based on the pre-set correspondence between each row and the coordinate axes, each column content written by the user is determined as each dataset. Here, the content located in a predetermined column among the content written by the user is determined as the title of each coordinate axis, and the data content in each other column cell is determined as each dataset.

[0237] In some examples, this embodiment can also determine which column of the two columns of cells is used as the X-axis data and which column is used as the Y-axis data based on the order of the drawing arrows, specifically as shown below.

[0238] Here, the data recognition result includes two columns of cells, and the drawing command of the bidirectional arrow includes a first arrow command and a second arrow command drawn sequentially. The first arrow command corresponds to the first column in the two columns of cells, and the second arrow command corresponds to the second column cell in the two columns of cells.

[0239] <00​​​​​​​​Here, the reception time of the first arrow command is earlier than the reception time of the second arrow command. The second direction of the second arrow command is different from the first direction. As can be easily understood, if the first direction of the first arrow command is the X-axis direction, the second direction of the second arrow command may be the Y-axis direction; if the first direction of the first arrow command is the Y-axis direction, the second direction of the second arrow command may be the X-axis direction. Here, the X-axis direction and the Y-axis direction may be defined in advance. The X-axis and the Y-axis directions are perpendicular. When the user draws the first arrow command and the second arrow command, there is a certain error allowed between the first direction of the normally drawn first arrow command and the X-axis direction, and similarly, there is a certain error allowed between the second direction and the Y-axis direction. The first arrow command and the second arrow command drawn within the error tolerance range can recognize the first direction of the first arrow command and the second direction of the second arrow command. Therefore, based on whether the recognized first direction is the X-axis direction or the Y-axis direction, a final determination can be made on whether to use the data in the first column as X-axis data or Y-axis data.

[0242] A graph display method in a note-taking scene without table lines. The principle of drawing the graph in this embodiment is to cell-process all the note-taking trajectory information of the user to obtain each cell after segmentation, and each cell contains note-taking trajectory information. The note-taking trajectory information in each cell is recognized by a handwriting recognition algorithm to obtain the data content in each cell, and the data content in each cell is recognized. That is, after recognizing each data content, each data set is determined from each data content, and each data set is drawn according to the graph type. Note that the table data includes the position of each cell and the data content in each cell. Here, the cell position is configured to determine which cell is one data set, and also to determine the connection order or the sorting order, etc. of the data sets. The data content is configured to determine the position of the specific data set on the coordinate axis, or the size of the data set in the graph, or the height and low of the data set in the graph.

[0243] In some embodiments, in writing scenes without table lines, the table data written by the user includes the title and / or data set for each coordinate axis. The table generated in this embodiment can be determined to add and display the titles for each coordinate axis to the generated table based on whether or not the table data contains the titles for each coordinate axis. In this embodiment, this is determined by pre-setting whether or not the table data contains the titles for each coordinate axis. For example, in this embodiment, the data in the top row of the table is pre-set as the titles for each coordinate axis, or the data in the leftmost column is pre-set as the titles for each coordinate axis. If the user knows in advance how the electronic whiteboard sets the titles for each coordinate axis, they may write the titles for each coordinate axis in the top row, or they may write the titles for each coordinate axis in the leftmost column. In this embodiment, the titles and data content of each coordinate axis in the table data are specifically determined in one of the following embodiments.

[0244] Method 1: The content written by the user that is located on a predetermined line is determined as the title of each coordinate axis, and the data content in each other cell is determined as the data content.

[0245] In this embodiment, titles for each coordinate axis may be drawn, and the data content in the first row cell of each cell may be used as the title for each coordinate axis, while the data content in other cells may be used as the data content.

[0246] Method 2: The content written by the user that is located in a predetermined column is determined as the title of each coordinate axis, and the data content in the other cells is determined as the data content.

[0247] In this embodiment, titles for each coordinate axis may be drawn, and the data content in the first column cell of each cell may be used as the title for each coordinate axis, while the data content in the other cells may be used as the data content.

[0248] Method 3: Determine the data content within each of the aforementioned cells as the data content.

[0249] In this embodiment, the data content within each cell can be graphed collectively as data content without distinguishing between the title and data content of each coordinate axis.

[0250] In some examples, graphs are plotted from table data in the following specific flow:

[0251] Flow 1: The graph type is determined according to the graph plotting command.

[0252] In this embodiment, there is a one-to-one correspondence between graph plotting commands and graph types.

[0253] Selectively, the graph types according to this embodiment include, but are not limited to, at least one of line graphs, pie charts, histograms, bar graphs, and bar charts.

[0254] Selectively, the graph drawing command according to this embodiment includes, but is not limited to, at least one of a touch gesture and a touch command. Here, the touch gesture includes at least one of a drawing gesture and a multi-finger touch gesture, and the touch command includes a click command.

[0255] In some examples, a touch gesture may include, but is not limited to, at least one of the following commands: drawing a bidirectional arrow, drawing a circle, drawing a polygon, drawing a cylinder, or drawing a rectangle. The user may also trigger the generation of a line graph when drawing a bidirectional arrow on the display interface or graph interface, or the generation of a pie chart when drawing a circle on the display interface or graph interface, or the generation of a histogram when drawing a rectangle on the display interface or graph interface, or the generation of a corresponding graph when clicking on the display interface, graph interface, or writing interface.

[0256] In some examples, the graph drawing instruction includes an instruction that is triggered after the user stops writing and waits for a predetermined time in the intelligent drawing graph mode of a known graph type.

[0257] In some examples, the graph drawing instruction includes at least one of a bidirectional arrow drawing instruction, a circular drawing instruction, a polygonal drawing instruction, a cylindrical drawing instruction, and a rectangular drawing instruction.

[0258] In Flow 3, based on each dataset corresponding to the content written by the user, a graph corresponding to the graph type is drawn.

[0259] Optionally, the plurality of datasets are determined by dividing the content written by the user column by column, or the plurality of datasets are determined by dividing the content written by the user row by row.

[0260] Determine the content of any column written by the user as one dataset, or determine the content of any row written by the user as one dataset.

[0261] Optionally, based on the pre-set correspondence between each row and the coordinate axis, determine the content of each column written by the user as each dataset, or based on the pre-set correspondence between each column and the coordinate axis, determine the content of each row written by the user as each dataset.

[0262] Optionally, determine the content located in a predetermined row among the content written by the user as the title of each coordinate axis, and determine the data content in each other cell as the data content, or determine the content located in a predetermined column among the content written by the user as the title of each coordinate axis, and determine the data content in each other cell as the data content.

[0263] Optionally, determine the data content in each cell as the data content.

[0264] In some examples, if titles exist for each axis, the graph may be drawn based on the graph type and each dataset, with the titles for each axis added to the corresponding axis positions. If there are no titles for each axis, the graph is drawn directly for each dataset. This embodiment can draw one or more types of graphs and can determine different types of graphs in response to a graph drawing command received from the user.

[0265] When a line graph is plotted, each determined dataset is understood as a coordinate point, and the line graph is plotted by connecting each coordinate point according to its relative ranking on the X or Y axis. When a histogram is plotted, each determined dataset is understood as a coordinate point, and the height of each coordinate point is determined from the Y-axis coordinate value at each coordinate point to plot the histogram. When a pie chart is plotted, each determined dataset contains data that corresponds one-to-one with each object, and the pie chart is plotted using the proportion of each data point to the total data.

[0266] As shown in Figure 4, when the graph type is a line graph, there are examples where XY coordinate axes are drawn and each dataset is labeled on the drawn XY coordinate axes. In this case, each dataset is treated as a coordinate point, and the coordinate points are connected in ascending order of the X-axis coordinate, or in ascending order of the Y-axis coordinate to obtain a line graph.

[0267] As shown in Figure 5, in some examples, when the graph type is a histogram, XY coordinate axes are drawn and each coordinate point is labeled on the drawn XY coordinate axes. In this case, each dataset can be treated as a coordinate point, and rectangles can be drawn based on the distance from each coordinate point to the X axis or the distance from each coordinate point to the Y axis. Here, the height of the rectangle is the distance from that coordinate point to the X axis or the distance from that coordinate point to the Y axis, and the width of the rectangle is a predetermined value.

[0268] As shown in Figure 6, in some examples, when the graph type is a histogram, each dataset contains one data item. Determine the proportion of the size represented by each data item to the sum of all data items, and then draw a pie chart based on the proportion of each data item.

[0269] In some embodiments, the graph drawing command includes a command that is triggered after the user stops writing and waits for a predetermined time in an intelligent drawing graph mode for a known graph type of drawing rectangles.

[0270] In some embodiments, a predetermined graph is displayed in the display area in response to a user editing command in smart graph mode of a known graph type, where the display mode of the predetermined graph is determined based on a known graph type.

[0271] In this embodiment, when the user selects a graph type, in the intelligent drawing graph mode for known graph types, if writing is stopped and a predetermined time is waited, the system can recognize the content written by the user on the writing presentation table, draw the graph, and display the drawn graph.

[0272] In some embodiments, this embodiment can also provide the user with a boot template that allows the user to fill in table contents based on the contents of a guide template, thereby standardizing the user's entry of table contents and improving recognition accuracy. In this embodiment, after determining the graph type, a guide template is displayed in the display area of ​​the display screen to guide the user in writing table contents according to the writing specifications corresponding to the graph type.

[0273] In some embodiments, a table template is displayed in the first partition of the guide template, a table template is displayed in the second partition of the guide template, and the table template contains predetermined content. The graph template contains a graph corresponding to the predetermined content.

[0274] In some embodiments, when it is determined that the user has stopped writing and no writing has been done after a predetermined time has passed, the system recognizes the writing trajectory information in the display area of ​​the display screen and obtains the data recognition result.

[0275] In some embodiments, the system recognizes trajectory points in the handwriting trajectory information of the display area of ​​the display screen where the pixel value of the trajectory point is greater than a pixel threshold, and obtains data recognition results.

[0276] In some embodiments, in response to an intelligent drawing graph activation command triggered by the user on a first display interface of a writing application, a second display interface in intelligent drawing graph mode is displayed in the display area.

[0277] After drawing and displaying a graph in the second display interface, the graph from the second display interface is inserted into the first display interface of the writing application and displayed in response to a user's insertion command.

[0278] The writing application in this embodiment is software that allows the user to write on a display interface via a stylus or finger and displays the content written by the user in real time. It includes, but is not limited to, other additional functions such as writing functionality, touch signal recognition functionality, and stylus recognition functionality. The writing functionality includes software entities that have basic writing functions such as writing, erasing, marking, annotating, and clearing, and in this embodiment, functions other than basic writing functions are not particularly limited.

[0279] After the user displays the graph, if annotation information related to the graph is added to the area where the graph exists, the graph and the annotation information are inserted into the first display interface and displayed in response to the user's insertion command.

[0280] In some embodiments, the intelligent drawing graph initiation command according to this embodiment includes, but is not limited to, at least one of a touch command or a touch gesture. Here, the touch command includes, but is not limited to, at least one of a click, a single click, or a double click. The touch gesture includes, but is not limited to, at least one of a writing command or a drawing command formed by the user touching with a finger or stylus.

[0281] In some embodiments, when the intelligent drawing graph activation command is a second touch gesture, a second display interface in intelligent drawing graph mode is displayed in the display area. Specifically, The system receives a second shape drawn by the user on a first display interface of a writing application, the second shape being a second shape generated based on the second touch gesture.

[0282] Based on the second figure, the system is triggered to display a second display interface in intelligent drawing graph mode in the display area.

[0283] In this embodiment, after the user draws a second figure on the first display interface of the writing application, the second display interface is triggered to enter intelligent drawing graph mode.

[0284] In some embodiments, after receiving a second figure drawn by the user on a first display interface of a writing application, the non-standard second figure drawn by the user is controlled to display a standard second standard rectangle on the display screen and recognized as a graphic. For example, if the rectangle's sides are not absolutely straight, after the processor performs recognition, it can control the display to display a standard rectangle on the display screen and generate a candidate menu on the first display interface. For example, next to the standard rectangle, the candidate menu includes a first launch icon corresponding to the second figure. The user can trigger a display of a graph-type second display interface corresponding to the second figure in the display area in response to the user's touch operation on the first launch icon.

[0285] In this embodiment, after the user draws a second shape on a first display interface of a writing application, a candidate menu is generated on the first display interface, and the candidate menu includes a first launch icon. The first launch icon is configured to launch a second display interface of the graph type corresponding to the second shape. Simultaneously, after a different second shape drawn by the user is recognized by the processor, a different first launch icon may be generated so that clicking the first launch icon leads to the second display interface. The graph types displayed by the second display interfaces generated by the different first launch icons will also be different. That is, the user may understand the click selection command of the first launch icon as a graph drawing command.

[0286] In some embodiments, the user according to this embodiment determines the corresponding graph type based on a second figure of a different shape drawn on the first display interface. The first launch icon in the candidate menu also differs depending on the shape drawn by the user. By displaying an icon in the candidate menu corresponding to the graph type of the second figure, the user can touch the icon to enter the second display interface for the corresponding graph type. Specific embodiments are as follows.

[0287] If the second shape is cylindrical, the first activation icon is a bar icon, and in response to a user touch operation on the bar icon, it triggers the display of a second display interface corresponding to a bar graph.

[0288] If the second shape is a rectangle, the first launch icon is a hist icon, and in response to a user touch operation on the hist icon, it triggers the display of a second display interface corresponding to a histograph.

[0289] If the second shape is circular, the first launch icon is a circular icon, and in response to a user touch operation on the circular icon, it triggers the display of a second display interface corresponding to a pie chart.

[0290] If the second figure is a polygon, the first activation icon is a radar icon, and in response to a user touch operation on the radar icon, it triggers the display of a second display interface corresponding to the radar map.

[0291] If the second shape is an arrow, the first activation icon is a line icon, and in response to a user touch operation on the line icon, it triggers the display of a second display interface corresponding to a line graph.

[0292] In some embodiments, the candidate menu further includes a second launch icon containing analysis template icons, which include, but are not limited to, SWOT (Enterprise Policy Analysis) icons and KANO (KANO model, a useful tool for classifying and prioritizing user needs, which shows a non-linear relationship between product performance and user satisfaction based on an analysis of the impact of user needs on user satisfaction) icons. After the second launch icon appears in the candidate menu, the following embodiments are performed.

[0293] In response to a user's touch operation on an analysis template icon, the analysis template corresponding to the analysis template icon is inserted into the first display interface.

[0294] In this embodiment, a SWOT icon is displayed in the candidate menu of the first display interface. When the user clicks this SWOT icon, the corresponding analysis template is inserted into the first display interface, and the user can perform editing operations and add content on that analysis template.

[0295] In some embodiments, for example, when a user draws a circle in the first display interface, a candidate menu appears in the first display interface, displaying a circle icon and a KANO icon. When the user clicks the circle icon, a pie chart can be drawn in the second display interface, which corresponds to a pie chart in intelligent drawing graph mode. When the user clicks the KANO icon, an analysis template corresponding to the first display interface is inserted, and the user can perform editing operations and add content on the analysis template. Embodiments corresponding to other shapes are also covered by the present invention based on the same principle, and their description is omitted here.

[0296] In some examples, this embodiment can also provide a method for dividing a display area to distinguish between a writing area that receives user writing and user writing trajectory information and a graph area that displays a graph. The specific implementation of the method before responding to the user's graph drawing command is as follows.

[0297] The display area of ​​the aforementioned display screen is configured to have a writing area and a graph area. The writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information.

[0298] The writing area is used to receive user writing path information and other touch operations. Users can perform normal writing, clicking, and other control operations in the writing area, while the graph area is used to display graphs. It is easy to understand that when the user writes the content they want to draw in the writing area, the graph corresponding to that content will be displayed in the graph area. By partitioning the display area, the user's writing path information received by the writing area can be displayed more clearly, and the drawn graph can be displayed in a graph area separate from the writing area. This improves the user experience.

[0299] To display a graph in the graph area, the system receives the user's touch gesture (writing trajectory information) in the graph area, waits for a predetermined time, and then triggers the display of the graph drawn in the graph area. In the writing area, after receiving the user's writing trajectory information and waiting for a predetermined time, other functions (such as the chart display function) are not triggered in order to ensure the normal writing function of the writing area. The system waits until the next time the user's writing trajectory information is received, allowing for efficient differentiation between the writing area and the graph area.

[0300] In some examples, the system recognizes the writing trajectory information within the writing area and, in response to the user's graph drawing command, draws a graph of the graph type corresponding to the graph drawing command based on the data recognition results, and the drawn graph is displayed in the graph area.

[0301] In this embodiment, at least one of a writing suggestion table and a predetermined graph is displayed in the display area of ​​the display screen, the data content in the writing suggestion table is empty, and the user can write table content into each cell of the writing suggestion table. In selectable embodiments, a writing area and a graph area are determined in the display area of ​​the display screen, the writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information. In the writing area of ​​the display area, a writing suggestion table corresponding to the graph type is displayed, and / or in the graph area of ​​the display area, a predetermined graph corresponding to the graph type is displayed, where the content in the predetermined graph is also empty, that is, only the graph specification is displayed without displaying the graph content. For example, the graph type is a predetermined graph of a pie chart, which can be a pie chart without partitions.

[0302] In some embodiments, after responding to a graph selection command from the user in intelligent drawing graph mode, determining the graph type corresponding to the graph selection command, or after displaying a guide template corresponding to the graph type in the display area of ​​the display screen, the processor is configured to perform the following actions:

[0303] In response to a user's editing command in a smart graph mode of a known graph type, the display area is shown a writing suggestion table and a predetermined graph. The system receives the writing trajectory information entered by the user in the writing suggestion table.

[0304] At this time, both the handwriting display table and the predetermined graph are empty. The handwriting display table is used to provide an empty table for easy input of table contents, and the predetermined graph provides graph specifications to easily draw content on the predetermined graph based on the table contents entered by the user. Here, the number of rows and columns of the handwriting display table is fixed, or the number of rows and columns of the handwriting display table is set based on parameters entered by the user, and the display specifications of the predetermined graph are determined based on the graph type corresponding to the guide template.

[0305] In some examples, the graph area is configured to receive graph drawing commands from the user. In some examples, when the user receives a graph drawing command in the graph area and waits for a predetermined time, the system recognizes the writing trajectory information in the writing area in response to the user's graph drawing command, draws a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and displays the drawn graph in the graph area.

[0306] In this embodiment, after determining the writing area and graph area within the display area of ​​the display screen, a graph drawing command from the user is received. Here, the graph drawing command can be a graph drawing command triggered by the user in the graph area, or a graph drawing command triggered by the user in the writing area.

[0307] The graph drawing command is a touch gesture, and the user triggers the graph drawing command in the graph area. For example, the user draws a bidirectional arrow in the graph area. The electronic whiteboard according to this embodiment is equipped with character recognition and shape recognition functions. In order to distinguish between the user's writing trajectory information in the writing area and the user's writing trajectory information in the graph area, when recognizing the writing trajectory information, the writing trajectory information in the writing area is recognized using character recognition and / or digital recognition, and the writing trajectory information in the graph area is recognized using shape recognition, mainly recognizing at least one of arrows, circles, or rectangles. Different recognition methods allow for faster and more accurate recognition of the writing trajectory information in the writing area, and also allow for rapid recognition of the user's graph drawing command in the graph area and rapid determination of the drawn graph type.

[0308] In some embodiments, it is also decided to divide the display area into a writing area and a graphing area, as follows:

[0309] In response to a user's editing command in smart graph mode for a known graph type, the display area is divided into a writing area and a graph area, and the graph area is configured to display a graph corresponding to the graph type.

[0310] In some embodiments, if the user has already selected a graph type and is in intelligent drawing graph mode for a known graph type, the display area can be divided into a writing area and a graph area by directly triggering an edit command, making it easier for the user to perform operations such as editing and writing in the writing area, and displaying the writing trajectory information in the writing area in the graph area.

[0311] In some embodiments, a writing area and a graph area are determined within the display area of ​​the display screen, the writing area is configured to receive user writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information. A writing presentation table corresponding to the graph type is displayed in the writing area of ​​the display area, and / or a predetermined graph corresponding to the graph type is displayed in the graph area of ​​the display area. In this embodiment, the user can write content in each cell of the writing presentation table and draw content on the predetermined graph based on the written content.

[0312] In some embodiments, the user can also drag the contents of the writing area to change the position of the contents of the writing area, and drag the contents of the graph area to change the position of the contents of the graph area.

[0313] In some embodiments, the contents of a writing area are moved in a corresponding direction in response to a drag command from the user to a writing area, or the contents of a graph area are moved in a corresponding direction in response to a drag command from the user to a graph area.

[0314] In some embodiments, the writing presentation table within the writing area according to this embodiment is draggable, and in response to a drag command from the user to the writing area, the writing presentation table is moved in the corresponding direction, or a predetermined graph within the graph area is also draggable, and in response to a drag command from the user to the graph area, the predetermined graph is moved in the corresponding direction.

[0315] In some embodiments, after receiving writing trajectory information entered by the user on the writing display table, the user stops writing in the writing area and waits for a predetermined time. At this time, the writing trajectory information on the writing display table is recognized, data recognition results are obtained, and based on the data recognition results, the data is plotted on the predetermined graph, and a graph corresponding to the data recognition results is obtained.

[0316] In some embodiments, after obtaining a graph corresponding to the data recognition result, the following embodiments are further included.

[0317] (1) In response to a user's correction command, the corresponding writing trajectory information is corrected, and the graph is updated based on the corrected writing trajectory information.

[0318] In this embodiment, if a user partially modifies the existing content of the writing area, the existing content of the writing area is first replaced with the modified writing trajectory information based on the modified writing trajectory information, the graph is redrawn using the modified content of the writing area, and the existing graph in the graph area is updated with the redrawn graph.

[0319] (2) In response to a user's command to add content, the system adds the corresponding writing trajectory information and updates the graph based on the added writing trajectory information.

[0320] In this embodiment, when a user adds new content based on the existing content of the writing area, the graph can be redrawn not only from the writing area but also from the writing trajectory information within the newly added writing area, and the redrawn graph can be updated and displayed in the graph area. Note that the new content is not limited to inserting new content before or in the middle of the original content. For example, inserting 0 into the original 12 results in new content of 102, or inserting 3 before 12 results in new content of 312.

[0321] (3) In response to the user's command to add a stroke, the corresponding stroke in the writing trajectory information is added, and the graph is updated based on the writing trajectory information after the stroke has been added.

[0322] In this embodiment, strokes can be added in addition to the original characters and numbers, the content written by the user can be re-recognized from the added strokes, the graph can be redrawn from the writing trajectory information after the strokes have been added, and the original graph can be updated with the redrawn graph.

[0323] In some embodiments, it is possible to determine whether the newly added writing trajectory information modifies the original content, is added to the original content, or is added to the original stroke, based on the position of the newly added writing trajectory information and the overlapping area with the existing writing trajectory information. In this embodiment, for example, if the position of the newly added writing trajectory information overlaps with the position of the existing writing trajectory information and the overlapping area is greater than a threshold, it is determined that the newly added writing trajectory information is a modification to the original content; if the position of the newly added writing trajectory information overlaps with the position of the existing writing trajectory information and the overlapping area is not greater than a threshold, it is determined that the newly added writing trajectory information is an increase to the original stroke; and if the position of the newly added writing trajectory information does not overlap with the position of the existing writing trajectory information and the position of the newly added writing trajectory information is located before or in the middle of the position of the existing writing trajectory information, it is determined that the newly added writing trajectory information is newly added content inserted before or in the middle of the original content.

[0324] (4) In response to the user's erase command, the corresponding writing trajectory information is erased, and the graph is updated based on the erased writing trajectory information.

[0325] In this embodiment, the user's erase command includes, but is not limited to, a palm erase command or an eraser command. Here, a palm erase command is a command that is activated when the user erases the corresponding content in the writing area using a palm simulation eraser, and an eraser command is a command that is activated when the user clicks the eraser option in the menu bar.

[0326] (5) In response to the user's clear command, all writing path information is cleared.

[0327] In this embodiment, the system can also be configured to clear the screen with a single click. Triggering a clear command clears all writing trajectory information in the writing area while retaining the graph in the graph area.

[0328] In some embodiments, after the display area is divided into a writing area and a graphing area, the user can execute one of the following commands on the graphing area:

[0329] In response to the user's full-screen preview command, the graph area is displayed in full screen, or, In response to a user's annotation operation in the graph area, display content corresponding to the said annotation operation, or In response to a user's erase operation in the graph area, either the content corresponding to the erase operation in the graph area is erased, or In response to a user's clear operation in the graph area, either clear the contents of the graph area, or In response to a user save operation in the graph area, either save the contents of the graph area, or In response to the user's command to switch graph types, either switch the current graph type or, In response to a user's command to insert into a graph area, the contents of the graph area of ​​the current display interface are inserted into the previous display interface.

[0330] In some examples, after determining the writing area and graph area within the display area of ​​the display screen, the system receives the user's writing trajectory information in the writing area, and in response to the user's graph drawing command in the graph area, it recognizes the writing trajectory information within the writing area, draws a graph of the corresponding type based on the determined table data based on the data recognition result, and displays the graph in the graph area.

[0331] The generated graph area is part or all of the display area of ​​the electronic whiteboard. If the generated graph area is part of the display area, the writing area of ​​the remaining display area receives the user's writing trajectory information and touch operations. The graph area is configured to receive the user's graph drawing command and display the graph. Here, the graph drawing command includes at least one of a touch gesture and a touch command, and the present invention is not particularly limited.

[0332] In some examples, this embodiment can divide the display area into a writing area and a graphing area by one of the following methods:

[0333] Method 1: The display area is divided in advance into a writing area and a graphing area.

[0334] In this method, when the device starts up, the writing area and graphing area are displayed directly in the display area, or, When the designated program is launched, the writing area and graph area are displayed directly on the display area.

[0335] In this embodiment, for example, in the intelligent drawing graph mode of the electronic whiteboard, the display area is divided into a writing area and a graph area in response to an intelligent drawing graph mode triggered by the user, or in response to an editing command from the user in a smart graph mode of a known graph type, the display area is divided into a writing area and a graph area.

[0336] Method 2: After receiving a partition button triggered by the user, the display area is divided into a writing area and a graphing area.

[0337] In this system, the user can perform normal writing interactions across the entire area of ​​the electronic whiteboard before receiving a partition button trigger. When the partition button is triggered, it triggers again, generating a new display area, which is then divided into a writing area and a graphing area.

[0338] Method 3: After confirming that the user has finished writing, the display area is divided into a writing area and a graph area.

[0339] In this system, the divided writing areas contain information about the writing trajectory recorded by the user.

[0340] Furthermore, in any of the above methods, users can distinguish between partitions of the display area by displaying a label, such as "Writing Area" in the writing area and "Graph Area" in the graph area.

[0341] In some examples, the display area of ​​this embodiment is divided into a writing area and a graphing area. The relationship between the writing area and the graphing area includes, but is not limited to, one or more of the following:

[0342] In the first case, as shown in Figure 7A, the positions of the writing area and the graph area do not overlap.

[0343] In the second case, as shown in Figure 7B, the graph area floats above the writing area and does not obstruct the contents of the writing area.

[0344] In the third case, the writing area and the graphing area have an axially symmetric relationship.

[0345] In the fourth type, as shown in Figure 7C, the writing area and the graphing area have a centrally symmetrical relationship.

[0346] In type 5, as shown in Figure 7D, the writing area and the graph area are either different in shape, or the writing area and the graph area have the same shape.

[0347] Type 6, as shown in Figure 7E, either the background of the writing area and the graph area are different, or the background of the writing area and the graph area are the same.

[0348] Type 7: The writing area and the graph area are of different sizes, or the writing area and the graph area are of the same size.

[0349] The relationship between the writing area and the graphing area described above is merely one example; other relationships between the writing area and the graphing area based on the same inventive concept also fall within the scope of protection of this invention and are not particularly limited here.

[0350] In some examples, the display area of ​​this embodiment is divided into a writing area and a graph area. Here, each of the writing area and the graph area occupies half of the display area, that is, the display area is equally divided into the writing area and the graph area. Alternatively, the display area is divided into a writing area and a graph area according to the proportional relationship between the area of ​​a predetermined writing area and the area of ​​the graph area. The shape and size of the writing area and the graph area may be the same or different, and are not particularly limited here.

[0351] In some examples, the graph region according to this embodiment is determined by one of the following methods.

[0352] Method 1: Based on predefined position information, the graph area within the display area of ​​the display screen is determined.

[0353] In this embodiment, after the electronic whiteboard is powered on or a predetermined program is started using this method, the display area is divided into a writing area and a graph area. The graph area can be positioned to the right, left, above, or below the writing area.

[0354] Method 2: The user responds to a first partition command within the display area and determines a graph area corresponding to the first partition command within the display area.

[0355] This method selectively pre-sets graph areas corresponding to different first partition instructions, and determines the graph area corresponding to a received first partition instruction based on that instruction. In this method, after the user sends a first partition instruction from the display area, the display area is divided into a writing area and a graph area. The position of the graph area corresponds to the first partition instruction. If the user has not received a first partition instruction, the display area is treated as a writing area and normal writing is performed. Here, different first partition instructions may be triggered by different keys, but are not particularly limited. In another embodiment, a graph area is pre-set, and when a first partition instruction is received, the pre-set graph area is displayed as is. In this method, when the user sends a first partition instruction in the display area, the display area is divided into a writing area and a graph area, and the position of the graph area is fixed.

[0356] In this embodiment, when there is no drawing request, the user can use the electronic whiteboard functions normally and perform normal writing interactions. When the user requires graph drawing, they can trigger a key on the current display area or partition the electronic whiteboard using other methods such as a touch pen key. Even after partitioning, the user can perform normal writing interactions in the writing area. When a graph drawing command is triggered in the graph area and a predetermined time has elapsed, the graph is displayed in the graph area.

[0357] In some cases, the user can partition the partitioned display area again, display the drawn graph in the graph area, and then partition the writing area and graph area again to convert the table contents that the user writes again into the corresponding graph and display it. In this case, the writing area contains the two table contents written by the user. The corresponding graph area contains two graphs corresponding to the two table contents written by the user. Specifically, this is as follows:

[0358] In response to a second partition command from the user, the writing area is divided into a first writing area and a second writing area, and the graph area is divided into a first graph area and a second graph area.

[0359] In this embodiment, the positional relationship between the first writing area and the second writing area, which are partitioned, may be adjacent or separated by a graph area. Similarly, the positional relationship between the first graph area and the second graph area may be adjacent or separated by a writing area. For example, in an example where both the writing area and the graph area are partitioned, the resulting first writing area and second writing area, the first graph area and the second graph area are, in the display area, arranged from left to right as follows: first writing area, second writing area, first graph area, second graph area, second graph area. Alternatively, they are, in order from left to right as follows: first writing area, first graph area, second writing area, second graph area. Alternatively, they are, in order from left to right as follows: first writing area, first writing area, second writing area, second graph area. In this embodiment, the positional relationship between the divided writing area and the graph area is not particularly limited.

[0360] Here, the first writing area is configured to display writing trajectory information received before division, the second writing area is configured to display writing trajectory information received in the second writing area after division, the first graph area is configured to display a graph corresponding to the writing trajectory information in the first writing area, and the second graph area is configured to display a graph corresponding to the writing trajectory information in the second writing area.

[0361] In this embodiment, after dividing the writing area into a first writing area and a second writing area, the second writing area can receive the user's writing trajectory information, and the user can input content that needs to be generated into the second writing area to generate a table. The content written by the user before the division is still stored in the first writing area. Selectively, the first writing area can also receive the user's writing trajectory information. The first graph area still displays the graph corresponding to the content written by the user before the division. The second graph area displays the graph corresponding to the newly written table content by the user. This allows meeting users to view two sets of table data and their corresponding tables simultaneously, facilitating comparison and improving the user's interactive meeting experience.

[0362] Furthermore, after receiving a second partition command from the user, before dividing the writing area into a first and second writing area, it is possible to perform operations such as scaling and panning of the trajectory information within the writing area. In response to the second partition command from the user, all the trajectory information in the writing area is moved to the first writing area, and the original trajectory information can be scaled according to the size of the first writing area.

[0363] In some examples, this embodiment further provides a method for correcting handwriting trajectory information and displaying it comparatively in a graph. By acquiring the corrected handwriting trajectory information and determining the data content within the corresponding corrected table, the graph is redrawn for all corrected data content and is designed to display the redrawn graph and the original table before correction on the same coordinate axis or on different pie charts, thereby visualizing and comparing the data content in the original table before correction and the data content in the corrected table in graph form, effectively improving the user's interactive experience in meetings.

[0364] In some cases, in response to a user's correction command, the corrected writing trajectory information is acquired and stored in a first memory area, and the memory area in which the original writing trajectory information exists is different from the first memory area.

[0365] In this embodiment, by storing the original handwriting trajectory information and the corrected handwriting trajectory information in different memory areas, it becomes possible to retrieve the handwriting trajectory information from the different memory areas more quickly when recognizing the handwriting trajectory information, and it becomes possible to distinguish between the corrected handwriting trajectory information and the handwriting trajectory information before correction during recognition.

[0366] In some examples, the user's modification commands according to this embodiment include, but are not limited to, commands triggered by a different tip of the stylus, commands triggered by a different key on the stylus, and commands triggered by a display area switching button.

[0367] In this embodiment, when a user modifies the original writing content in the writing area, this embodiment determines whether or not the writing trajectory has been modified as follows.

[0368] Method 1: Based on the received writing switch command from the stylus, it is determined that the writing trajectory information has been modified.

[0369] Specifically, the system determines that it has received a command to switch the stylus writing mode using the following method:

[0370] Method 1-1: Based on the received writing command for a different pen tip of the stylus, it is confirmed that a command to switch the writing direction of the stylus has been received.

[0371] In this embodiment, when using a dual-touch pen, the writing commands touched by the dual-touch pen will be different. If the first pen is used when writing in the writing area for the first time, and the second pen is used when correcting the writing content, the electronic whiteboard receives a writing command triggered by the second pen, then determines that it has received a command to switch the touch pen, and at this time determines that the writing trajectory information has been corrected. When graphing the data later, the writing trajectory information triggered by the second pen is stored in the first memory area.

[0372] Method 1-2 determines that a writing switch command for the stylus has been received, based on the selection command triggered by a different key on the received stylus.

[0373] In this embodiment, when writing in the writing area for the first time, the first stylus corresponding to the first button is used, and when correcting the written content, the electronic whiteboard receives a writing command triggered by the second stylus, and then determines that it has received a command to switch the stylus writing method. At this time, it is determined that the writing trajectory information has been corrected, and the writing trajectory information triggered by the second stylus is stored in the first memory area for use when drawing graphs later.

[0374] Method 2: Based on a switching command triggered by the user pressing the display area switching button, it is determined that the handwriting trajectory information has been modified.

[0375] In some examples, the system determines that the writing trajectory information has been changed based on a switching command from the receiving user to the writing area switching button. In one embodiment, a switching button is provided in the graph area. When the user clicks the switching button once (turns the switching button on), the writing trajectory information is stored in a first memory area. When the user clicks the switching button twice (turns the switching button off), the writing trajectory information is stored in a memory area other than the first memory area.

[0376] In some examples, in response to user modification commands, this embodiment can also distinguish and display the modifications made by the user. Specifically, A step of determining the first content before modification and the second content after modification that correspond to the modification order, The step includes simultaneously displaying the first content and the second content, The display formats for the content of the first and second sections described above are different.

[0377] Here, if the display colors of the first and second contents are different, or the font styles are different, or the font sizes are different, or if a deletion mark is placed on the first content and the second content is displayed in bold, it will look like Figures 8A and 8B.

[0378] In some cases, after acquiring the corrected writing trajectory information of this embodiment and storing it in a first memory area, if the user issues another graph drawing command, the system recognizes all the corrected writing trajectory information and draws a graph. Here, the coordinate axes used by the graph and the coordinate axes used by the graph in which all the original writing trajectory information before correction is drawn are the same, and based on the retention of the originally displayed graph, the user can perform data matching. To improve the interactive experience of the meeting, the system redraws the graph corresponding to the corrected writing trajectory information on the coordinate axes of the graph.

[0379] In some examples, the system is configured to retrieve the corrected writing trajectory information, store it in a first memory area, and then perform the next step.

[0380] Step 1: The corrected writing trajectory information is obtained from the first memory area, and a first graph is drawn based on the original writing trajectory information and the corrected writing trajectory information.

[0381] In this embodiment, after acquiring the corrected writing trajectory information, when the user's graph drawing command is received again, the generation of the first graph is triggered in response to the user's graph drawing command again. Here, the first graph includes first table data corresponding to the writing trajectory information report before correction and second table data corresponding to the corrected writing trajectory information report.

[0382] In some examples, the first graph is drawn in the following steps.

[0383] Step 1-1: From among the cells separated by the writing trajectory information before correction, determine the first cell corresponding to the corrected writing trajectory information report.

[0384] Step 1-2: Replace the uncorrected writing trajectory information in the first cell with the corrected writing trajectory information.

[0385] Steps 1-3: Recognize the handwriting trajectory information within each replaced cell and obtain the data content within each cell. In this embodiment, a handwriting recognition algorithm is used to recognize the handwriting trajectory information and obtain the data content within each cell.

[0386] Steps 1-4: Based on the data content in each of the aforementioned cells, the second table data is determined.

[0387] Step 2: The first table data corresponding to the handwriting trajectory information report before correction and the second table data are simultaneously displayed on the first graph, with different display specifications for the first table data and the second table data.

[0388] In some examples, as shown in Figure 9A, when the graph type corresponding to the graph drawing command is a line graph, the first table data and the second table data are displayed simultaneously on the same coordinate axis.

[0389] As shown in Figure 9B, when the graph type corresponding to the graph plotting command is a histogram, the first table data and the second table data are displayed simultaneously on the same coordinate axis.

[0390] In some cases, after obtaining the corrected writing trajectory information and storing it in a first memory area, it is possible to draw a second graph and a third graph, which is specifically done as follows.

[0391] The corrected writing trajectory information is obtained from the first memory area, a second graph is drawn based on the writing trajectory information before correction, and a third graph is drawn based on the corrected writing trajectory information. The second graph and the third graph are displayed simultaneously.

[0392] As shown in Figure 9C, if the graph type corresponding to the graph drawing command is a pie chart, the second graph and the third graph are displayed simultaneously as different pie charts.

[0393] The display specifications for the second graph and the third graph are different. In some examples, the display style includes, but is not limited to, color, shape, and line thickness. In this embodiment, to distinguish between the second graph and the third graph, they may be drawn using lines of different colors, lines of different shapes, or lines of different thicknesses, but this is not particularly limited.

[0394] In some examples, the recognition of handwriting trajectory information (including numbers, shapes, titles for each coordinate axis, etc.) in this embodiment can be achieved using neural networks such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), and for example, a long short-term memory (LSTM) model can be used.

[0395] In some embodiments, this embodiment also provides a text recognition model that recognizes handwriting trajectory information, as shown below.

[0396] The handwriting trajectory information is input to a text recognition model for recognition, and data content corresponding to the handwriting trajectory information is output. The text recognition model includes multiple BiLSTM network layers and a fully connected layer. At least two outputs are connected to the input of any BiLSTM network layer except the first two, and at least three outputs are connected to the fully connected layer.

[0397] The at least two outputs include outputs from any two BiLSTM network layers prior to any one of the BiLSTM network layers, and the at least three outputs include outputs from any three BiLSTM network layers prior to the fully connected layer.

[0398] In some embodiments, the output from any BiLSTM network layer prior to the last BiLSTM network layer is used as the input to at least two subsequent BiLSTM network layers.

[0399] In some embodiments, any of the BiLSTM network layers are obtained after removing network parameters using a Dropout layer during model training. Any of the BiLSTM network layers include an implicit number of layers less than a predetermined value.

[0400] In some embodiments, the text recognition model includes four BiLSTM network layers and a fully connected layer. The first BiLSTM network layer includes a first input and a first output, the second BiLSTM network layer includes a second input and a second output, the third BiLSTM network layer includes a third input and a third output, and the fourth BiLSTM network layer includes a fourth input and a fourth output.

[0401] The first input is configured to input the handwriting trajectory information. The second input connects the first output. The third input connects the first output and the second output. The fourth input connects the first output, the second output, and the third output. The fourth output connects the fully connected layer inputs, and the output from the fully connected layer is configured to output data content corresponding to the handwriting trajectory information.

[0402] In some examples, as shown in Figure 10, a graph drawing method provided by an embodiment of the present invention in a writing scene without table lines will be explained using a partition as an example.

[0403] Step 1000: Determine the writing area and graph area of ​​the electronic whiteboard's display area.

[0404] Step 1001: The system receives the user's writing trajectory information in the writing area, confirms that writing is complete, and then acquires all the writing trajectory information in the writing area and stores it in the memory area.

[0405] Step 1002: In response to a graph drawing command triggered by the user in the graph area, the handwriting trajectory information in the memory area is divided into individual cells using a clustering algorithm, the handwriting trajectory information in each cell is recognized based on a handwriting recognition algorithm, and the first table data is obtained.

[0406] Step 1003: Based on the first table data, a graph of the graph type corresponding to the graph drawing command is drawn.

[0407] Step 1004: Display the drawn graph in the graph area.

[0408] Step 1005: In response to a correction command triggered by the user in the writing area, the corrected writing trajectory information is obtained and stored in the first memory area.

[0409] Step 1006: In response to a graph drawing command triggered by the user in the graph area, the corrected writing trajectory information is retrieved from the first storage area, and a first graph is drawn based on the original writing trajectory information and the corrected writing trajectory information. Here, the first graph includes a first table data and a second table data corresponding to the corrected writing trajectory information.

[0410] Here, the display methods for the first table data and the second table data are different. For example, the display colors and line formats may differ.

[0411] In some examples, as shown in Figure 11, the implementation flow for drawing a line graph provided by an embodiment of the present invention in a writing scene without table lines is specifically as follows:

[0412] Step 1100: Determine the writing area and graph area of ​​the electronic whiteboard's display area.

[0413] Step 1101: The system receives the user's writing trajectory information in the writing area, confirms that writing is complete, and then acquires all the writing trajectory information in the writing area and stores it in the memory area.

[0414] Step 1102: In response to a triggered bidirectional arrow drawing command in the graph area, the handwriting trajectory information in the memory area is divided into individual cells according to an automatic cell algorithm, the handwriting trajectory information in each cell is recognized based on a handwriting recognition algorithm, and the first table data is obtained.

[0415] Step 1103: Draw a line graph based on the data in the first table.

[0416] Step 1104: Display the drawn line graph in the graph area.

[0417] Step 1105: In response to a correction command triggered by the user in the writing area, the corrected writing trajectory information is obtained and stored in the first memory area.

[0418] Step 1106: In response to a triggered bidirectional arrow drawing command in the graph area, the system retrieves the corrected writing trajectory information from the first storage area and draws a first line graph based on the original writing trajectory information and the corrected writing trajectory information. Here, the first line graph includes the first table data and second table data corresponding to the corrected writing trajectory information.

[0419] Here, the display methods for the data in the first table and the data in the second table are different.

[0420] In some examples, as shown in Figure 12, the implementation flow for drawing a histogram provided by an embodiment of the present invention in a writing scene without table lines is specifically as follows:

[0421] Step 1200: Determine the writing area and graph area of ​​the electronic whiteboard's display area.

[0422] Step 1201: The system receives the user's writing trajectory information in the writing area, confirms that writing is complete, and then acquires all the writing trajectory information in the writing area and stores it in the memory area.

[0423] Step 1202: In response to a rectangle drawing command triggered by the user in the graph area, the handwriting trajectory information in the memory area is divided into individual cells according to an automatic cell algorithm, the handwriting trajectory information in each cell is recognized based on a handwriting recognition algorithm, and the first table data is obtained.

[0424] Step 1203: Draw a histogram based on the first table data.

[0425] Step 1204: Display the drawn histogram in the graph area.

[0426] Step 1205: In response to a correction command triggered by the user in the writing area, the corrected writing trajectory information is obtained and stored in the first memory area.

[0427] Step 1206: In response to a rectangle drawing command triggered by the user in the graph area, the corrected writing trajectory information is retrieved from the first storage area, and a first histogram is drawn based on the original writing trajectory information and the corrected writing trajectory information. Here, the first histogram includes the first table data and second table data corresponding to the corrected writing trajectory information.

[0428] Here, the display methods for the data in the first table and the data in the second table are different.

[0429] In some examples, as shown in Figure 13, the implementation flow for drawing a pie chart provided by an embodiment of the present invention in a writing scene without table lines is specifically as follows:

[0430] Step 1300: Determine the writing area and graph area of ​​the electronic whiteboard's display area.

[0431] Step 1301: The system receives the user's writing trajectory information in the writing area, confirms that writing is complete, and then acquires all the writing trajectory information in the writing area and stores it in the memory area.

[0432] Step 1302: In response to a circular drawing command triggered by the user in the graph area, the clustering algorithm divides the handwriting trajectory information in the memory area into individual cells, recognizes the handwriting trajectory information of each cell based on the handwriting recognition algorithm, and retrieves the data content within each cell.

[0433] Step 1303: Draw the first pie chart based on the data content in each cell.

[0434] Step 1304: Display the drawn first pie chart in the graph area.

[0435] Step 1305: In response to a correction command triggered by the user in the writing area, the corrected writing trajectory information is obtained and stored in the first memory area.

[0436] Step 1306: In response to a circular drawing command triggered by the user in the graph area, the corrected writing trajectory information is retrieved from the first memory area, and a second pie chart is drawn based on the original writing trajectory information and the corrected writing trajectory information.

[0437] Step 1307: Display the first pie chart and the second pie chart simultaneously, where the display methods for the first pie chart and the second pie chart are different.

[0438] In some embodiments, as shown in Figure 14, the method for displaying a graph in a writing scene with table lines provided by this embodiment specifically performs the following steps.

[0439] Step 1400: In response to a graph selection command from the user in intelligent drawing graph mode, the graph type corresponding to the graph selection command is determined.

[0440] Step 1401: A guide template corresponding to the graph type is displayed in the display area of ​​the display screen. The first partition of the guide template displays a table template, and the second partition displays a graph template.

[0441] Step 1402: In response to a user editing command in smart graph mode for a known graph type, the display area is divided into a writing area and a graph area, a writing suggestion table is displayed in the writing area, and a predetermined graph is displayed in the graph area.

[0442] Step 1403: The system receives the handwriting trajectory information that the user has written on the handwriting presentation table.

[0443] Step 1404: In intelligent drawing graph mode for known graph types, when the user stops writing and waits for a predetermined time, the system recognizes the writing trajectory information on the writing presentation table, obtains the data recognition result, draws a predetermined graph based on the data recognition result, and displays a graph of the corresponding graph type.

[0444] In some embodiments, the data recognition results include multiple datasets, each dataset containing the content written by the user in any column of the handwriting presentation table, or the content written by the user in any row of the handwriting presentation table. In this embodiment, the handwriting trajectory information contained in each cell is determined based on the positional relationship between each trajectory point in the handwriting trajectory information and the cell in the handwriting presentation table. The handwriting trajectory information contained in each cell is recognized, and the data content of each cell is obtained. Multiple datasets are obtained by dividing the data content of each cell by column, or by dividing the data content of each cell by row.

[0445] As shown in Figures 15A to 15E, in a writing scene with table lines, we will use a bar graph as an example and explain the different interfaces of the electronic whiteboard display during the process of displaying the graph as follows.

[0446] As shown in Figure 15A, the user, in intelligent drawing graph mode, triggers a graph selection command - bar graph - and in response to the graph selection command, determines that the graph type corresponding to the graph selection command is a bar graph and displays a guide template corresponding to the bar graph in the display area, the guide template including a first partition and a second partition, where the first partition is an editing area and configured to display a table template, and the second partition is a graph preview area and configured to display a graph template, where the first column of the first row in the table template is grayed out to indicate that it is not writable, and the remaining columns are configured to display the writable content corresponding to each column. For example, the content of the first column is the horizontal axis title, and the content of the first row is the example and is configured to display the cylinder name. Different examples are indicated by cylinders with different fill content. The remaining content is specific data, indicating the corresponding title and cylinder height of the example. Additionally, an edit button is displayed in the display interface, or a close button is clicked in the display interface to trigger an edit command for the user. Furthermore, if a user switches to another table after editing a bar graph and then returns to the bar graph, the edited content will be displayed as is, and the bar graph's guide template will not be accessed. The guide template will only be displayed the first time the intelligent plotting graph function for a known graph type is turned on.

[0447] As shown in Figure 15B, the user triggers an edit command in the display interface of the bar graph guide template. In response to the user's edit command in the intelligent drawing graph mode of the bar graph, the display area is divided into a writing area and a graph area. A writing suggestion table is displayed in the writing area, and a predetermined graph is displayed in the graph area. Here, the data content in the writing suggestion table is empty, the cell in the first row and first column is not editable, and the number of rows and columns in the writing suggestion table is constant. For example, a 3x12 table consisting of 4*13 table lines can be constructed. The writing suggestion table is composed of table lines. The style of the predetermined graph is determined according to a predetermined UI style.

[0448] Selectively, you can also drag the writing presentation table within the writing area. For example, you can set the initial position of the writing presentation table and then drag it down until the critical row table is fully displayed. Here, the user can drag the writing presentation table using a multi-finger touch method, for example, using a 2-4 finger touch method.

[0449] As shown in Figure 15C, the user can enter content into the writing presentation table, and the content entered by the user into the writing presentation table can be graphed and displayed in real time on a predetermined graph. The basic functions that can be set for writing include a hard pen at 6.5px, erase, one-touch clear, end, and insert command. The insert command is configured to insert the content of the graph area of ​​the current display interface into the previous display interface. When the user triggers the insert command, if the inserted graph is selected by default in the display interface, operations such as move, scale, delete, copy, and rotate can be performed on the graph.

[0450] The rules for writing bar graphs include, but are not limited to, the following: (1) At least one cell must be filled with valid content before the corresponding data is generated on the bar graph, and the first row, first column cell cannot be filled with content. (2) Write Title 1 in the first row, corresponding to the example; the example will not appear unless the user writes it. Here, the example in the bar graph displays the cylinder name. Different cylinders can be distinguished by different colors, content, etc. (3) Write Title 2 in the first column, corresponding to the title on the horizontal axis; Title 2 will not appear on the horizontal axis unless the user writes content in the first column. (4) Write data starting from the second row, second column; the data format only supports integers and decimals, and the corresponding graph will not be generated unless the user writes data. If non-data content is written in the corresponding cell, the corresponding cylinder will not be displayed. (5) The content written in each column is treated as one dataset, and the corresponding graph is generated from each dataset 1 second after writing stops. (6) The bar graph is drawn based on the cell containing the written content in the lower right corner of the left-hand writing display table. If there are any blank cells before that, a 0 is added. If Title 1 is blank, Title 1 is not displayed. If Title 2 is blank, Title 2 is not displayed.

[0451] Recognition of writing trajectory information begins 1 second after the pen stops writing, and at this time, the handwritten font that the user has written on the writing presentation table is retained. If the user adds or modifies data in the writing presentation table, the writing trajectory information in the writing presentation table is re-recognized 1 second after the user stops writing. However, if the written content in the writing presentation table after recognition does not meet the writing requirements, the bar graph will not be updated.

[0452] Selectively, users can also drag the writing display table after writing. When this happens, both the writing display table and the content written on it are dragged simultaneously. The position of the writing trajectory information within each cell of the writing display table is moved in accordance with the position of the writing display table, ensuring that the position of the content written by the user on the writing display table does not change.

[0453] Selectively, buttons for switching between different functions can be displayed in the display interface. For example, clicking a formula button switches the current display interface to the formula function display interface, and clicking a line graph button switches the current display interface to the line graph function display interface. Selectively, editing shapes (e.g., writing shapes, erase shapes, end shapes, undo shapes) can also be displayed in the display interface, and the current writing function can be switched using these editing shapes while writing in the user writing area. For example, clicking an erase shape after writing erases the corresponding content. If the user clicks an end shape, a dialog box will appear asking whether to confirm the end.

[0454] As shown in Figure 15D, the display interface has a full-screen button that allows for a full-screen preview of the displayed bar graph. For example, arrow buttons are provided on the dividing line between the writing area and the graph area. Triggering these arrow buttons switches to full-screen mode, and the bar graph display interface in full-screen mode is shown in Figure 15E. In full-screen mode, the graph within the original graph area is displayed in the center, and its size scale matches the predetermined UI design. In full-screen mode, the user can write in the graph area for markers or annotations. Basic writing functions include a red hard pen, a general-purpose eraser, one-touch clear, and palm erase. The current contents of the graph area can also be saved. For example, the graph displayed in the graph area with annotations can be saved. In full-screen mode, the user can exit full-screen mode by clicking the right-slide button and return to the previous mode, i.e., return to editing mode. When exiting full-screen mode, the contents of the graph area are retained, and even when switching to other graph types in intelligent drawing graph mode, the contents of the graph area in full-screen mode are retained.

[0455] As shown in Figures 16A to 16E, in a writing scene with table lines, taking a radargraph as an example, the different interfaces of the electronic whiteboard display during graph display are described below.

[0456] As shown in Figure 16A, the user triggers a graph selection command—radar map—in intelligent drawing graph mode, and in response to the graph selection command, it determines that the graph type corresponding to the graph selection command is a radar map. A guide template corresponding to the radar map is displayed in the display area, and the guide template includes a first partition and a second partition. Here, the first partition is an editing area and is configured to display a table template, and the second partition is a graph preview area and is configured to display a graph template. Here, the first column of the first row in the table template is grayed out to indicate that it is not writable, and the remaining columns are configured to present the writable content corresponding to each column. For example, the content of the first column is the title of each vertex, and the content of the first row is an example figure and is configured to display different polar coordinate names. Different example figures are indicated by different format title prompts. For example, set 1 is indicated by a solid title prompt and set 2 is indicated by a dotted line prompt. The remaining content is concrete data, showing the corresponding title and polar coordinates of the polar coordinate names. Furthermore, an edit button should be displayed on the display interface, or a close button within the display interface should be clicked to trigger an edit command for the user. It should be noted that if the user switches to another table after editing the radar map and then returns to the radar map, the edited content will be displayed as is, and access to the radar map's guide template will be lost. The guide template should only be displayed the first time the intelligent plotting graph function for known graph types is enabled.

[0457] As shown in Figure 16B, the user triggers an edit command in the display interface of the radar map guide template, and in response to the user's edit command in the radar map's intelligent drawing graph mode, the display area is divided into a writing area and a graph area. A writing suggestion table is displayed in the writing area, and a predetermined graph is displayed in the graph area, where the data content in the writing suggestion table is empty, the cell in the first row and first column is not editable, and the number of rows and columns in the writing suggestion table is constant. For example, a 3x12 table consisting of 4x13 table lines can be constructed. The writing suggestion table is composed of table lines, and the style of the predetermined graph is determined according to a predetermined UI style.

[0458] Selectively, you can also drag the writing presentation table within the writing area. For example, you can set the initial position of the writing presentation table and then drag it down until the critical row table is fully displayed. Here, the user can drag the writing presentation table using a multi-finger touch method, for example, using a 2-4 finger touch method.

[0459] As shown in Figures 16C-16D, the user can enter content into the writing presentation table, and the content entered by the user into the writing presentation table can be graphed and displayed in real time on a predetermined graph. The basic functions that can be set for writing include a hard pen at 6.5px, erase, one-touch clear, end, and insert command. The insert command is configured to insert the content of the graph area of ​​the current display interface into the previous display interface. When the user triggers the insert command, if the inserted graph is selected by default in the display interface, operations such as move, scale, delete, copy, and rotate can be performed on the graph.

[0460] The rules for writing on the radar map include, but are not limited to, the following: (1) At least one cell must be filled with legal content before generating data on the radar map, and the first row, first column cell cannot be filled. (2) Write Title 1 in the first row, corresponding to the example; the example will not appear unless the user writes it. Here, the example on the radar map indicates the polar coordinate name. (3) Write Title 2 in the first column, corresponding to the title of each vertex; Title 2 will not be found at the vertices on the radar map unless the user writes content in the first column. (4) Write data starting from the second row, second column; the data format only supports integers and decimals, and the corresponding graph will not be generated unless the user writes data. If non-data content is written in the corresponding cell, the corresponding polar coordinate will not be displayed. (5) The content written in each column is treated as one dataset, and after 1 second of writing, each polar coordinate, polar coordinate name, and radar map vertex title are generated based on each dataset. (6) The radar map is drawn based on the cell containing the written content in the lower right corner of the left-hand writing display table. If there are any blank cells before this cell, 0 is added. If Title 1 is blank, Title 1 is not displayed. If Title 2 is blank, Title 2 is not displayed. The radar map has at least 3 vertices, and the radar map is at least a polygon and at most a 12-sided polygon. If the amount of data the user writes in the writing display table is 3 lines or less, a polygonal radar graph is drawn using a 3-row table, and any empty cells in those 3 rows are added with 0.

[0461] Recognition of writing trajectory information begins 1 second after the pen stops writing, and at this time, the handwritten font that the user has written on the writing presentation table is retained. If the user adds or modifies data in the writing presentation table, the writing trajectory information in the writing presentation table is re-recognized 1 second after the user stops writing. However, if the written content in the writing presentation table after recognition does not meet the writing requirements, the radar map will not be updated.

[0462] Selectively, users can also drag the writing display table after writing. When this happens, both the writing display table and the content written on it are dragged simultaneously. The position of the writing trajectory information within each cell of the writing display table is moved in accordance with the position of the writing display table, ensuring that the position of the content written by the user on the writing display table does not change.

[0463] Selectively, buttons for switching between different functions can be displayed in the display interface. For example, clicking a formula button switches the current display interface to the formula function display interface, and clicking a line graph button switches the current display interface to the line graph function display interface. Selectively, editing shapes (e.g., writing shapes, erase shapes, end shapes, undo shapes, etc.) can also be displayed in the display interface, and the current writing function can be switched using these editing shapes while writing in the user writing area. For example, clicking an erase shape after writing erases the corresponding content. If the user clicks an end shape, a dialog box will appear asking whether to confirm the end.

[0464] Clicking the fullscreen button on the display interface allows for a fullscreen preview of the displayed radar map. For example, arrow buttons can be provided on the dividing line between the writing area and the graph area. Triggering these arrow buttons switches to fullscreen mode, and the radar map display interface in fullscreen mode is shown in Figure 16E. In fullscreen mode, the graph within the original graph area is displayed in the center, and its size scale matches the predetermined UI design. In fullscreen mode, the user can write in the graph area for markers or annotations. Basic writing functions include a red hard pen, a general-purpose eraser, one-touch clear, and palm erase. The current contents of the graph area can also be saved. For example, a graph with annotations displayed in the graph area can be saved. In fullscreen mode, clicking the right-slide button exits fullscreen mode and returns to the previous mode, i.e., back to editing mode. When exiting fullscreen mode, the contents of the graph area are retained, and even when switching to other graph types in intelligent drawing graph mode, the contents of the graph area in fullscreen mode are retained.

[0465] As shown in Figures 17A to 17E, in a writing scene with table lines, using a pie chart as an example, the different interfaces of the electronic whiteboard display during graph display will be explained as follows.

[0466] As shown in Figure 17A, the user triggers a graph selection command—a pie chart—in intelligent drawing graph mode, and in response to the graph selection command, it is determined that the graph type corresponding to the graph selection command is a pie chart. A guide template corresponding to the pie chart is displayed in the display area, and the guide template includes a first partition and a second partition. Here, the first partition is the editing area and is configured to display a table template, and the second partition is the graph preview area and is configured to display a graph template. Here, the first column in the table template is for writing the title (i.e., example figure) of each partition, and the second column is for writing the data corresponding to each title. Different examples figures are represented by different filling sectors. The data corresponding to different titles indicates the occupancy rate of those sectors. In addition, an edit button is displayed in the display interface, or a close button is clicked in the display interface to trigger an edit command for the user. Note that if the user switches to another table after editing the pie chart and then returns to the pie chart, the edited content is displayed as is, and the guide template for the pie chart is not accessed. The guide template is displayed only when the intelligent drawing graph function for a known graph type is turned on for the first time. Selectively, an arrow button is displayed on the boundary line between the first and second partitions within the guide template, allowing the user to trigger the arrow button to display the graph template within the second partition in full screen.

[0467] As shown in Figure 17B, the user triggers an edit command in the display interface of the pie chart guide template, and in response to the user's edit command in the intelligent drawing graph mode of the pie chart, the display area is divided into a writing area and a graph area. A writing suggestion table is displayed in the writing area, and a predetermined graph is displayed in the graph area, where the data content in the writing suggestion table is empty and the number of rows and columns in the writing suggestion table is constant. For example, a writing suggestion table with 2*13 elements is composed of table lines, and the style of the predetermined graph is determined according to a predetermined UI style.

[0468] Selectively, you can also drag the writing presentation table within the writing area. For example, you can set the initial position of the writing presentation table and then drag it down until the critical row table is fully displayed. Here, the user can drag the writing presentation table using a multi-finger touch method, for example, using a 2-4 finger touch method.

[0469] As shown in Figure 17C, the user can enter content into the writing presentation table, and the content entered by the user into the writing presentation table can be graphed and displayed in real time on a predetermined graph. The basic functions that can be set for writing include a hard pen at 6.5px, erase, one-touch clear, end, and insert command. The insert command is configured to insert the content of the graph area of ​​the current display interface into the previous display interface. When the user triggers the insert command, if the inserted graph is selected by default in the display interface, operations such as move, scale, delete, copy, and rotate can be performed on the graph.

[0470] The rules for writing on the pie chart include, but are not limited to, the following: (1) Write legal content in at least one cell before generating data on the pie chart. (2) Write a title (containing at least one of letters, English text, numbers, or symbols) in the first column, corresponding to an example; the example will not appear unless the user writes one. Here, the example in the pie chart shows the title of each partition. (3) Write data in the second column, corresponding to the occupancy rate of each partition; the data format supports integers and decimals; the corresponding graph will not be generated unless the user writes data. If non-data content is written in the corresponding cell, the corresponding partition will not be displayed. (4) The pie chart partitions are arranged clockwise from the 12 o'clock position in the order of the rows in the writing presentation table. (5) The content written in each column is treated as one dataset, and after 1 second of writing, the title and occupancy rate of each partition are generated based on each dataset. (6) The pie chart is drawn based on the cell containing the written content in the lower right corner of the left-hand writing display table. If there are any blank cells before that, a 0 is added. If the title is blank, the title of each partition is not displayed.

[0471] After the pen stops writing for 1 second, the recognition of the writing trajectory information begins, and at this time, the handwritten font that the user has written on the writing presentation table is retained. As shown in Figure 17D, if the user adds or modifies the data content in the writing presentation table, the writing trajectory information in the writing presentation table is re-recognized after the user stops writing for 1 second, and at the same time, the displayed pie chart is updated. However, if the written content in the writing presentation table after recognition does not meet the writing requirements, the pie chart will not be updated.

[0472] Selectively, users can also drag the writing display table after writing. When this happens, both the writing display table and the content written on it are dragged simultaneously. The position of the writing trajectory information within each cell of the writing display table is moved in accordance with the position of the writing display table, ensuring that the position of the content written by the user on the writing display table does not change.

[0473] Selectively, buttons for switching between different functions can be displayed in the display interface. For example, clicking a formula button switches the current display interface to the formula function display interface, and clicking a line graph button switches the current display interface to the line graph function display interface. Selectively, editing shapes (e.g., writing shapes, erase shapes, end shapes, undo shapes, etc.) can also be displayed in the display interface. While writing in the user writing area, the current writing function can be switched using the editing shapes. For example, clicking an erase shape after writing erases the corresponding content. If the user clicks an end shape, a dialog box will appear asking whether to confirm the end.

[0474] Clicking the fullscreen button on the display interface allows you to view a fullscreen preview of the pie chart. For example, arrow buttons can be placed on the dividing line between the writing area and the graph area. Triggering these arrow buttons switches to fullscreen mode, and the pie chart display interface in fullscreen mode looks like Figure 17E. In fullscreen mode, the graph within the original graph area is displayed in the center, and its size scale matches the predetermined UI design. In fullscreen mode, the user can write in the graph area for markers or annotations. Basic writing functions include a red hard pen, a general-purpose eraser, one-touch clear, and palm erase. The contents of the current graph area can also be saved. For example, the graph displayed in the graph area with annotations can be saved. In fullscreen mode, you can exit fullscreen mode by clicking the right-slide button and return to the previous mode, i.e., return to editing mode. When exiting fullscreen mode, the contents of the graph area are retained, and even when switching to other graph types in intelligent drawing graph mode, the contents of the graph area in fullscreen mode are retained.

[0475] As shown in Figures 18A to 18D, in a writing scene with table lines, the different interfaces of the electronic whiteboard display during graph display will be explained below, using a line graph as an example.

[0476] As shown in Figure 18A, the user triggers a graph selection command in intelligent drawing graph mode, resulting in a line graph. In response to the graph selection command, the system determines that the graph type corresponding to the command is a line graph. A guide template corresponding to the line graph is displayed in the display area, and the guide template includes a first partition and a second partition. Here, the first partition is the editing area and is configured to display a table template, and the second partition is the graph preview area and is configured to display a graph template. Here, the first row in the table template is for writing the titles (i.e., example figures) for each coordinate axis, and the remaining rows are for writing the data corresponding to each title. Different example figures are represented by titles for different coordinate axes. The data corresponding to different titles is for displaying coordinate points. In addition, an edit button is displayed in the display interface, or a close button is clicked in the display interface to trigger an edit command for the user. Note that if the user switches to another table after editing the line graph and then returns to the line graph, the edited content is displayed as is, and the guide template for the line graph is not accessed. The guide template is displayed only the first time the intelligent plotting graph function for known graph types is turned on. Selectively, an arrow button is displayed on the boundary line between the first and second partitions within the guide template, allowing the user to trigger the arrow button to display the graph template in the second partition in full screen.

[0477] As shown in Figure 18B, the user triggers an editing command in the display interface of the line graph guide template, and in response to the user's editing command in the intelligent drawing graph mode of the line graph, the display area is divided into a writing area and a graph area. A writing suggestion table is displayed in the writing area, and a predetermined graph is displayed in the graph area, where the data content in the writing suggestion table is empty and the number of rows and columns in the writing suggestion table is constant. For example, a writing suggestion table with 2*13 elements is composed of table lines, and the style of the predetermined graph is determined according to a predetermined UI style.

[0478] Selectively, you can also drag the writing presentation table within the writing area. For example, you can set the initial position of the writing presentation table and then drag it down until the critical row table is fully displayed. Here, the user can drag the writing presentation table using a multi-finger touch method, for example, using a 2-4 finger touch method.

[0479] As shown in Figure 18C, the user can enter content into the writing presentation table, and the content entered by the user into the writing presentation table can be graphed and displayed in real time on a predetermined graph. The basic functions that can be set for writing include a hard pen at 6.5px, erase, one-touch clear, end, and insert command. The insert command is configured to insert the content of the graph area of ​​the current display interface into the previous display interface. When the user triggers the insert command, if the inserted graph is selected by default in the display interface, operations such as move, scale, delete, copy, and rotate can be performed on the graph.

[0480] The rules for writing on line graphs include, but are not limited to, the following: (1) Write at least one valid content in one cell before generating data on the line graph. (2) The first row is for writing a title (containing at least one of letters, English text, numbers, or symbols) and corresponds to an example; if the user does not write an example, the example will not appear. Here, the example in the line graph displays the title of each coordinate axis. (3) The remaining rows are for writing data and corresponding to each coordinate point; the data format supports integers and decimals, and if the user does not write data, the corresponding graph will not be generated. If non-data content is written in a corresponding cell, the corresponding coordinate point will not be displayed. (4) The content written in each column is treated as one dataset, and 1 second after the pen stops writing, each coordinate point and the title of the coordinate axis are generated based on each dataset. (5) The line graph is drawn based on the cell with writing content in the lower right corner of the left-hand writing display table; if there are blank cells before that, 0 is added, and if the title is blank, the coordinate axis title is not displayed.

[0481] After the pen stops writing for 1 second, the recognition of the writing trajectory information begins, and at this time, the handwritten font that the user has written on the writing presentation table is retained. If the user adds or modifies data in the writing presentation table, the writing trajectory information in the writing presentation table is recognized again after the user stops writing for 1 second, and the displayed line graph is updated. However, if the written content in the writing presentation table after recognition does not meet the writing requirements, the line graph will not be updated.

[0482] Selectively, users can also drag the writing display table after writing. When this happens, both the writing display table and the content written on it are dragged simultaneously. The position of the writing trajectory information within each cell of the writing display table is moved in accordance with the position of the writing display table, ensuring that the position of the content written by the user on the writing display table does not change.

[0483] Selectively, buttons for switching between different functions can be displayed in the display interface. For example, the user can click a formula button to switch the current display interface to the formula function display interface, or click a bar graph button to switch the current display interface to the bar graph function display interface. Selectively, editing shapes (e.g., writing shapes, erase shapes, end shapes, undo shapes, etc.) can also be displayed in the display interface. The current writing function can be switched using editing shapes while writing in the user writing area. For example, the user can erase the corresponding content by clicking an erase shape after writing. If the user clicks an end shape, a dialog box will appear asking whether to confirm the end.

[0484] Clicking the fullscreen button on the display interface allows you to view a fullscreen preview of the displayed line graph. For example, arrow buttons can be provided on the dividing line between the writing area and the graph area. Triggering these arrow buttons switches to fullscreen mode, and the line graph display interface in fullscreen mode is as shown in Figure 18D. In fullscreen mode, the graph within the original graph area is displayed in the center, and its size scale matches the predetermined UI design. In fullscreen mode, the user can write in the graph area for markers or annotations. Basic writing functions include a red hard pen, a general-purpose eraser, one-touch clear, and palm erase. The contents of the current graph area can also be saved. For example, a graph with annotations displayed in the graph area can be saved. In fullscreen mode, you can exit fullscreen mode by clicking the right-slide button and return to the previous mode, i.e., return to editing mode. When exiting fullscreen mode, the contents of the graph area are retained, and even when switching to other graph types in intelligent drawing graph mode, the contents of the graph area in fullscreen mode are retained.

[0485] Selectively, if the content written by the user in the writing table exceeds a predetermined number of lines, the content within the predetermined number of lines in the writing table is recognized, and the remaining lines are discarded. Alternatively, if the content written by the user in the writing table exceeds a predetermined number of columns, the content within the predetermined number of columns in the writing table is recognized, and the remaining columns are discarded.

[0486] Selectively, if the number of bits in the integer part of a recognized digit in the handwriting presentation table is greater than the first bit threshold, it is represented using the scientific counting method. If the number of digits in the decimal part of the recognized digit exceeds the second digit threshold, the number of digits after the decimal point is retained up to a predetermined number of digits.

[0487] Selectively, if the pixel value of an individual trajectory point in the handwriting trajectory information is smaller than the pixel threshold, that trajectory point is removed. The cell to which the handwriting trajectory information belongs is based on the starting point of the coordinates recognized as the standard font.

[0488] In some embodiments, as shown in Figure 19, this embodiment provides a method for displaying a graph in a writing scene that displays a title prompt. The specific steps for implementing this method are as follows.

[0489] Step 1900: In response to a user editing command in smart graph mode for a known graph type, the display area is divided into a writing area and a graph area, and a title prompt is displayed in the writing area.

[0490] Step 1901: In the first area corresponding to the title prompt, obtain first writing trajectory information representing the title, and in the second area corresponding to the title prompt, obtain second writing trajectory information representing the content.

[0491] Here, the first region and the second region are separated by the title prompt.

[0492] Step 1902: In intelligent drawing graph mode for known graph types, when the user stops writing and waits for a predetermined time, the system recognizes the writing trajectory information within the writing area, obtains data recognition results, and draws and displays a graph corresponding to the graph type in the graph area based on the data recognition results.

[0493] In some embodiments, the data recognition result includes the title of each coordinate axis and a plurality of cells, and the data recognition result includes the position of each cell and the data content within each cell.

[0494] Specifically, the processor determines the data recognition result as follows: The handwriting trajectory information of the first region is recognized, and the title of each coordinate axis is obtained. The handwriting trajectory information of the second region is divided into cells, the handwriting trajectory information in each cell is recognized, and the data content of each cell in the second region is obtained. Based on the title, the data content of each cell in the second region, and the positional relationship between the title and each cell, the data recognition result is determined.

[0495] As shown in Figure 20, in a writing scene where a title prompt is displayed, we will use a line graph as an example to explain the different interfaces of the electronic whiteboard display while a graph is being shown as follows.

[0496] As shown in Figure 20, two horizontal line segments are displayed in the writing area of ​​the display area. The user writes the title above the horizontal line segment and the content below the horizontal line segment. When the user stops writing for 1 second, the writing trajectory information written by the user in the writing area is recognized, and the content above the recognized horizontal line segment is used as the title on the coordinate axis, and the data of each line below the recognized horizontal line segment is used as coordinate points to draw a line graph. Selectively, the number of characters above the horizontal line is less than a quantity threshold, and the number of lines below the horizontal line is less than a line count threshold.

[0497] The display area may show buttons to switch between different graph types, such as mathematical formulas, or buttons for basic writing functions, such as displaying a writing shape, erasing a shape, undoing a shape, or redoing a shape. It may also allow full-screen display of a line graph via a line graph displayed by multi-finger touch. To operate the functions corresponding to the generated graph, at least one of the following buttons may be displayed: complete, save, or exit. Selectively, with the undo / redo buttons, one-step forward / backward movement does not update the coordinate axes in real time with the data changed by the undo / redo. In this embodiment, clicking the complete button inserts the current coordinate axis content into a whiteboard canvas (an electronic whiteboard that is not in graph mode, or the previous display interface) at a scale below the large window, and it cannot be re-edited. Up to six graphs can be inserted into one canvas (unlimited copying is possible, but not copies). If there is nothing in the graph area, the complete button is grayed out and the whiteboard canvas is not inserted. If there is no data in the graph area except for the coordinate axes, the whiteboard canvas can also be inserted. In this embodiment, when the user clicks a mathematical formula shape in the current display interface to switch to mathematical formula mode, the contents of the current display area are retained.

[0498] Here, the writing rules are: (1) write two columns of data in the left writing area, with the data format for each column being: title (content above the horizontal line) + data (number below the horizontal line), and start identifying the pen after writing has stopped. The data in the first column is the x coordinate, and the data in the second column is the y coordinate. Here, the horizontal axis scale is equally spaced to match the user's writing data, and the vertical axis scale increases proportionally to the data.

[0499] If the user writes table data only in the writing area in intelligent drawing graph mode, they can generate coordinate axes by hand-drawing horizontal and vertical lines in the graph area (without arrow recognition algorithm; the user can form coordinate axes with two hand-drawn strokes). The recognized data from the two columns on the left is then incorporated into the coordinate axes, and a line graph is generated, with the content above the horizontal line becoming the title of the horizontal and vertical axes.

[0500] Selectively, the contents of the writing area can be erased with the eraser. The scribe area is also an erasable area. After the user erases or modifies the contents, the graph can be regenerated to update the original graph. A one-touch clear button allows for the complete clearing of all contents in the graph area, including the coordinate axes.

[0501] In this embodiment, if the data in the two left rows is different, the blank space is not displayed when the coordinate axis is first generated. An appropriate predetermined pitch is set for every two columns depending on the writing style, and if the distance between two numbers exceeds the predetermined pitch, it is determined to be two columns of data. It is also possible to generate coordinate axes with fixed positions and sizes. The coordinate points are updated in real time according to changes in the writing content on the left. The line graph is generated according to the user's writing order (left to right) in the writing area. Selectively, dragging a single finger within the graph area changes the window size of the graph area and scales the graph displayed in the graph area uniformly.

[0502] In some embodiments, as shown in Figure 21, the policy for matrix partitioning the table-less content written by the user in the writing scene without table lines and the writing scene with a title prompt is as follows:

[0503] Step 2100: The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke.

[0504] Step 2101: Based on the stroke length, the trajectory points in the stroke are resampled and the number of trajectory points in the stroke is increased.

[0505] Here, resampling increases the number of trajectory points during a stroke according to the stroke length, and aims to prevent the problem that occurs when using projection methods, where the number of trajectory points is too sparse to determine the separation position.

[0506] Step 2102: Project the resampled trajectory points onto the X-axis to obtain the X-axis projection histogram, then perform a scaled-down mapping on the X-axis projection histogram to obtain the first projection histogram.

[0507] Alternatively, the projection histogram of the x-axis may be selectively scaled down, then normalized to obtain a first projection histogram after normalization, and the following steps may be performed on the first projection histogram after normalization.

[0508] Step 2103: Determine the column separator lines based on the valley positions in the first projected histogram.

[0509] Step 2104: Project the handwriting trajectory information contained in any column onto the Y-axis to obtain a Y-axis projection histogram, then perform a scaled-down mapping of the Y-axis projection histogram to obtain a second projection histogram.

[0510] Alternatively, the Y-axis projection histogram may be selectively scaled down, then normalized to obtain a second projection histogram after normalization, and the following steps may be performed on the second projection histogram after normalization.

[0511] Step 2105: Determine the row separator line for either column based on the valley position in the second projected histogram of either column.

[0512] Step 2106: Determine each cell based on the column separator and row separator.

[0513] Step 2107: The stroke is divided into cells based on the positional relationship between the center of gravity of the stroke and each cell.

[0514] Step 2108: Recognize the handwriting trajectory information in each cell and retrieve the data content in each cell.

[0515] Step 2109: Determine the data recognition result based on the position of each cell and the data content within each cell.

[0516] In this embodiment, an example of matrix partitioning is provided, using the case where the user's handwritten content is in a 5x2 grid.

[0517] As shown in Figure 21A, the resampled trajectory points are projected onto the X-axis to obtain an X-axis projection histogram. As shown in Figure 21B, the X-axis projection histogram is scaled down to obtain a first projection histogram, and the intermediate X-coordinate values ​​of the X-coordinate range (valley width) where the Y-coordinate value is 0 in the first projection histogram are used as the positions of the column separators. The handwriting trajectory information is divided into two columns by the determined column separators.

[0518] As shown in Figure 21C, this is a Y-axis projected histogram obtained by projecting the writing trajectory information contained in the first column onto the Y axis. As shown in Figure 21D, this is a second projected histogram of the first column obtained by scaling down the Y-axis projected histogram. The intermediate Y coordinate values ​​of the Y coordinate range (valley width) where the X coordinate value is 0 in the second projected histogram are used as the positions of the row separators in the first column, and the first column is divided into multiple rows.

[0519] Similarly, as shown in Figure 21E, this is a Y-axis projected histogram obtained by projecting the writing trajectory information contained in the second column onto the Y axis, and as shown in Figure 21F, this is a second projected histogram of the second column obtained by scaling down the Y-axis projected histogram. The intermediate Y coordinate values ​​of the Y coordinate range (valley width) where the X coordinate value is 0 in the second projected histogram are used as the positions of the row separators in the second column, and the second column is divided into multiple rows.

[0520] Furthermore, in the cell determination method of this embodiment, by first dividing the columns and then dividing the rows for each column, the problem of the user's handwriting being illegible and the contents of different columns not being aligned in each row can be solved. Even if the contents belonging to different columns in each row of the user are not on the same horizontal line, the method of this embodiment can accurately divide the data contents of each cell and accurately realize the division of rows and columns. By using this embodiment in a vertically written table, the problem of handwritten columns in a vertically written table being misaligned and displayed in the wrong row, affecting recognition, is solved. As shown in Figures 21G and 21H, this embodiment provides schematic diagrams of several divisions.

[0521] In some embodiments, as shown in Figure 22, this embodiment describes the policy for matrix partitioning the table-less content written by the user in a writing scene without table lines and in a writing scene with a title prompt as follows:

[0522] Step 2200: The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke.

[0523] Step 2201: Based on the stroke length, resample the trajectory points during the stroke and increase the number of trajectory points during the stroke.

[0524] Step 2202: Project the resampled trajectory points onto the Y-axis to obtain a Y-axis projected histogram, then scale down the Y-axis projected histogram to obtain a second projected histogram.

[0525] Alternatively, the Y-axis projection histogram may be selectively scaled down, then normalized to obtain a second projection histogram after normalization, and the following steps may be performed on the second projection histogram after normalization.

[0526] Step 2203: Determine the row separator lines based on the valley positions in the second projection histogram.

[0527] Step 2204: Project the handwriting trajectory information contained in any row onto the X-axis to obtain the X-axis projection histogram, and then perform a reduction mapping of the X-axis projection histogram to obtain the first projection histogram.

[0528] Alternatively, the projection histogram of the x-axis may be selectively scaled down, then normalized to obtain a first projection histogram after normalization, and the following steps may be performed on the first projection histogram after normalization.

[0529] Step 2205: Determine the column separator line for any row based on the valley position in the first projected histogram of any row.

[0530] Step 2206: Determine each cell based on the column separator and row separator.

[0531] Step 2207: The stroke is divided into cells based on the positional relationship between the center of gravity of the stroke and each cell.

[0532] Step 2208: Recognize the handwriting trajectory information in each cell and retrieve the data content in each cell.

[0533] Step 2209: Determine the data recognition result based on the position of each cell and the data content within each cell.

[0534] In the cell determination method of this embodiment, the problem of blank rows appearing in the table due to unsuitable user writing can be solved by first determining row delimiters and then determining column delimiters for each row. However, in the method according to this embodiment, blank rows can be identified and the data content within each cell can be accurately delimited, thereby accurately achieving row and column division.

[0535] In some embodiments, as shown in Figure 23, the policy for matrix partitioning the table-less content written by the user in the writing scene without table lines and the writing scene with a title prompt is as follows:

[0536] Step 2300: The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke.

[0537] Step 2301: Based on the stroke length, resample the trajectory points during the stroke and increase the number of trajectory points during the stroke.

[0538] Step 2302: The resampled trajectory points are simultaneously projected onto the X and Y axes to obtain the X-axis projection histogram and the Y-axis projection histogram. The X-axis projection histogram and the Y-axis projection histogram are then scaled down to obtain the first projection histogram and the second projection histogram.

[0539] Selectively, the Y-axis projection histogram is scaled down and then normalized to obtain a second projection histogram after normalization. Alternatively, the X-axis projection histogram is scaled down and then normalized to obtain a first projection histogram after normalization, and the following procedure can be performed on both the first and second projection histograms after normalization.

[0540] Step 2303: Determine the column separators based on the valley positions in the first projected histogram, and determine the row separators based on the valley positions in the second projected histogram.

[0541] Step 2304: Determine each cell based on the column separator and row separator.

[0542] Step 2305: The stroke is divided into cells based on the positional relationship between the center of gravity of the stroke and each cell.

[0543] Step 2306: Recognize the handwriting trajectory information in each cell and retrieve the data content in each cell.

[0544] Step 2307: Determine the data recognition result based on the position of each cell and the data content within each cell.

[0545] Furthermore, the cell determination method in this embodiment solves the problem of empty cells appearing in the table due to unsuitable user writing by simultaneously achieving matrix division. According to the method in this embodiment, empty cells, i.e., cells in which no writing trajectory information has been written, can be identified, and the data content within each cell can be accurately separated by cells without writing trajectory information, thereby accurately achieving row and column division.

[0546] In some embodiments, this embodiment provides a text recognition model comprising multiple recurrent neural network (RNN) network layers. At least two outputs are concatenated to the inputs of any of the RNN network layers except the first two.

[0547] The at least two outputs include outputs from any two RNN network layers prior to any of the aforementioned RNN network layers.

[0548] In some embodiments, the text recognition model further includes a fully connected layer to which at least three outputs are concatenated. These at least three outputs include outputs from any three RNN network layers preceding the fully connected layer. The outputs from any of the preceding RNN network layers, except for the last one, are used as inputs to at least two subsequent RNN network layers.

[0549] Selectively, the RNN network layer may be a BiLSTM network layer or a GRU (Gate Recurrent Unit) network layer, and can be selected according to the actual needs of the user, but is not particularly limited here.

[0550] Taking BiLSTM as an example, this embodiment also provides a text recognition model comprising multiple BiLSTM network layers. At least two outputs are concatenated to the input of any BiLSTM network layer except the first two, and these at least two outputs include outputs from any two BiLSTM network layers preceding any of the aforementioned BiLSTM network layers.

[0551] The text recognition model provided by this embodiment further includes a fully connected layer. At least three outputs are concatenated to the fully connected layer. The at least three outputs include outputs from any three BiLSTM network layers preceding the fully connected layer.

[0552] In some embodiments, the output from any BiLSTM network layer prior to the last BiLSTM network layer is used as input to at least two subsequent BiLSTM network layers.

[0553] In this embodiment, the output of each BiLSTM network layer is a one-dimensional matrix. If a BiLSTM network layer has two inputs, the BiLSTM network layer actually adds the one-dimensional matrices of the outputs from the first two BiLSTM network layers, i.e., adds the values ​​at the corresponding positions to obtain a new one-dimensional matrix. This new one-dimensional matrix is ​​then processed using the BiLSTM network layer.

[0554] In some embodiments, the specific connectivity of the text recognition model provided by this embodiment is shown in Figure 24, where the text recognition model includes four BiLSTM network layers and a fully connected layer. The first BiLSTM network layer includes a first input and a first output. The second BiLSTM network layer includes a second input and a second output. The third BiLSTM network layer includes a third input and a third output. The fourth BiLSTM network layer includes a fourth input and a fourth output.

[0555] The first input is configured to receive the handwriting trajectory information. The second input connects the first output. The third input connects the first output and the second output. The fourth input connects the first output, the second output, and the third output. The fourth output connects the inputs of the fully connected layer. The output from the fully connected layer is configured to output data content corresponding to the handwriting trajectory information.

[0556] Specifically, the input to the first layer BiLSTM is the handwriting trajectory information, and the output is out1. The input to the second layer BiLSTM is out1, and the output is out2. The input to the third layer BiLSTM is out1 + out2, and the output is out. The input to the fourth layer BiLSTM is out1 + out2 + out3, and the output is out4. The input to the fully connected layer is out2 + out3 + out4, and the output is the data content corresponding to the handwriting trajectory information report.

[0557] In this embodiment, each BiLSTM network layer includes 64 implicit layers, and the fully connected layer includes 3816 neurons, where 3816 are the final classification clusters, including 3755 first-grade Chinese characters, 26 English letters, 10 Arabic numerals, and 25 common punctuation marks.

[0558] Here, any of the BiLSTM network layers have had their network parameters reduced using a Dropout layer during model training. If a Dropout layer with a probability of 0.5 is used after each BiLSTM during model training, then in the case of text recognition model data, 50% of the parameters being transmitted are randomly set to 0. That is, by randomly removing partial connections, model overfitting is reduced and model convergence is promoted.

[0559] The text recognition model in this embodiment reduces the number of implicit layers compared to conventional text recognition models, thereby reducing the size of the model and compressing it by several times. Network degradation is also improved by changing the connectivity relationships, and the accuracy of the text recognition model is guaranteed. In this embodiment, the model size is compressed from 5.1MB to 1.3MB, making it easier to deploy to terminal devices (the original model uses 128 implicit layers in each of the four bilstm network layers, but the text recognition model in this embodiment changes the number of implicit layers per bilstm from 128 to 64, resulting in a reduction of approximately four times).

[0560] In some embodiments, this embodiment also provides an automatic grid projection method that automatically divides the content written by the user into individual cells in order to automatically recognize the writing trajectory information within each cell using a trajectory recognition algorithm. The core idea of ​​the automatic grid projection method in this embodiment is to determine each cell by obtaining row and column dividing lines by projecting the writing trajectory information onto the X and Y axes. The specific implementation flow of this method is as follows.

[0561] The handwriting trajectory information of the display area on the display screen is acquired. Based on the first projection histogram on the first coordinate axis of the handwriting trajectory information and the second projection histogram on the second coordinate axis of the handwriting trajectory information, each row separator and each column separator is determined. Each cell is determined based on the row separators and column separators.

[0562] Here, the first coordinate axis and the second coordinate axis are perpendicular to each other.

[0563] In some embodiments, the handwriting trajectory information is divided into multiple rows by row separators, the handwriting trajectory information is divided into multiple columns based on column separators, the table borders are determined based on the minimum and maximum values ​​of the first and second coordinate axes of the trajectory points in all the handwriting trajectory information, and the borders and each cell are determined based on the column separators and row separators.

[0564] In some embodiments, the row separators and column separators are determined as follows: Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, the column separator lines are determined, and based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information contained in any of the columns, the row separator lines for any of the columns are determined, or, Based on the valley position in the second projection histogram on the second coordinate axis of the handwriting trajectory information, a row separator line is determined, and based on the valley position in the first projection histogram on the first coordinate axis of the handwriting trajectory information contained in any row, a column separator line for any row is determined, or Based on the valley positions in the first projected histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined, and based on the valley positions in the second projected histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined.

[0565] In some embodiments, the column separators are determined as follows:

[0566] The column separators are determined based on the valley locations in the first projection histogram where the valley width is greater than the threshold.

[0567] In some embodiments, the line separators are determined as follows:

[0568] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0569] Furthermore, in the cell determination method of this embodiment, by first dividing the columns and then dividing the rows for each column, the problem of the user's writing being illegible and the contents of different columns in each row not being aligned can be solved. The problem of handwritten columns in a vertically written table being misaligned and affecting recognition can be solved. By first separating the rows and then dividing the columns for each row, the problem of the user's writing being illegible and the occurrence of empty rows can be solved. By simultaneously separating the rows and columns, the problem of the user's writing being illegible and the appearance of empty cells can be solved.

[0570] Example 2, based on the same inventive concept, further provides an embodiment of the present invention that provides a method for displaying graphs. This method is a method for using an electronic whiteboard according to an embodiment of the present invention, and since the solution principle is the same as that of the electronic whiteboard, the implementation of the electronic whiteboard can be referenced, and the explanation of overlapping parts will be omitted.

[0571] As shown in Figure 25, the implementation of this method is as follows.

[0572] Step 2500: The handwriting trajectory information in the display area of ​​the display screen is recognized, and the data recognition result is obtained.

[0573] Step 2501: In response to a graph drawing command from the user, determine the graph type corresponding to the graph drawing command.

[0574] Step 2502: In response to a user's graph drawing command, the system draws a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and displays the drawn graph in the display area.

[0575] As a selective embodiment, before recognizing the handwriting trajectory information in the display area of ​​the display screen and obtaining the data recognition result, further: A writing presentation table is displayed in the aforementioned display area, and the writing presentation table is presented in such a way that the user writes in each cell of the writing presentation table. The system receives the handwriting trajectory information entered by the user on the handwriting display table.

[0576] As an optional embodiment, displaying a writing presentation table in the display area is, specifically, Based on the position of the writing trajectory information first written by the user in the display area, the reference position of the writing display table is determined. A writing display table with a fixed number of rows and columns is displayed in the display area according to the reference position of the writing display table, and the writing trajectory information written by the user for the first time is located within a cell of the writing display table.

[0577] As a selective embodiment, after displaying the writing presentation table, further, If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of rows and / or columns in the writing presentation table will be increased according to the position of the writing trajectory information that the user continues to write, or If the writing trajectory information that the user continues to write is in a critical row of the writing presentation table, the number of rows in the writing presentation table is increased, and / or if the writing trajectory information that the user continues to write is in a critical column of the writing presentation table, the number of columns in the writing presentation table is increased.

[0578] In an optional embodiment, the number of rows and columns in the handwriting display table is fixed or determined based on parameters entered by the user.

[0579] As a selective embodiment, the data recognition result is determined by one of the following methods.

[0580] Method 1: Based on the positional relationship between each trajectory point in the handwriting trajectory information and the cell in the handwriting presentation table, the handwriting trajectory information contained in each cell is determined, the handwriting trajectory information contained in each cell is recognized, and the data content of each cell is acquired.

[0581] Method 2: The method recognizes the handwriting trajectory information, obtains the content written by the user and its position in the display area of ​​the content, and determines the data content of each cell based on the position of the content and the position of the cell in the handwriting presentation table.

[0582] In a selective embodiment, the data recognition result includes a plurality of cells, and the data recognition result includes the position of each cell and the data content within each cell.

[0583] As a selective embodiment, the data recognition result is determined as follows.

[0584] Each cell is determined, and the handwriting trajectory information is divided into each cell. Here, each cell is used to correspond to each cell in the table, and the handwriting trajectory information contained in each cell is used to correspond to the data content in each cell. The handwriting trajectory information in each cell is recognized, and the data content in each cell is retrieved. The data recognition result is determined based on the position of each cell and the data content within each cell.

[0585] In a selective embodiment, the writing trajectory is divided into cells as follows:

[0586] The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke. Based on the positional relationship between the center of gravity of the stroke and each cell, the stroke is divided into individual cells.

[0587] In a selective embodiment, the writing trajectory is divided into cells as follows:

[0588] The writing order and coordinates of each trajectory point in the aforementioned writing trajectory information are obtained. Based on the writing order and coordinates of each trajectory point, the degree of similarity between each trajectory point is determined. The trajectory points whose similarity satisfies a predetermined threshold are divided into individual cells.

[0589] As an optional embodiment, each cell is determined as follows:

[0590] Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, the column separator lines are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information contained in any of the columns, the row separator lines for any of the columns are determined; and each cell is determined based on the column separator lines and row separator lines, or, Based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information contained in any row, column separators for any row are determined; and each cell is determined based on the row separators and column separators, or, Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; and each cell is determined based on the row separators and column separators.

[0591] As an optional embodiment, each cell is determined as follows:

[0592] Each cell is determined using a clustering algorithm.

[0593] As an optional embodiment, the column separators are determined as follows:

[0594] Based on the valley locations where the valley width in the first projection histogram is greater than a threshold, the column separator lines are determined, and / or, The line separators are determined as follows:

[0595] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0596] In a selective embodiment, the writing trajectory information is obtained after performing a resampling operation on the trajectory points in the writing trajectory information. The resampling is configured to increase the number of trajectory points in the corresponding stroke according to the stroke length in the writing trajectory information.

[0597] In a selective embodiment, the first projected histogram is a projected histogram obtained by projecting the writing trajectory information onto a first coordinate axis, and is obtained by reduction mapping.

[0598] The second projection histogram is obtained by projecting the writing trajectory information onto a second coordinate axis, and is obtained by reduction mapping.

[0599] As a selective embodiment, before recognizing the handwriting trajectory information in the display area of ​​the display screen and obtaining the data recognition result, The display area is configured to display a title prompt, and the title prompt is configured to indicate the writing position of the user table title. In the first region corresponding to the title prompt, first writing trajectory information is acquired, and in the second region corresponding to the title prompt, second writing trajectory information is acquired.

[0600] As a selective embodiment, the data recognition results are determined as follows: The handwriting trajectory information of the first region is recognized, and the title of each coordinate axis is obtained.

[0601] The handwriting trajectory information in the second region is divided into individual cells, the handwriting trajectory information in each cell is recognized, and the data content in each cell of the second region is obtained.

[0602] The data recognition result is determined based on the aforementioned title, the data content within each cell of the second region, and the positional relationship between the title and each cell.

[0603] In an optional embodiment, the graph drawing command includes a graph selection command, which determines the graph type as follows:

[0604] In response to a graph selection command from the user in intelligent drawing graph mode, the system determines the graph type corresponding to the graph selection command.

[0605] In an optional embodiment, the graph drawing command includes a first touch gesture and determines the graph type as follows:

[0606] In response to a first touch gesture by the user in intelligent drawing graph mode, the system recognizes the first shape generated by the first touch gesture and determines the graph type based on the first shape.

[0607] In an optional embodiment, the graph drawing command includes at least one of the following: an arrow drawing command, a circle drawing command, a polygon drawing command, a cylindrical drawing command, and a rectangle drawing command.

[0608] In an optional embodiment, the graph drawing instruction includes at least one of a touch gesture and a touch instruction.

[0609] As a selective embodiment, recognizing a figure corresponding to the graph drawing command and determining the graph type based on the figure is, specifically, If the graph drawing command includes an arrow drawing command, the graph type is determined to be a line graph, or If the graph drawing instruction includes a circular drawing instruction, the graph type is determined to be a pie chart, or, If the graph drawing command includes a rectangle drawing command, the graph type is determined to be a histogram, or If the graph drawing command includes a cylindrical drawing command, the graph type is determined to be a bar graph, or If the graph drawing command includes a polygon drawing command, the graph type is determined to be a radar map.

[0610] As an optional embodiment, further, The data recognition result includes two columns of cells, the arrow drawing command includes a first arrow command and a second arrow command drawn sequentially, the first arrow command corresponds to the first column cell in the two-column cell arrangement, and the second arrow command corresponds to the second column cell in the two-column cell arrangement. If the first direction of the first arrow command is the X-axis direction, the first column cells are used as the X-axis data of the graph, and the second column cells are used as the Y-axis data of the graph.

[0611] If the first direction of the first arrow command is the Y-axis direction, the first column cells are used as the Y-axis data of the graph, and the second column cells are used as the X-axis data of the graph.

[0612] Here, the reception time for the first arrow command is earlier than the reception time for the second arrow command. Also, the second direction of the second arrow command is different from the first direction.

[0613] As an optional embodiment, further, In response to an intelligent drawing graph activation command triggered by the user on the first display interface of the writing application, a second display interface in intelligent drawing graph mode is displayed in the display area. After drawing and displaying a graph in the second display interface, the graph from the second display interface is inserted into the first display interface of the writing application and displayed in response to a user's insertion command.

[0614] In an optional embodiment, the intelligent drawing graph activation command includes a second touch gesture to display a second display interface in intelligent drawing graph mode in the display area. Specifically, The system receives a second shape drawn by the user on the first display interface of the writing application, wherein the second shape is a second shape generated based on the second touch gesture. Based on the second figure, the system is triggered to display a second display interface in intelligent drawing graph mode in the display area.

[0615] In an optional embodiment, after receiving a second figure drawn by the user on the first display interface of the writing application, further, In the first display interface, a candidate menu is generated, and the candidate menu includes a first launch icon corresponding to the second figure. In response to a user touching the first launch icon, the second graphic is displayed in the display area.

[0616] As a selective embodiment, generating candidate menus in the first display interface is, specifically If the second shape is cylindrical, the first activation icon is a bar icon, and in response to a user touch operation on the bar icon, it triggers the display of a second display interface corresponding to a bar graph. If the second shape is a rectangle, the first launch icon is a hist icon, and in response to a user touch operation on the hist icon, it triggers the display of a second display interface corresponding to the histograph. If the second shape is circular, the first launch icon is a circular icon, and in response to a user touch operation on the circular icon, it triggers the display of a second display interface corresponding to a pie chart. If the second figure is a polygon, the first activation icon is a radar icon, and in response to a user touch operation on the radar icon, it triggers the display of a second display interface corresponding to the radar map. If the second shape is an arrow, the first activation icon is a line icon, and in response to a user touch operation on the line icon, it triggers the display of a second display interface corresponding to a line graph.

[0617] In an optional embodiment, the candidate menu further includes a second launch icon, the second launch icon including an analysis template icon.

[0618] In response to a user's touch operation on an analysis template icon, the analysis template corresponding to the analysis template icon is inserted into the first display interface.

[0619] In an optional embodiment, before responding to the user's graph plotting command, The display area of ​​the aforementioned display screen is configured to have a writing area and a graph area. The writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information.

[0620] As an optional embodiment, further, The writing area and graph area are determined within the display area of ​​the aforementioned display screen, the writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information. In the writing area of ​​the display area, a writing presentation table corresponding to the graph type is displayed, and / or, In the graph area of ​​the display area, a predetermined graph corresponding to the graph type is displayed.

[0621] In an optional embodiment, the graph region is configured to receive graph drawing commands from the user.

[0622] As a selective embodiment, determining the writing area and graph area within the display area of ​​the display screen specifically means, In response to a user's editing command in smart graph mode for a known graph type, the display area is divided into a writing area and a graph area, and the graph area is configured to display a graph corresponding to the graph type.

[0623] As an optional embodiment, further, In response to a user's drag command on the writing area, the contents of the writing area are moved in the corresponding direction, or In response to a user's drag command on a graph area, the contents of the graph area are moved in the corresponding direction.

[0624] As an optional embodiment, further, In response to the user's full-screen preview command, the graph area is displayed in full screen, or, In response to a user's annotation operation in the graph area, display content corresponding to the said annotation operation, or In response to a user's erase operation in the graph area, either the content corresponding to the erase operation in the graph area is erased, or In response to a user's clear operation in the graph area, either clear the contents of the graph area, or In response to a user save operation in the graph area, either save the contents of the graph area, or In response to a user command to switch graph types, the system switches the current graph type.

[0625] In a selective embodiment, the positions of the writing area and the graph area do not overlap, or The graph area is raised above the writing area and does not obstruct the contents of the writing area.

[0626] As a selective embodiment, determining the graph area within the display area of ​​the display screen specifically means, Based on predefined location information, determine the graph area within the display area of ​​the display screen, or, The user responds to a first partition command within the display area to determine the graph area corresponding to the first partition command within the display area.

[0627] As a selective embodiment, after displaying the drawn graph in the graph area, further, In response to the user's second partition command, the writing area is divided into a first writing area and a second writing area, and the graph area is divided into a first graph area and a second graph area. The first writing area is configured to display writing trajectory information received before division, the second writing area is configured to display writing trajectory information received after division, the first graph area is configured to display a graph corresponding to the writing trajectory information in the first writing area, and the second graph area is configured to display a graph corresponding to the writing trajectory information in the second writing area.

[0628] As a selective embodiment, the handwriting trajectory information is recognized as follows.

[0629] The handwriting trajectory information is input to a text recognition model for recognition, and data content corresponding to the handwriting trajectory information is output. The text recognition model includes multiple BiLSTM network layers, and at least two outputs are connected to the inputs of any of the BiLSTM network layers except the first two. The at least two outputs include outputs from any two BiLSTM network layers prior to any one of the BiLSTM network layers.

[0630] In an optional embodiment, the text recognition model further includes a fully connected layer, to which at least three outputs are coupled, the at least three outputs including outputs from any three BiLSTM network layers prior to the fully connected layer.

[0631] In a selective embodiment, the output from any BiLSTM network layer prior to the last BiLSTM network layer is used as an input to at least two subsequent BiLSTM network layers.

[0632] In an optional embodiment, the text recognition model includes four BiLSTM network layers and a fully connected layer, the first BiLSTM network layer including a first input and a first output, the second BiLSTM network layer including a second input and a second output, the third BiLSTM network layer including a third input and a third output, and the fourth BiLSTM network layer including a fourth input and a fourth output. The first input is configured to input the handwriting trajectory information. The second input connects the first output. The third input connects the first output and the second output, the fourth input connects the first output, the second output and the third output, the fourth output connects the fully connected layer inputs, and the output from the fully connected layer is configured to output data content corresponding to the handwriting trajectory information.

[0633] As a selective embodiment, after responding to a graph selection command in the user's intelligent drawing graph mode, further, The display area displays a writing presentation table, and the user receives the writing trajectory information entered on the writing presentation table. When user writing stops and a predetermined time is waited, a graph of the graph type corresponding to the graph selection command is drawn and displayed.

[0634] As an optional embodiment, before the user responds to a graph drawing command in intelligent drawing graph mode, The system receives handwriting trajectory information written by the user, recognizes the handwriting trajectory information, and obtains the data recognition result. After responding to a graph drawing command in the user's intelligent drawing graph mode, the processor is configured to perform the following actions:

[0635] Based on the data recognition results, a graph of the graph type corresponding to the graph drawing command is drawn and displayed.

[0636] As a selective embodiment, obtaining the data recognition results specifically means, Multiple datasets are determined by dividing the user's handwritten content into columns, or, Multiple datasets are determined by dividing the user's handwritten content line by line.

[0637] In a selective embodiment, the data recognition results here include multiple datasets, and the multiple datasets are determined as follows.

[0638] When the handwriting display table is displayed in the aforementioned display area, the system either obtains multiple datasets by dividing the content entered by the user in the handwriting display table into columns or rows, or If the handwriting presentation table is not displayed in the aforementioned display area, after determining each cell, the data content of each cell is divided by column or row to obtain multiple datasets.

[0639] As a selective embodiment, determining the plurality of datasets specifically means, Based on the pre-defined correspondence between each row and the coordinate axis, the contents of each column written by the user are determined as each dataset, or, Based on the pre-defined correspondence between each column and the coordinate axes, the content of each line written by the user is determined to be a separate dataset.

[0640] As a selective embodiment, determining the plurality of datasets specifically means, The content written by the user that is located on a predetermined line is determined to be the title of each of the coordinate axes, and the remaining content is determined to be the data content, or, The content written by the user that is located in a predetermined column is determined to be the title of each of the coordinate axes, and the remaining content is determined to be the data content.

[0641] As a selective embodiment, in response to the user's graph drawing command, drawing a graph of the graph type corresponding to the graph drawing command based on the data recognition result is, specifically, The graph type is determined according to the graph plotting command. Based on the data sets corresponding to the content written by the user, a graph corresponding to the graph type is drawn.

[0642] As an optional embodiment, if any of the datasets contain content in any of the columns written by the user, the graph is drawn as follows:

[0643] If the graph type is determined to be a bar graph, the classification titles on the predetermined coordinate axes are determined based on the dataset corresponding to the predetermined column, and the column names are determined based on the first content corresponding to the predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column. Based on the content excluding the first content in the dataset corresponding to the remaining columns, the column heights corresponding to each classification title and column name are determined, and the bar graph is drawn based on each classification title, column name, and column height, or If the graph type is determined to be a radar map, the title of each vertex is determined based on the dataset corresponding to a predetermined column, the polar coordinate name is determined based on the second content corresponding to a predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column, the polar coordinates corresponding to each title and polar coordinate name are determined based on the content excluding the second content in the dataset corresponding to the remaining columns, and the radar map is drawn based on each title, polar coordinate name and polar coordinate, or If the graph type is determined to be a line graph, the title of the coordinate axis is determined based on the content corresponding to a predetermined index value in the dataset. The coordinate points are determined based on the content corresponding to the same index value other than the predetermined index value in different datasets. Based on the title of the coordinate axis and the coordinate points, a line graph is drawn, or If the graph type is determined to be a pie chart, the title of each partition is determined based on the dataset corresponding to the designated column. The occupancy rate of each partition is determined based on the dataset corresponding to the columns other than the designated column, and the pie chart is drawn based on the title and occupancy rate of each partition.

[0644] As a selective embodiment, drawing a graph of the graph type corresponding to the graph drawing command based on the data recognition result is, specifically, In intelligent drawing graph mode for known graph types, the graph drawing is triggered after it is determined that the user has stopped writing and has waited for a predetermined amount of time.

[0645] As an optional embodiment, further, In response to a user's editing command in smart graph mode for a known graph type, a predetermined graph is displayed in the display area, where the display specifications of the predetermined graph are determined based on a known graph type.

[0646] As an optional embodiment, further, After determining the graph type, a guide template corresponding to the graph type is displayed in the display area of ​​the display screen, and the guide template guides the user to write table contents based on the writing specifications corresponding to the graph type.

[0647] As a selective embodiment, displaying a guide template corresponding to the graph type in the display area of ​​the display screen is, specifically, A table template is displayed in the first partition of the guide template, and a graph template is displayed in the second partition of the guide template, with the table template containing predetermined content. The graph template contains a graph corresponding to the predetermined content.

[0648] As a selective embodiment, obtaining the data recognition results specifically means, If the user stops writing and it is determined that no writing has occurred after a predetermined time, the system recognizes the writing trajectory information in the display area of ​​the display screen and obtains the data recognition result.

[0649] As a selective embodiment, recognizing the handwriting trajectory information in the display area of ​​the display screen and obtaining the data recognition result specifically means, The system recognizes trajectory points within the handwriting trajectory information of the display area of ​​the aforementioned display screen where the pixel value of the trajectory point is greater than the pixel threshold, and obtains the data recognition result.

[0650] As a selective embodiment, after displaying the drawn graph in the display area, further, In response to a user's command to add content, the system adds the corresponding writing trajectory information and updates the graph based on the added writing trajectory information, or, In response to a user's command to add a stroke, the system adds a stroke to the corresponding writing trajectory information and updates the graph based on the writing trajectory information after the stroke has been added, or, In response to the user's erase command, the corresponding writing trajectory information is erased, and the graph is updated based on the erased writing trajectory information, or, In response to the user's clear command, all writing trajectory information is cleared.

[0651] As a selective embodiment, after displaying the drawn graph in the display area, further, In response to a user's correction command, the system corrects the corresponding writing trajectory information and updates the graph based on the corrected writing trajectory information.

[0652] As an optional embodiment, in response to the user's modification command, further, The corrected writing trajectory information is acquired and stored in a first memory area, which is different from the memory area where the original writing trajectory information is stored.

[0653] In an optional embodiment, the modification command includes at least one of the following: a command triggered by a different tip of the stylus, a command triggered by a different key on the stylus, and a command triggered by a display area switching button.

[0654] As an optional embodiment, in response to the user's modification command, further, Determine the first content before modification and the second content after modification corresponding to the aforementioned modification order. The first and second contents are displayed simultaneously, and the display formats for the first and second contents are different.

[0655] As a selective embodiment, after acquiring the modified writing trajectory information and storing it in the first memory area, further, The corrected writing trajectory information is obtained from the first memory area, and a first graph is drawn based on the writing trajectory information before correction and the corrected writing trajectory information. The first graph includes a first table data corresponding to the handwriting trajectory information report before correction and a second table data corresponding to the handwriting trajectory information report after correction.

[0656] As a selective embodiment, after acquiring the modified writing trajectory information and storing it in the first memory area, further, The corrected writing trajectory information is obtained from the first memory area, a second graph is drawn based on the writing trajectory information before correction, and a third graph is drawn based on the corrected writing trajectory information. The second graph and the third graph are displayed simultaneously.

[0657] Example 3, based on the same inventive concept, further provides an embodiment of the present invention that provides a graph display device. This device is an apparatus relating to a method according to an embodiment of the present invention, and since the principle of solving the problem of this device is similar to that of the method, the implementation of this device can be referenced from the implementation of the method, and any overlapping parts will be omitted.

[0658] As shown in Figure 26, the device is A recognition unit 2600 is configured to recognize handwriting trajectory information in the display area of ​​the aforementioned display screen and to acquire data recognition results, A determination unit 2601 is configured to determine the graph type corresponding to a graph drawing command in response to a user's graph drawing command, The system includes a drawing unit 2602 configured to respond to a user's graph drawing command, draw a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and display the drawn graph in the display area.

[0659] As a selective embodiment, recognize the handwriting trajectory information in the display area of ​​the display screen and, before acquiring the data recognition result, A writing presentation table is displayed in the aforementioned display area, and the writing presentation table is presented in such a way that the user writes in each cell of the writing presentation table. The system further includes a table presentation unit configured to receive handwriting trajectory information entered by the user on the handwriting presentation table.

[0660] As a selective embodiment, the table presentation unit specifically, Based on the position of the writing trajectory information first written by the user in the display area, the reference position of the writing display table is determined. A writing display table with a fixed number of rows and columns is displayed in the display area according to the reference position of the writing display table, and the writing trajectory information written by the user for the first time is located within a cell of the writing display table.

[0661] As a selective embodiment, the table presentation unit specifically, If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of rows and / or columns in the writing presentation table will be increased according to the position of the writing trajectory information that the user continues to write, or If the writing trajectory information that the user continues to write is in a critical row of the writing presentation table, the number of rows in the writing presentation table is increased, and / or if the writing trajectory information that the user continues to write is in a critical column of the writing presentation table, the number of columns in the writing presentation table is increased.

[0662] In an optional embodiment, the number of rows and columns in the handwriting display table is fixed or determined based on parameters entered by the user.

[0663] In a selective embodiment, the recognition unit 2600 specifically determines the data recognition result in one of the following ways:

[0664] Method 1: Based on the positional relationship between each trajectory point in the handwriting trajectory information and the cell in the handwriting presentation table, the handwriting trajectory information contained in each cell is determined, the handwriting trajectory information contained in each cell is recognized, and the data content of each cell is acquired.

[0665] Method 2: The method recognizes the handwriting trajectory information, obtains the content written by the user and its position in the display area of ​​the content, and determines the data content of each cell based on the position of the content and the position of the cell in the handwriting presentation table.

[0666] In a selective embodiment, the data recognition result includes a plurality of cells, and the data recognition result includes the position of each cell and the data content within each cell.

[0667] In a selective embodiment, the recognition unit 2600 specifically determines the data recognition result as follows:

[0668] Each cell is determined, and the handwriting trajectory information is divided into each cell. Here, each cell is used to correspond to each cell in the table, and the handwriting trajectory information contained in each cell is used to correspond to the data content in each cell. The handwriting trajectory information in each cell is recognized, and the data content in each cell is retrieved. The data recognition result is determined based on the position of each cell and the data content within each cell.

[0669] In a selective embodiment, the recognition unit 2600 specifically divides the writing trajectory into individual cells as follows: The writing trajectory information is converted into a stroke, and the centroid of the stroke is determined based on the average coordinate values ​​of the trajectory points in the stroke. Based on the positional relationship between the center of gravity of the stroke and each cell, the stroke is divided into individual cells.

[0670] In a selective embodiment, the recognition unit 2600 specifically divides the writing trajectory into individual cells as follows: The writing order and coordinates of each trajectory point in the aforementioned writing trajectory information are obtained.

[0671] Based on the writing order and coordinates of each trajectory point, the degree of similarity between each trajectory point is determined.

[0672] The trajectory points whose similarity satisfies a predetermined threshold are divided into individual cells.

[0673] In a selective embodiment, the recognition unit 2600 is configured such that each cell is determined as follows:

[0674] Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, the column separator lines are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information contained in any of the columns, the row separator lines for any of the columns are determined; and each cell is determined based on the column separator lines and row separator lines, or, Based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information contained in any row, column separators for any row are determined; and each cell is determined based on the row separators and column separators, or, Based on the valley positions in the first projection histogram on the first coordinate axis of the handwriting trajectory information, column separators are determined; based on the valley positions in the second projection histogram on the second coordinate axis of the handwriting trajectory information, row separators are determined; and each cell is determined based on the row separators and column separators.

[0675] In a selective embodiment, the recognition unit 2600 is configured such that each cell is determined as follows:

[0676] Each cell is determined using a clustering algorithm.

[0677] In a selective embodiment, the recognition unit 2600 specifically determines the column separator lines as follows:

[0678] Based on the valley locations where the valley width in the first projection histogram is greater than a threshold, the column separator lines are determined, and / or, The line separators are determined as follows:

[0679] The row separators are determined based on the locations of valleys in the second projection histogram where the valley width is greater than the threshold.

[0680] In a selective embodiment, the writing trajectory information is obtained after performing a resampling operation on the trajectory points in the writing trajectory information. The resampling is configured to increase the number of trajectory points in the corresponding stroke according to the stroke length in the writing trajectory information.

[0681] In a selective embodiment, the first projected histogram is a projected histogram obtained by projecting the writing trajectory information onto a first coordinate axis, and is obtained by reduction mapping.

[0682] The second projection histogram is obtained by projecting the writing trajectory information onto a second coordinate axis, and is obtained by reduction mapping.

[0683] As a selective embodiment, recognize the handwriting trajectory information in the display area of ​​the display screen and, before acquiring the data recognition result, A title prompt is displayed in the aforementioned display area, and the title prompt indicates the writing position of the user table title. The system further includes a display prompt unit configured to acquire first writing trajectory information in a first region corresponding to the title prompt, and to acquire second writing trajectory information in a second region corresponding to the title prompt.

[0684] In an optional embodiment, the display prompt unit is further configured to determine the data recognition result as follows:

[0685] The handwriting trajectory information of the first region is recognized, and the title of each coordinate axis is obtained.

[0686] The handwriting trajectory information in the second region is divided into individual cells, the handwriting trajectory information in each cell is recognized, and the data content in each cell of the second region is obtained.

[0687] The data recognition result is determined based on the aforementioned title, the data content within each cell of the second region, and the positional relationship between the title and each cell.

[0688] In an optional embodiment, the graph drawing command includes a graph selection command, and the drawing unit 2602 specifically determines the graph type as follows:

[0689] In response to a graph selection command from the user in intelligent drawing graph mode, the system determines the graph type corresponding to the graph selection command.

[0690] In an optional embodiment, the graph drawing command includes a first touch gesture, and the drawing unit 2602 specifically determines the graph type as follows: In response to a first touch gesture by the user in intelligent drawing graph mode, the system recognizes the first shape generated by the first touch gesture and determines the graph type based on the first shape.

[0691] In an optional embodiment, the graph drawing command includes at least one of the following: an arrow drawing command, a circle drawing command, a polygon drawing command, a cylindrical drawing command, and a rectangle drawing command.

[0692] In an optional embodiment, the graph drawing instruction includes at least one of a touch gesture and a touch instruction.

[0693] As an optional embodiment, the drawing unit 2602 specifically: If the graph drawing command includes an arrow drawing command, the graph type is determined to be a line graph, or If the graph drawing instruction includes a circular drawing instruction, the graph type is determined to be a pie chart, or, If the graph drawing command includes a rectangle drawing command, the graph type is determined to be a histogram, or If the graph drawing command includes a cylindrical drawing command, the graph type is determined to be a bar graph, or If the graph drawing command includes a polygon drawing command, the graph type is determined to be a radar map.

[0694] As an optional embodiment, the drawing unit 2602 specifically, The data recognition result includes two columns of cells, the arrow drawing command includes a first arrow command and a second arrow command drawn sequentially, the first arrow command corresponds to the first column cell in the two-column cell arrangement, and the second arrow command corresponds to the second column cell in the two-column cell arrangement. If the first direction of the first arrow command is the X-axis direction, the first column cells are used as the X-axis data of the graph, and the second column cells are used as the Y-axis data of the graph.

[0695] If the first direction of the first arrow command is the Y-axis direction, the first column cells are used as the Y-axis data of the graph. The second column cells are used as the X-axis data of the graph. Here, the reception time for the first arrow command is earlier than the reception time for the second arrow command. Also, the second direction of the second arrow command is different from the first direction.

[0696] As an optional embodiment, In response to an intelligent drawing graph activation command triggered by the user on the first display interface of the writing application, a second display interface in intelligent drawing graph mode is displayed in the display area. The system further includes an insertion function unit configured to, after drawing and displaying a graph in the second display interface, insert and display the graph from the second display interface in the first display interface of the writing application in response to an insertion command from the user.

[0697] In an optional embodiment, the intelligent drawing graph activation command includes a second touch gesture, and the insertion function unit specifically, The system receives a second shape drawn by the user on the first display interface of the writing application, wherein the second shape is a second shape generated based on the second touch gesture. Based on the second figure, the system is triggered to display a second display interface in intelligent drawing graph mode in the display area.

[0698] In an optional embodiment, after receiving a second figure drawn by the user on the first display interface of the writing application, In the first display interface, a candidate menu is generated that includes a first launch icon corresponding to the second figure. The system further includes a startup function unit configured to display the second graphic in the display area in response to a user touch operation on the first startup icon.

[0699] As a selective embodiment, the activation function unit is: If the second shape is cylindrical, the first activation icon is a bar icon, and in response to a user touch operation on the bar icon, it triggers the display of a second display interface corresponding to a bar graph. If the second shape is a rectangle, the first launch icon is a hist icon, and in response to a user touch operation on the hist icon, it triggers the display of a second display interface corresponding to the histograph. If the second shape is circular, the first launch icon is a circular icon, and in response to a user touch operation on the circular icon, it triggers the display of a second display interface corresponding to a pie chart. If the second figure is a polygon, the first activation icon is a radar icon, and in response to a user touch operation on the radar icon, it triggers the display of a second display interface corresponding to the radar map. If the second shape is an arrow, the first activation icon is a line icon, and in response to a user touch operation on the line icon, it triggers the display of a second display interface corresponding to a line graph.

[0700] In an optional embodiment, the candidate menu further includes a second launch icon, the second launch icon includes an analysis template icon. The launch function unit specifically includes, In response to a user's touch operation on an analysis template icon, the analysis template corresponding to the analysis template icon is inserted into the first display interface.

[0701] In an optional embodiment, the system further includes a first partition unit configured to determine a writing area and a graph area in the display area of ​​the display screen before responding to the user's graph drawing command. The writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information.

[0702] As an optional embodiment, a second partition unit is further included. The second partition unit determines the writing area and graph area in the display area of ​​the display screen, the writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information. In the writing area of ​​the display area, a writing presentation table corresponding to the graph type is displayed, and / or, In the graph area of ​​the display area, a predetermined graph corresponding to the graph type is displayed.

[0703] In an optional embodiment, the graph region is configured to receive graph drawing commands from the user.

[0704] As an optional embodiment, the first partition unit specifically comprises: In response to a user's editing command in smart graph mode for a known graph type, the display area is divided into a writing area and a graph area, and the graph area is configured to display a graph corresponding to the graph type.

[0705] As an optional embodiment, the first partition unit specifically comprises: In response to a drag command from the user to the writing area, the contents of the writing area are moved in the corresponding direction, or In response to a user's drag command on a graph area, the contents of the graph area are moved in the corresponding direction.

[0706] As an optional embodiment, the first partition unit specifically comprises: In response to the user's full-screen preview command, the graph area is displayed in full screen, or, In response to a user's annotation operation in the graph area, display content corresponding to the said annotation operation, or In response to a user's erase operation in the graph area, either the content corresponding to the erase operation in the graph area is erased, or In response to a user's clear operation in the graph area, either clear the contents of the graph area, or In response to a user save operation in the graph area, either save the contents of the graph area, or In response to a user command to switch graph types, the system switches the current graph type.

[0707] In a selective embodiment, the positions of the writing area and the graph area do not overlap, or The graph area is raised above the writing area and does not obstruct the contents of the writing area.

[0708] As an optional embodiment, the first partition unit specifically comprises: Based on predefined location information, determine the graph area within the display area of ​​the display screen, or, The user responds to a first partition command within the display area to determine the graph area corresponding to the first partition command within the display area.

[0709] In an optional embodiment, the system further includes a repartitioning unit after displaying the drawn graph in the graph area.

[0710] The repartitioning unit, in response to the user's second partition command, divides the writing area into a first writing area and a second writing area, and divides the graph area into a first graph area and a second graph area. The first writing area is configured to display writing trajectory information received before division, the second writing area is configured to display writing trajectory information received after division, the first graph area is configured to display a graph corresponding to the writing trajectory information in the first writing area, and the second graph area is configured to display a graph corresponding to the writing trajectory information in the second writing area.

[0711] In a selective embodiment, the recognition unit 2600 specifically recognizes the writing trajectory information as follows:

[0712] The handwriting trajectory information is input to a text recognition model for recognition, and data content corresponding to the handwriting trajectory information is output. The text recognition model includes multiple BiLSTM network layers, and at least two outputs are connected to the inputs of any of the BiLSTM network layers except the first two. The at least two outputs include outputs from any two BiLSTM network layers prior to any one of the BiLSTM network layers.

[0713] In an optional embodiment, the text recognition model further includes a fully connected layer to which at least three outputs are coupled. The at least three outputs include outputs from any three BiLSTM network layers prior to the fully connected layer.

[0714] In a selective embodiment, the output from any BiLSTM network layer prior to the last BiLSTM network layer is used as an input to at least two subsequent BiLSTM network layers.

[0715] In an optional embodiment, the text recognition model includes four BiLSTM network layers and a fully connected layer, the first BiLSTM network layer including a first input and a first output, the second BiLSTM network layer including a second input and a second output, the third BiLSTM network layer including a third input and a third output, and the fourth BiLSTM network layer including a fourth input and a fourth output. The first input is configured to input the handwriting trajectory information. The second input connects the first output. The third input connects the first output and the second output, the fourth input connects the first output, the second output and the third output, the fourth output connects the fully connected layer inputs, and the output from the fully connected layer is configured to output data content corresponding to the handwriting trajectory information.

[0716] In a selective embodiment, after responding to a graph selection command in the user's intelligent drawing graph mode, the drawing unit 2602 further, specifically, The display area displays a writing presentation table, and the user receives the writing trajectory information entered on the writing presentation table. When user writing stops and a predetermined time is waited, a graph of the graph type corresponding to the graph selection command is drawn and displayed.

[0717] In a selective embodiment, before the user responds to a graph drawing command in intelligent drawing graph mode, the recognition unit 2600 further specifically: The system receives handwriting trajectory information written by the user, recognizes the handwriting trajectory information, and obtains the data recognition result.

[0718] After responding to a graph drawing command in the user's intelligent drawing graph mode, the drawing unit 2602, specifically, Based on the data recognition results, a graph of the graph type corresponding to the graph drawing command is drawn and displayed.

[0719] As a selective embodiment, the recognition unit 2600 specifically comprises: Multiple datasets are determined by dividing the user's handwritten content into columns, or, Multiple datasets are determined by dividing the user's handwritten content line by line.

[0720] In a selective embodiment, the data recognition results here include multiple datasets, and the multiple datasets are determined as follows.

[0721] When the handwriting display table is displayed in the aforementioned display area, the system either obtains multiple datasets by dividing the content entered by the user in the handwriting display table into columns or rows, or If the handwriting presentation table is not displayed in the aforementioned display area, after determining each cell, the data content of each cell is divided by column or row to obtain multiple datasets.

[0722] As a selective embodiment, the recognition unit 2600 specifically comprises: Based on the pre-defined correspondence between each row and the coordinate axis, the contents of each column written by the user are determined as each dataset, or, Based on the pre-defined correspondence between each column and the coordinate axes, the content of each line written by the user is determined to be a separate dataset.

[0723] As a selective embodiment, the recognition unit 2600 specifically comprises: The content written by the user that is located on a predetermined line is determined to be the title of each of the coordinate axes, and the remaining content is determined to be the data content, or, The content written by the user that is located in a predetermined column is determined to be the title of each of the coordinate axes, and the remaining content is determined to be the data content.

[0724] As an optional embodiment, the drawing unit 2602 specifically: The graph type is determined according to the graph plotting command. Based on each dataset corresponding to the content written by the user, a graph corresponding to the aforementioned graph type is drawn.

[0725] In an optional embodiment, if any of the datasets contains the contents of any column written by the user, the plotting unit 2602 plots the graph as follows:

[0726] If the graph type is determined to be a bar graph, then, based on the dataset corresponding to the predetermined column, the classification titles on the predetermined coordinate axes are determined; based on the first content corresponding to the predetermined index value in the dataset corresponding to the remaining columns (excluding the predetermined column), the column names are determined; based on the content excluding the first content in the dataset corresponding to the remaining columns, the column heights corresponding to each classification title and column name are determined; and based on each classification title, column name, and column height, the bar graph is drawn, or, If the graph type is determined to be a radar map, the title of each vertex is determined based on the dataset corresponding to a predetermined column, the polar coordinate name is determined based on the second content corresponding to a predetermined index value in the dataset corresponding to the remaining columns excluding the predetermined column, the polar coordinates corresponding to each title and polar coordinate name are determined based on the content excluding the second content in the dataset corresponding to the remaining columns, and the radar map is drawn based on each title, polar coordinate name and polar coordinate, or If the graph type is determined to be a line graph, the title of the coordinate axis is determined based on the content corresponding to a predetermined index value in the dataset. The coordinate points are determined based on the content corresponding to the same index value other than the predetermined index value in different datasets. Based on the title of the coordinate axis and the coordinate points, a line graph is drawn, or If the graph type is determined to be a pie chart, the title of each partition is determined based on the dataset corresponding to the designated column. The occupancy rate of each partition is determined based on the dataset corresponding to the columns other than the designated column, and the pie chart is drawn based on the title and occupancy rate of each partition.

[0727] As an optional embodiment, the drawing unit 2602 specifically, In intelligent drawing graph mode for known graph types, the graph drawing is triggered after it is determined that the user has stopped writing and has waited for a predetermined amount of time.

[0728] As an optional embodiment, the system further includes an editable display unit configured to display a predetermined graph in the display area in response to a user edit command in a smart graph mode of a known graph type, wherein the display specifications of the predetermined graph are determined based on a known graph type.

[0729] In an optional embodiment, after determining the graph type, a guide template corresponding to the graph type is displayed in the display area of ​​the display screen, and the guide template further includes a guide template display unit configured to guide the user so that they can write table contents based on the writing specifications corresponding to the graph type.

[0730] As a selective embodiment, the guide template display unit specifically, A table template is displayed in the f...

Claims

1. A display device, Includes a display screen and control circuit, The aforementioned display screen is configured to display content, The control circuit comprises a processor and memory, the memory being configured to store a program that the processor can execute. The processor reads the program in the memory, The handwriting trajectory information in the display area of ​​the aforementioned display screen is recognized, and the data recognition result is obtained. In response to a user's graph drawing command, the system determines the graph type corresponding to the graph drawing command. Based on the data recognition results, the system is configured to draw a graph of the graph type corresponding to the graph drawing command and to display the drawn graph in the display area. Before recognizing the handwriting trajectory information in the display area of ​​the aforementioned display screen and obtaining the data recognition result, the processor: A writing presentation table is displayed in the aforementioned display area. The system is configured to receive the handwriting trajectory information entered by the user on the handwriting display table. The writing presentation table is used to present the user to write in each cell of the writing presentation table, and the writing presentation table is a display device including cells.

2. The display device includes an electronic whiteboard, The aforementioned electronic whiteboard includes a touch component, The display device according to claim 1, wherein the touch component is configured to acquire handwritten trajectory information.

3. The aforementioned processor, Based on the position of the writing trajectory information first written by the user in the display area, the reference position of the writing display table is determined. The display device according to claim 1, wherein a writing presentation table with a fixed number of rows and columns is displayed in the display area according to the reference position of the writing presentation table, and the writing trajectory information written by the user for the first time is located within a cell of the writing presentation table.

4. After displaying the aforementioned handwriting presentation table, the processor: If the writing trajectory information that the user continues to write exceeds the range included in the writing presentation table, the number of rows and / or columns in the writing presentation table will be increased according to the position of the writing trajectory information that the user continues to write, or The display device according to claim 1, wherein if the writing trajectory information that the user continues to write is in a critical row of the writing display table, the number of rows in the writing display table is increased, and / or if the writing trajectory information that the user continues to write is in a critical column of the writing display table, the number of columns in the writing display table is increased.

5. The aforementioned processor, The display device according to claim 1, wherein the number of rows and columns of the handwriting display table is fixed or determined based on parameters entered by the user.

6. The data recognition result includes multiple cells, and the result includes the position of each cell and the data content within each cell. The aforementioned processor, Each cell is determined, the writing trajectory information is divided into each cell, each cell is used to correspond to each cell in the writing presentation table, and the writing trajectory information contained in each cell is used to correspond to the data content in each cell. The handwriting trajectory information in each of the aforementioned cells is recognized, and the data content in each of the aforementioned cells is acquired. The display device according to claim 1, which determines the data recognition result based on the position of each cell and the data content within each cell.

7. Before recognizing the handwriting trajectory information in the display area of ​​the aforementioned display screen and obtaining the data recognition result, the processor: A title prompt is displayed in the aforementioned display area, and the title prompt indicates the writing position of the user table title. The display device according to claim 1, configured to acquire first writing trajectory information in a first region corresponding to the title prompt, and to acquire second writing trajectory information in a second region corresponding to the title prompt.

8. The aforementioned processor, The handwriting trajectory information of the first region is recognized, and the title of each coordinate axis is obtained. The writing trajectory information of the second region is divided into the cells, the writing trajectory information in each cell is recognized, and the data content in each cell of the second region is obtained. The display device according to claim 7, which determines the data recognition result based on the title, the data content in each cell of the second region, and the positional relationship between the title and each cell.

9. The graph drawing instruction includes a graph selection instruction, and the processor, It is configured to determine the graph type corresponding to the graph selection command in response to the user's intelligent drawing graph mode, The graph drawing command includes a first touch gesture, The display device according to claim 1, wherein the processor is configured to respond to a first touch gesture in the user's intelligent drawing graph mode, recognize a first figure indicated by the first touch gesture, and determine the graph type based on the first figure.

10. The display device according to claim 1, wherein the graph drawing command includes at least one of the following: an arrow drawing command, a circle drawing command, a polygon drawing command, a cylindrical drawing command, and a rectangle drawing command.

11. The aforementioned processor, The data recognition result includes two columns of cells, the graph drawing command includes an arrow drawing command, the arrow drawing command includes a first arrow command and a second arrow command drawn sequentially, the first arrow command corresponds to the first column cell in the two columns of cells, and the second arrow command corresponds to the second column cell in the two columns of cells. If the first direction of the first arrow command is the X-axis direction, the first column cells will be used as the X-axis data of the graph, and the second column cells will be used as the Y-axis data of the graph. If the first direction of the first arrow command is the Y-axis direction, the first column cells will be used as the Y-axis data of the graph, and the second column cells will be used as the X-axis data of the graph. The display device according to claim 9, wherein the reception time of the first arrow command is earlier than the reception time of the second arrow command, and the second direction of the second arrow command is different from the first direction.

12. The aforementioned processor, In response to an intelligent drawing graph activation command triggered by the user on the first display interface of the writing application, a second display interface in intelligent drawing graph mode is displayed in the display area. The display device according to claim 1, wherein after drawing a graph in the second display interface and displaying the graph, in response to an insertion command from the user, the graph of the second display interface is inserted into the first display interface of the writing application and displayed.

13. The intelligent drawing graph invocation instruction includes a second touch gesture, and the processor, The system receives a second figure drawn by the user on the first display interface of the writing application, wherein the second figure is a second figure generated based on the second touch gesture. The display device according to claim 12, which triggers the display of the second display interface in the intelligent drawing graph mode in the display area based on the second figure.

14. After receiving the second figure drawn by the user on the first display interface of the writing application, the processor: In the first display interface, a candidate menu is generated, and the candidate menu includes a first launch icon corresponding to the second figure. The display device according to claim 13, which displays the second figure in the display area in response to a touch operation by the user on the first startup icon.

15. The aforementioned processor, The display device according to claim 12, wherein after drawing a graph in the second display interface and displaying the graph, in response to an insertion command from the user, the graph of the second display interface is inserted into the first display interface of the writing application and displayed.

16. Before responding to the user's graph plotting command, the processor: The display device according to claim 1, wherein a writing area and a graph area are determined in the display area of ​​the display screen, the writing area is configured to receive the user's writing trajectory information, and the graph area is configured to display a graph corresponding to the writing trajectory information.

17. After displaying the drawn graph in the graph area, the processor: In response to the user's second partition command, the writing area is divided into a first writing area and a second writing area, and the graph area is divided into a first graph area and a second graph area. The display device according to claim 16, wherein the first writing area is configured to display the writing trajectory information received before division, the second writing area is configured to display the writing trajectory information received after division, the first graph area is configured to display a graph corresponding to the writing trajectory information in the first writing area, and the second graph area is configured to display a graph corresponding to the writing trajectory information in the second writing area.

18. The aforementioned processor, The display device according to claim 1, which, in an intelligent drawing graph mode of a known graph type, triggers graph drawing when it is determined that a gesture command from the user has been received.

19. If the gesture instruction is an arrow gesture instruction, the processor will: Based on the arrow direction in the received gesture command, determine the table data corresponding to the X-axis, or, The display device according to claim 18, which determines table data corresponding to the Y-axis based on the direction of the arrow in the received gesture command.

20. The aforementioned processor, Multiple datasets are determined by dividing the content written by the user into columns, or, The display device according to claim 1, wherein multiple datasets are determined by dividing the content written by the user into line by line.

21. After displaying the drawn graph in the display area, the processor: The display device according to claim 1, which modifies the corresponding writing trajectory information in response to the user's modification command and updates the graph based on the modified writing trajectory information.

22. In response to the user's modification instruction, the processor: Determine the first content before modification and the second content after modification corresponding to the aforementioned modification order. The display device according to claim 21, which simultaneously displays the first content and the second content, and wherein the display formats of the first content and the second content are different.

23. After acquiring the corrected writing trajectory information and storing it in the first memory area, the processor: The corrected writing trajectory information is obtained from the first memory area, and a first graph is drawn based on the writing trajectory information before correction and the corrected writing trajectory information. The display device according to claim 22, wherein the first graph includes first table data corresponding to the writing trajectory information before correction and second table data corresponding to the corrected writing trajectory information.

24. After acquiring the corrected writing trajectory information and storing it in the first memory area, the processor: The corrected writing trajectory information is obtained from the first memory area, a second graph is drawn based on the writing trajectory information before correction, and a third graph is drawn based on the corrected writing trajectory information. The display device according to claim 21, which simultaneously displays the second graph and the third graph.

25. The method of displaying the graph, The steps include displaying a writing presentation table in the display area, The steps include receiving the handwriting trajectory information entered by the user on the handwriting display table, The steps include: recognizing the handwriting trajectory information in the display area of ​​the display screen and obtaining the data recognition result; The steps include: determining the graph type corresponding to the graph drawing command in response to the user's graph drawing command; The process includes the steps of: in response to the user's graph drawing command, drawing a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and displaying the drawn graph in the display area; The writing presentation table is used to present the user to write in each cell of the writing presentation table, and the writing presentation table includes cells, a method for displaying a graph.

26. A computer storage medium in which a computer program is stored, wherein the computer program is executed by a processor, The writing presentation table is displayed in the display area. The system receives the handwriting trajectory information entered by the user on the handwriting display table. The system recognizes the handwriting trajectory information in the display area of ​​the screen and obtains the data recognition result. In response to the user's graph drawing command, the graph type corresponding to the graph drawing command is determined. In response to the user's graph drawing command, it is used to draw a graph of the graph type corresponding to the graph drawing command based on the data recognition result, and to display the drawn graph in the display area. The writing presentation table is used to present the user to write in each cell of the writing presentation table, and the writing presentation table is a computer storage medium containing cells.

Images