Table recognition reconstruction method and device, storage medium and interactive panel

Abstract

The embodiment of the application provides a table recognition reconstruction method and device, a storage medium and an interactive tablet. The method comprises the following steps: when a trigger operation of a circle selection button on a stylus is listened to, obtaining target content obtained by the stylus through circle selection on a touch screen; when the target content comprises a table image, performing corner point detection on the table image to obtain each corner point of the table image and the position coordinates of the corner points; obtaining the position coordinates of each cell of the table image according to the position coordinates of the corner points; in the table image, cutting and obtaining the image of each cell; obtaining the text content in each cell; reconstructing the table structure of the table image according to the position coordinates of each cell; filling the text content in each cell into the corresponding cell in the reconstructed table structure to obtain a reconstructed table. The embodiment of the application saves the development cost, has small running amount, occupies less memory, has simple design, and improves the table reconstruction efficiency.

Description

Technical Field

[0001] This application relates to the field of table recognition, and in particular to a table recognition and reconstruction method, apparatus, storage medium, and interactive flat panel. Background Technology

[0002] Users can write on a whiteboard or display images on it. In some related technologies, when reconstructing a table from a whiteboard image using OCR, it's necessary to identify not only the table content but also its structure. Generally, a table image includes a table structure composed of cells and the text content within those cells. Some related technologies separate table structure detection from text content detection. Specifically, a text detection model detects the text positions in all cells of the table image; then, based on these positions, the table image is cropped to extract the text image, which is then input into a text recognition model to obtain the text content. Simultaneously, the table structure is predicted to obtain the overall table information and the position of each cell; then, the text positions are matched one-to-one with the cell positions, and finally, the corresponding text content is filled into the corresponding cells in the table to obtain the reconstructed table.

[0003] In the process of realizing this invention, the inventors discovered that: text detection models require a large amount of training data and the construction of complex deep learning models, which increases development costs. Moreover, the deep learning detection method using text detection models has a large amount of runtime, consumes a lot of memory, and is inefficient. In addition, after obtaining the text position using the text detection model, it is necessary to match the text position with the cell position one by one, which is computationally complex and results in low table reconstruction efficiency. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this application provides a table recognition and reconstruction method, apparatus, storage medium, and interactive flat panel, which can save development costs, reduce operating workload, occupy less memory, and have a simple design, thereby improving table reconstruction efficiency.

[0005] According to a first aspect of the embodiments of this application, a table recognition and reconstruction method is provided, comprising the following steps:

[0006] When the trigger operation of the selection button on the stylus is detected, the target content selected by the stylus on the touch screen is obtained;

[0007] When the target content includes a table image, a corner detection algorithm is used to detect corners in the table image to obtain each corner of the table image and the position coordinates of the corners.

[0008] Based on the position coordinates of the corner points, each corner point is classified into rows and columns to obtain the row and column relationships of each corner point;

[0009] Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0010] Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex.

[0011] If there exists a corner point whose distance is less than a preset threshold, then the corner point whose distance is less than the preset threshold is taken as the fourth vertex of the cell.

[0012] If no distance is less than a preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-established as the second vertex of the cell; the ordinate of the re-established second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained.

[0013] The position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0014] Select a new corner point as the first vertex of the next cell, and repeat the above steps to obtain the position coordinates of the next cell until all corner points are traversed and the position coordinates of each cell are obtained.

[0015] In the table image, the images of each cell are obtained by cropping according to the position coordinates of each cell;

[0016] Perform text recognition on the images of each cell to obtain the text content within each cell;

[0017] Based on the position coordinates of each cell, reconstruct the table structure of the table image;

[0018] The text content in each of the cells is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table;

[0019] On the touchscreen, the reconstructed table replaces the table image.

[0020] According to a second aspect of the embodiments of this application, a table recognition and reconstruction method is provided, comprising the following steps:

[0021] Obtain the image of the table to be recognized;

[0022] The table image is subjected to corner detection using a corner detection algorithm to obtain each corner of the table image and the position coordinates of the corners;

[0023] Based on the position coordinates of the corner points, each corner point is classified into rows and columns to obtain the row and column relationships of each corner point;

[0024] Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0025] Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex.

[0026] If there exists a corner point whose distance is less than a preset threshold, then the corner point whose distance is less than the preset threshold is taken as the fourth vertex of the cell.

[0027] If no distance is less than a preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-established as the second vertex of the cell; the ordinate of the re-established second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained.

[0028] The position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0029] Select a new corner point as the first vertex of the next cell, and repeat the above steps to obtain the position coordinates of the next cell until all corner points are traversed and the position coordinates of each cell are obtained.

[0030] In the table image, the images of each cell are obtained by cropping according to the position coordinates of each cell;

[0031] Perform text recognition on the images of each cell to obtain the text content within each cell;

[0032] Based on the position coordinates of each cell, reconstruct the table structure of the table image;

[0033] The text content in each of the cells is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

[0034] According to a third aspect of the embodiments of this application, a table recognition and reconstruction apparatus is provided, comprising:

[0035] The target content acquisition module is used to acquire the target content selected by the stylus on the touch screen when the trigger operation of the selection button on the stylus is detected.

[0036] The first corner information acquisition module is used to perform corner detection on the table image using a corner detection algorithm when the target content includes a table image, and obtain each corner of the table image and the position coordinates of the corners;

[0037] The first position coordinate acquisition module is used to classify each corner point into rows and columns based on the position coordinates of the corner points to obtain the row and column relationships of each corner point; select a corner point as the first vertex of a cell, select the first corner point located in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and select the first corner point located in the same column as the first vertex and below the first vertex as the third vertex of the cell; use the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, and calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex; if there is a distance less than a preset threshold, select the corner point whose distance is less than the preset threshold as the fourth vertex of the cell; If no distance is less than a preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-established as the second vertex of the cell; the ordinate of the re-established second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained; the position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex; a new corner point is selected as the first vertex of the next cell, and the position coordinates of the next cell are obtained again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained;

[0038] The first image cropping module is used to crop the image of each cell in the table image according to the position coordinates of each cell;

[0039] The first text content acquisition module is used to perform text recognition on the images of each cell to obtain the text content in each cell.

[0040] The first table structure acquisition module is used to reconstruct the table structure of the table image based on the position coordinates of each cell;

[0041] The first table reconstruction module is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

[0042] A table image replacement module is used to replace the table image with the reconstructed table on the touch screen.

[0043] According to a fourth aspect of the embodiments of this application, a table recognition and reconstruction apparatus is provided, comprising:

[0044] The table image acquisition module is used to acquire the table image to be recognized.

[0045] The second corner information acquisition module is used to perform corner detection on the table image using a corner detection algorithm to obtain each corner of the table image and the position coordinates of the corner.

[0046] The second position coordinate acquisition module is used to classify each corner point into rows and columns based on the position coordinates of the corner points to obtain the row and column relationships of each corner point; select a corner point as the first vertex of a cell, select the first corner point located in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and select the first corner point located in the same column as the first vertex and below the first vertex as the third vertex of the cell; use the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, and calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex; if there is a distance less than a preset threshold, select the corner point whose distance is less than the preset threshold as the fourth vertex of the cell; If no distance is less than a preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-established as the second vertex of the cell; the ordinate of the re-established second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained; the position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex; a new corner point is selected as the first vertex of the next cell, and the position coordinates of the next cell are obtained again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained;

[0047] The second image cropping module is used to crop the image of each cell in the table image according to the position coordinates of each cell;

[0048] The second text content acquisition module is used to perform text recognition on the images of each cell to obtain the text content in each cell.

[0049] The second table structure acquisition module is used to reconstruct the table structure of the table image based on the position coordinates of each cell;

[0050] The second table reconstruction module is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

[0051] According to a fifth aspect of the embodiments of this application, an electronic device is provided, including a processor and a memory; the memory stores a computer program adapted to be loaded by the processor and executed as described above in the table recognition and reconstruction method.

[0052] According to a sixth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the table recognition and reconstruction method as described above.

[0053] This application embodiment acquires the target content circled on the touchscreen by the stylus when the trigger operation of the selection button on the stylus is detected; when the target content includes a table image, corner point detection is performed on the table image to obtain each corner point and its position coordinates; based on the position coordinates of the corner points, row and column structure recognition is performed on each corner point to obtain the position coordinates of each cell in the table image; based on the position coordinates of each cell, the table image is cropped to obtain the image of each cell; text recognition is performed on the image of each cell to obtain the text content within each cell; based on the position coordinates of each cell... The table structure of the table image is reconstructed; the text content in each cell is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table; on the touch screen, the reconstructed table replaces the table image. The target content can be selected directly by triggering the selection button on the stylus, avoiding the need for users to frequently switch recognition modes manually on the touch screen. The operation is convenient and fast, and it does not require a text detection model to obtain text content, a large amount of training data, or the construction of a complex deep learning text detection model. It also does not require matching text positions with cells one by one, saving development costs, reducing runtime and memory usage, simplifying the design, and improving table reconstruction efficiency.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application.

[0055] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram illustrating the application environment of a table recognition and reconstruction method according to an embodiment of this application;

[0058] Figure 2 A flowchart illustrating a table recognition and reconstruction method according to one embodiment of this application;

[0059] Figure 3 This is a rendering of the character recognition and reconstruction according to one embodiment of this application;

[0060] Figure 4 This is a rendering of the recognition and reconstruction table shown in one embodiment of this application;

[0061] Figure 5 This is a block diagram illustrating the principle of a table recognition and reconstruction method according to one embodiment of this application;

[0062] Figure 6 A flowchart illustrating a table recognition and reconstruction method according to another embodiment of this application;

[0063] Figure 7 This is a schematic block diagram illustrating a table recognition and reconstruction apparatus according to one embodiment of this application;

[0064] Figure 8 A schematic block diagram of a table recognition and reconstruction apparatus shown in another embodiment of this application;

[0065] Figure 9 This is a schematic diagram of the structure of an electronic device according to one embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0067] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0068] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in the various drawings represent the same or similar elements. In the description of this application, it should be understood that the terms “first,” “second,” “third,” etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. The word “if” / “suppose” as used herein can be interpreted as “when,” “when,” or “in response to a determination.”

[0069] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0070] Before introducing the technical solution of this application, the relevant technologies mentioned in this application are explained below:

[0071] OCR (Optical Character Recognition) refers to the process by which electronic devices (such as scanners or digital cameras) examine printed characters on paper, determine their shapes by detecting dark and light patterns, and then translate the shapes into computer text using character recognition methods. In other words, for printed characters, optical methods are used to convert the text in paper documents into black and white dot matrix image files, and recognition software converts the text in the image into text format for further editing and processing by word processing software.

[0072] To better understand the technical solution of this application, some OCR recognition schemes in related technologies are explained below:

[0073] Users can write on the whiteboard or display images on it. In some related technologies, when OCR recognition of handwriting or images is required, the whiteboard's text recognition function needs to be activated while writing or displaying the image, and then deactivated after the text recognition is complete. In the process of developing this invention, the inventors discovered that when recognizing partial handwriting or partial images, the text recognition function needs to be frequently switched on and off on the whiteboard, which is cumbersome.

[0074] Furthermore, when recognizing table images, it is necessary to identify not only the table content but also the table structure. Generally, a table image includes a table structure composed of cells and the text content within those cells. In some related technologies, the method for recognizing and reconstructing table images separates table structure detection from text content detection. That is, a text detection model detects the text positions in all cells of the table image; then, based on the text positions, the table image is cropped to extract the text image, which is then input into a text recognition model to obtain the text content; simultaneously, the table structure is predicted to obtain the predicted overall table information and the position of each cell; then, a one-to-one correspondence is established between the text positions and the cell positions, and finally, the text content of the corresponding cells is filled into the corresponding cells in the table to obtain the reconstructed table.

[0075] In the process of realizing this invention, the inventors discovered that: text detection models require a large amount of training data and the construction of complex deep learning models, which increases development costs. Moreover, the deep learning detection method using text detection models has a large amount of runtime, consumes a lot of memory, and is inefficient. In addition, after obtaining the text position using the text detection model, it is necessary to match the text position with the cell position one by one, which is computationally complex and results in low table reconstruction efficiency.

[0076] To address the problems existing in related technologies, embodiments of this application provide a table recognition and reconstruction method. To better understand the technical solution of this application, before introducing the table recognition and reconstruction method provided in the embodiments of this application, the application environment of the table recognition and reconstruction method in the embodiments of this application will be described below.

[0077] Please refer to Figure 1 This is a schematic diagram illustrating the application environment of a table recognition and reconstruction method according to an embodiment of this application. The application environment of the table recognition and reconstruction method according to this embodiment includes a table recognition and reconstruction terminal 110 and a stylus 120.

[0078] The hardware referred to by the table recognition and reconstruction terminal 110 is essentially a computer device with capabilities equivalent to a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, including a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in secondary storage into memory for execution, executing the instructions in the program and interacting with the input / output devices to complete specific functions. Optionally, the table recognition and reconstruction terminal 110 can be any terminal device such as a mobile phone, tablet computer, PDA (Personal Digital Assistant), or smart TV; this application does not limit this. In this embodiment, the table recognition and reconstruction terminal 110 is an interactive tablet.

[0079] The table recognition and reconstruction terminal 110 may have at least one type of operating system installed, including but not limited to Android, Linux, and Windows systems. The table recognition and reconstruction terminal 110 may also have at least one application installed, which may be an application that comes pre-installed with the operating system or an application downloaded from a third-party device or server. In this embodiment, the table recognition and reconstruction terminal includes a whiteboard application; after the whiteboard application is launched, a whiteboard can be displayed on the table recognition and reconstruction terminal, on which handwriting can be written and images can be displayed.

[0080] The table recognition and reconstruction terminal 110 includes a touchscreen; a touchscreen is a sensing display device that integrates a display screen and touch components, and is sometimes referred to as a touch screen, touch panel, or touch panel in the art. When the touchscreen is touched, it can drive various connected devices according to a pre-programmed program to control the system. Then, with the system's built-in software, it can realize different functional applications, such as providing corresponding audio and video data feedback, and creating vivid audio-visual effects through the displayed screen. According to the working principle and information transmission medium, touchscreens can be divided into four types: resistive, capacitive, infrared, and surface acoustic wave, etc.

[0081] The stylus 120 is a small pen-shaped tool used to input commands onto touchscreen devices such as computer screens, mobile devices, and drawing tablets. Users can use the stylus to tap the touchscreen to select documents, draw, or write. Furthermore, the stylus 120 can also be equipped with several function buttons. The stylus can also wirelessly connect to the table recognition and reconstruction terminal 110 via methods such as Bluetooth, ultrasound, or infrared, allowing the table recognition and reconstruction terminal 110 to receive trigger operations from the stylus's function buttons.

[0082] Example 1

[0083] The following will be combined with the appendix Figures 2 to 4 This paper provides a detailed description of the table recognition and reconstruction method provided in the embodiments of this application.

[0084] Please see Figure 2 The table recognition and reconstruction method provided in this application includes the following steps:

[0085] Step S101: When the trigger operation of the selection button on the stylus is detected, the target content selected by the stylus on the touch screen is obtained.

[0086] The table recognition and reconstruction method of this application embodiment is applied to a table recognition and reconstruction terminal, wherein the table recognition and reconstruction terminal can be an interactive tablet; a whiteboard runs on the interactive tablet; and a selection button detection program can run on the whiteboard. When the selection button detection program detects the trigger operation of the selection button on the stylus, the whiteboard will acquire the target content selected by the stylus on the touch screen.

[0087] Step S102: When the target content includes a table image, perform corner detection on the table image to obtain each corner of the table image and the position coordinates of the corners.

[0088] The corner point can be the intersection of two lines or a point located on two adjacent objects with different main directions. In this embodiment, the corner point is the intersection of a horizontal edge and a vertical edge in the table image.

[0089] Optionally, the Harris corner detection method can be used to detect corners in the target image to obtain the corners of the table image and their coordinates. Specifically, a fixed window is slid across the table image in any direction, and the changes in pixel grayscale in the window before and after the sliding are compared. If there is a large change in grayscale in any direction, then we can assume that there are corners in the window. Then, by detecting the position with the largest change in grayscale, we can obtain the corners of the table image and their coordinates.

[0090] It should be understood that this application may also use the Moravec corner detection algorithm or the Shi-Tomasi algorithm for corner detection, and this application does not impose any restrictions.

[0091] Optionally, the target content may include handwritten notes, printed text, and / or images.

[0092] Please see Figure 3 When the target content includes handwriting, the target content is matched against a graphics library and a text library to obtain the matched graphics and / or text content. On the touchscreen, the matched graphics and / or text content replaces the target content. The graphics and text libraries can be known recognition libraries, either local or online.

[0093] When the target content includes printed text, the printed text remains unchanged, that is, the target image remains unchanged.

[0094] When the target content includes an image, the image content is extracted using an image recognition library. If the image does not contain a table, the image is left unchanged without any further action. Please refer to [link to relevant documentation]. Figure 4 If the image includes a table image, the table is identified and reconstructed according to the table recognition and reconstruction method of this application, and the table image is replaced with the reconstructed table.

[0095] Step S103: Based on the position coordinates of the corner points, perform row and column structure recognition on each corner point to obtain the position coordinates of each cell in the table image.

[0096] Specifically, when facing the table image, a rectangular coordinate system can be established with the center of the table image as the starting point, the horizontal axis as the X-axis, and the vertical axis as the Y-axis, to obtain the position coordinates of each corner point of the table image in the rectangular coordinate system.

[0097] The row and column structure is used to determine the row and column relationship between corner points. By combining the row and column relationship with the position coordinates of each corner point, the position coordinates of each cell in the table image can be obtained.

[0098] Step S104: In the table image, the images of each cell are obtained by cropping according to the position coordinates of each cell.

[0099] In the table image, each cell occupies a certain area of ​​the image. Once the position coordinates of each cell are known, the cell images can be cropped from the target image based on the position coordinates of each cell.

[0100] Step S105: Perform text recognition on the images of each cell to obtain the text content within each cell.

[0101] The text recognition model can be used to perform text recognition on each cell image to obtain the text content within each cell. The text recognition model can be trained based on cell image samples with labeled text content.

[0102] Step S106: Reconstruct the table structure of the table image based on the position coordinates of each cell.

[0103] The table structure is actually composed of cells, and the table structure can be reconstructed based on the position coordinates of each cell.

[0104] Step S107: Fill the text content in each cell into the corresponding cell in the reconstructed table structure to obtain the reconstructed table.

[0105] Step S108: On the touchscreen, replace the table image with the reconstructed table.

[0106] This application embodiment acquires the target content circled on the touchscreen by the stylus when the trigger operation of the selection button on the stylus is detected; when the target content includes a table image, corner point detection is performed on the table image to obtain each corner point and its position coordinates; based on the position coordinates of the corner points, row and column structure recognition is performed on each corner point to obtain the position coordinates of each cell in the table image; based on the position coordinates of each cell, the table image is cropped to obtain the image of each cell; text recognition is performed on the image of each cell to obtain the text content within each cell; based on the position coordinates of each cell... The table structure of the table image is reconstructed; the text content in each cell is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table; on the touch screen, the reconstructed table replaces the table image. The target content can be selected directly by triggering the selection button on the stylus, avoiding the need for users to frequently switch recognition modes manually on the touch screen. The operation is convenient and fast, and it does not require a text detection model to obtain text content, a large amount of training data, or the construction of a complex deep learning text detection model. It also does not require matching text positions with cells one by one, saving development costs, reducing runtime and memory usage, simplifying the design, and improving table reconstruction efficiency.

[0107] In one embodiment, the step S101, which involves listening to the trigger operation of the selection button on the stylus and obtaining the target content selected by the stylus on the touchscreen, includes:

[0108] Step S1011: When a trigger operation is detected by pressing the selection button on the stylus, the selection operation of the stylus on the touch screen is obtained when the selection button is pressed.

[0109] Optionally, the stylus is equipped with a first Bluetooth module, and the whiteboard is equipped with a second Bluetooth module, with a selection button detection program running on the whiteboard. When the user presses the selection button on the stylus, triggering the selection button, the stylus will generate trigger information corresponding to the selection button according to the HID (Human Interface Device) protocol, and send the trigger information to the whiteboard through the first and second Bluetooth modules. After obtaining the trigger information, the whiteboard's selection button detection program will acquire the selection operation of the stylus on the touch screen and draw a movement trajectory line on the whiteboard following the stylus.

[0110] In some embodiments, the stylus and whiteboard can also interact wirelessly through other means, such as ultrasonic or infrared, so that the whiteboard can detect the trigger operation caused by pressing the selection button on the stylus.

[0111] Step S1012: When the stylus is lifted or the selection button is not pressed, the touch points detected during the selection operation are connected to obtain the target content.

[0112] Optionally, when the stylus is lifted but the selection button on the stylus is still pressed, the whiteboard will connect the touch points detected during the selection process to obtain the target content. At this time, if the stylus is used to select again on the touch screen, the next target content will be obtained.

[0113] Optionally, before the stylus is lifted, if the selection button on the stylus is no longer pressed, the whiteboard will respond to the action of the selection button on the stylus no longer being pressed. That is, when the whiteboard does not receive the trigger information sent by the stylus, the whiteboard will also connect the touch points detected during the selection operation to obtain the target content.

[0114] The movement trajectory corresponding to the selection operation may be open or closed.

[0115] When the movement trajectory corresponding to the selection operation is closed, the touch points detected during the selection operation are sequentially connected to obtain a closed trajectory curve, and the content of the closed trajectory curve is used as the target content.

[0116] When the movement trajectory corresponding to the selection operation is not closed, the touch points detected during the selection operation are connected in sequence, and then the first and last touch points are connected to each other to form a closed trajectory. The content of the closed trajectory curve is used as the target content.

[0117] In this embodiment, the user only needs to trigger the selection button on the stylus. The touch screen then listens to the triggering operation of the selection button on the stylus and connects the touch points detected by the stylus during the selection operation to obtain the target content. This enables the recognition of the target content and avoids the user having to frequently switch recognition modes manually on the touch screen, making the operation convenient and quick.

[0118] In one embodiment, step S103, which involves identifying the row and column structure of each corner point based on its position coordinates to obtain the position coordinates of each cell in the target table, includes:

[0119] Step S1031: Based on the position coordinates of the corner points, classify each corner point into rows and columns to obtain the row and column relationships of each corner point.

[0120] Step S1032: Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0121] Step S1033: Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex.

[0122] Step S1034: If there exists a distance less than a preset threshold, the corner point whose distance is less than the preset threshold shall be taken as the fourth vertex of the cell.

[0123] like Figure 5 As shown, corner point A is selected as the first vertex of a cell. The first corner point C, which is in the same row as the first vertex and located to the right of the first vertex, is selected as the second vertex of the cell. The first corner point B, which is in the same column as the first vertex and located below the first vertex, is selected as the third vertex of the cell. If the distance between corner point D and the predicted fourth vertex is less than a preset threshold, then corner point D, whose distance is less than the preset threshold, is selected as the fourth vertex of the cell.

[0124] Step S1035: If there is no distance less than the preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-designated as the second vertex of the cell; the ordinate of the re-determined second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained.

[0125] like Figure 5As shown, corner point E is selected as the first vertex of a cell. The first corner point G, which is in the same row as the first vertex and located to the right of the first vertex, is selected as the second vertex of the cell. The first corner point F, which is in the same column as the first vertex and located below the first vertex, is selected as the third vertex of the cell. If no corner point is found to be less than a preset threshold in distance from the predicted fourth vertex, then the second corner point H, which is in the same row as the first vertex and located to the right of the first vertex, is selected as the second vertex of the cell again. The ordinate of the newly determined second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex. The distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated. If a corner point I is found to be less than a preset threshold in distance from the predicted fourth vertex, then the corner point I with a distance less than the preset threshold is selected as the fourth vertex of the cell.

[0126] Step S1036: Obtain the position coordinates of the cell based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0127] like Figure 5 As shown, based on the coordinates of corner point A, corner point B, corner point C, and corner point D, the coordinates of the four vertices of cell 1 can be obtained, thus determining the position coordinates of cell 1.

[0128] Step S1037: Select a new corner point as the first vertex of the next cell, and obtain the position coordinates of the next cell again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained.

[0129] This application embodiment determines the first vertex of a cell by selecting a corner point based on the row and column relationships of each corner point, designating the first corner point in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell, and designating the first corner point in the same column as the first vertex and located below the first vertex as the third vertex of the cell. The coordinates of the predicted fourth vertex are then calculated, and the distance between the coordinates of each corner point and the predicted coordinates of the fourth vertex is calculated to obtain the corresponding fourth vertex. This allows for the quick acquisition of the position coordinates of each cell.

[0130] Based on the above embodiments, step S1031, which involves classifying each corner point into rows and columns according to their position coordinates to obtain a set of row and column corner points, includes:

[0131] Step S10311: Calculate the first distance between the ordinates of each corner point, and obtain the set of corner points whose first distance is less than a first preset threshold.

[0132] The first preset threshold can be set according to the required precision. Optionally, the first preset threshold can generally be set to 10 pixels.

[0133] Step S10312: Calculate the second distance between the abscissas of each corner point in the corner point set, obtain the corner points whose second distance is less than a second preset threshold, and obtain the row and column relationship of each corner point.

[0134] The second preset threshold can be set according to the required precision. Optionally, the first preset threshold can generally be set to 10 pixels.

[0135] In this embodiment, the first distance between the ordinates of each corner point is compared with a first preset threshold, and the second distance between the abscissas of each corner point is compared with a second preset threshold, so as to quickly and accurately obtain the row and column relationship of each corner point.

[0136] Example 2

[0137] This application also provides a table recognition and reconstruction method. Please refer to [link to relevant documentation]. Figure 6 The table recognition and reconstruction method disclosed in this application includes the following steps:

[0138] Step S201: Obtain the image of the table to be recognized;

[0139] Step S202: Perform corner detection on the table image to obtain each corner of the table image and the position coordinates of the corners;

[0140] Step S203: Based on the position coordinates of the corner points, perform row and column structure recognition on each corner point to obtain the position coordinates of each cell in the table image;

[0141] Step S204: In the table image, the image of each cell is obtained by cropping according to the position coordinates of each cell;

[0142] Step S205: Perform text recognition on the images of each cell to obtain the text content within each cell;

[0143] Step S206: Reconstruct the table structure of the table image based on the position coordinates of each cell;

[0144] Step S207: Fill the text content in each cell into the corresponding cell in the reconstructed table structure to obtain the reconstructed table.

[0145] This embodiment of the application acquires a table image to be identified, performs corner detection on the table image to obtain each corner point and its position coordinates; based on the position coordinates of the corner points, performs row and column structure recognition on each corner point to obtain the position coordinates of each cell in the table image; in the table image, based on the position coordinates of each cell, crops to obtain the image of each cell; performs text recognition on the image of each cell to obtain the text content within each cell; based on the position coordinates of each cell, reconstructs the table structure of the table image; and fills the text content within each cell into the corresponding cells in the reconstructed table structure to obtain the reconstructed table. This embodiment of the application does not require a text detection model to obtain text content, does not require a large amount of training data or the construction of a complex deep learning text detection model, and does not require matching text positions with cells one by one, saving development costs, reducing runtime, minimizing memory usage, simplifying the design, and improving table reconstruction efficiency.

[0146] In one embodiment, step S203, which involves identifying the row and column structure of each corner point based on its position coordinates to obtain the position coordinates of each cell in the target table, includes:

[0147] Step S2031: Based on the position coordinates of the corner points, classify each corner point into rows and columns to obtain the row and column relationships of each corner point.

[0148] Step S2032: Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0149] Step S2033: Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex.

[0150] Step S2034: If there exists a distance less than a preset threshold, the corner point whose distance is less than the preset threshold shall be taken as the fourth vertex of the cell.

[0151] Step S2035: If there is no distance less than the preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex is re-designated as the second vertex of the cell; the ordinate of the re-determined second vertex and the abscissa of the third vertex are used as the coordinates of the predicted fourth vertex, and the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained.

[0152] Step S2036: Obtain the position coordinates of the cell based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0153] Step S2037: Select a new corner point as the first vertex of the next cell, and obtain the position coordinates of the next cell again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained.

[0154] This application embodiment determines the first vertex of a cell by selecting a corner point based on the row and column relationships of each corner point, designating the first corner point in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell, and designating the first corner point in the same column as the first vertex and located below the first vertex as the third vertex of the cell. The coordinates of the predicted fourth vertex are then calculated, and the distance between the coordinates of each corner point and the predicted coordinates of the fourth vertex is calculated to obtain the corresponding fourth vertex. This allows for the quick acquisition of the position coordinates of each cell.

[0155] Based on the above embodiments, the step S2031, which involves classifying each corner point into rows and columns according to their position coordinates to obtain a set of row and column corner points, includes:

[0156] Step S20311: Calculate the first distance between the ordinates of each corner point, and obtain the corner points whose first distance is less than a first preset threshold.

[0157] The first preset threshold can be set according to the required precision. Optionally, the first preset threshold can generally be set to 10 pixels.

[0158] Step S20312: Calculate the second distance between the abscissas of each corner point in the corner point set, obtain the corner points whose second distance is less than a second preset threshold, and obtain the row and column relationship of each corner point.

[0159] The second preset threshold can be set according to the required precision. Optionally, the first preset threshold can generally be set to 10 pixels.

[0160] In this embodiment, the first distance between the ordinates of each corner point is compared with a first preset threshold, and the second distance between the abscissas of each corner point is compared with a second preset threshold, so as to quickly and accurately obtain the row and column relationship of each corner point.

[0161] Example 3

[0162] The following are device embodiments corresponding to the method in Embodiment 1 of this application, which can be used to execute the content of the method in Embodiment 1 of this application. For details not disclosed in the device embodiments of this application, please refer to the content of the method in Embodiment 1 of this application.

[0163] Please see Figure 7 The table recognition and reconstruction apparatus 500 of this application includes:

[0164] The target content acquisition module 301 is used to acquire the target content selected by the stylus on the touch screen when the trigger operation of the selection button on the stylus is detected.

[0165] The first corner information acquisition module 302 is used to perform corner detection on the table image when the target content includes a table image, and obtain each corner of the table image and the position coordinates of the corner.

[0166] The first position coordinate acquisition module 303 is used to perform row and column structure recognition on each corner point according to the position coordinates of the corner points, and obtain the position coordinates of each cell of the table image;

[0167] The first image cropping module 304 is used to crop the image of each cell in the table image according to the position coordinates of each cell;

[0168] The first text content acquisition module 305 is used to perform text recognition on the images of each cell to obtain the text content in each cell.

[0169] The first table structure acquisition module 306 is used to reconstruct the table structure of the table image based on the position coordinates of each cell.

[0170] The first table reconstruction module 307 is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

[0171] The table image replacement module 308 is used to replace the table image with the reconstructed table on the touch screen.

[0172] This application embodiment acquires the target content circled on the touchscreen by the stylus when the trigger operation of the selection button on the stylus is detected; when the target content includes a table image, corner point detection is performed on the table image to obtain each corner point and its position coordinates; based on the position coordinates of the corner points, row and column structure recognition is performed on each corner point to obtain the position coordinates of each cell in the table image; based on the position coordinates of each cell, the table image is cropped to obtain the image of each cell; text recognition is performed on the image of each cell to obtain the text content within each cell; based on the position coordinates of each cell... The table structure of the table image is reconstructed; the text content in each cell is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table; on the touch screen, the reconstructed table replaces the table image. The target content can be selected directly by triggering the selection button on the stylus, avoiding the need for users to frequently switch recognition modes manually on the touch screen. The operation is convenient and fast, and it does not require a text detection model to obtain text content, a large amount of training data, or the construction of a complex deep learning text detection model. It also does not require matching text positions with cells one by one, saving development costs, reducing runtime and memory usage, simplifying the design, and improving table reconstruction efficiency.

[0173] In one embodiment, the target content acquisition module 301 includes:

[0174] The selection operation acquisition module is used to detect the trigger operation generated by pressing the selection button on the stylus, and to acquire the selection operation of the stylus on the touch screen when the selection button is in the pressed state.

[0175] The touch point connection module is used to connect the touch points detected during the selection operation when the stylus is lifted or the selection button is no longer pressed, so as to obtain the target content.

[0176] In one embodiment, the first position coordinate acquisition module 303 includes:

[0177] The first row-column relationship acquisition module is used to classify each corner point into rows and columns based on the position coordinates of the corner points, and obtain the row-column relationship of each corner point.

[0178] The first vertex selection module selects a corner point as the first vertex of a cell, selects the first corner point located in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and selects the first corner point located in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0179] The first distance calculation module is used to calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex, using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex.

[0180] The first vertex determination module is used to determine the fourth vertex of the cell if there is a corner point whose distance is less than a preset threshold.

[0181] The first vertex re-determination module is used to, if there is no distance less than a preset threshold, re-determine the second corner point located in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell; use the ordinate of the re-determined second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex, until the fourth vertex of the cell is obtained.

[0182] The first position coordinate determination module is used to obtain the position coordinates of the cell based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0183] The first cell determination module is used to select a new corner point as the first vertex of the next cell, and obtain the position coordinates of the next cell again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained.

[0184] In one embodiment, the first row-column relationship acquisition module includes:

[0185] The first corner point set acquisition module is used to calculate the first distance between the ordinates of each corner point and acquire the corner point set whose first distance is less than a first preset threshold.

[0186] The first row-column relationship determination module is used to calculate the second distance between the horizontal coordinates of each corner point in the corner point set, obtain the corner points whose second distance is less than a second preset threshold, and obtain the row-column relationship of each corner point.

[0187] Example 4

[0188] The following are device embodiments corresponding to the method in Embodiment 2 of this application, which can be used to execute the content of the method in Embodiment 2 of this application. For details not disclosed in the device embodiments of this application, please refer to the content of the method in Embodiment 2 of this application.

[0189] Please see Figure 8 The table recognition and reconstruction apparatus 400 of this application embodiment includes:

[0190] The table image acquisition module 401 is used to acquire the table image to be recognized;

[0191] The second corner information acquisition module 402 performs corner detection on the table image to obtain each corner of the table image and the position coordinates of the corner.

[0192] The second position coordinate acquisition module 403 is used to perform row and column structure recognition on each corner point according to the position coordinates of the corner points, and obtain the position coordinates of each cell of the table image;

[0193] The second image cropping module 404 is used to crop the image of each cell in the table image according to the position coordinates of each cell;

[0194] The second text content acquisition module 405 is used to perform text recognition on the images of each cell to obtain the text content in each cell.

[0195] The second table structure acquisition module 406 is used to reconstruct the table structure of the table image based on the position coordinates of each cell.

[0196] The second table reconstruction module 407 is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

[0197] This application embodiment acquires the target content circled on the touchscreen by the stylus when the trigger operation of the selection button on the stylus is detected; when the target content includes a table image, corner point detection is performed on the table image to obtain each corner point and its position coordinates; based on the position coordinates of the corner points, row and column structure recognition is performed on each corner point to obtain the position coordinates of each cell in the table image; based on the position coordinates of each cell, the table image is cropped to obtain the image of each cell; text recognition is performed on the image of each cell to obtain the text content within each cell; based on the position coordinates of each cell... The table structure of the table image is reconstructed; the text content in each cell is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table; on the touch screen, the reconstructed table replaces the table image. The target content can be selected directly by triggering the selection button on the stylus, avoiding the need for users to frequently switch recognition modes manually on the touch screen. The operation is convenient and fast, and it does not require a text detection model to obtain text content, a large amount of training data, or the construction of a complex deep learning text detection model. It also does not require matching text positions with cells one by one, saving development costs, reducing runtime and memory usage, simplifying the design, and improving table reconstruction efficiency.

[0198] In one embodiment, the target content acquisition module 301 includes:

[0199] The selection operation acquisition module is used to detect the trigger operation generated by pressing the selection button on the stylus, and to acquire the selection operation of the stylus on the touch screen when the selection button is in the pressed state.

[0200] The touch point connection module is used to connect the touch points detected during the selection operation when the stylus is lifted or the selection button is no longer pressed, so as to obtain the target content.

[0201] In one embodiment, the second position coordinate acquisition module 303 includes:

[0202] The second row and column relationship acquisition module is used to classify each corner point into rows and columns based on the position coordinates of the corner points, and obtain the row and column relationship of each corner point.

[0203] The second vertex selection module selects a corner point as the first vertex of a cell, selects the first corner point located in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and selects the first corner point located in the same column as the first vertex and below the first vertex as the third vertex of the cell.

[0204] The second distance calculation module is used to calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex, using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex.

[0205] The second vertex determination module is used to determine the fourth vertex of the cell if there is a corner point whose distance is less than a preset threshold.

[0206] The second vertex re-determination module is used to, if there is no distance less than a preset threshold, re-determine the second corner point located in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell; use the ordinate of the re-determined second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex, until the fourth vertex of the cell is obtained.

[0207] The second position coordinate determination module is used to obtain the position coordinates of the cell based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex.

[0208] The second cell determination module is used to select a new corner point as the first vertex of the next cell, and obtain the position coordinates of the next cell again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained.

[0209] In one embodiment, the second row-column relationship acquisition module includes:

[0210] The second corner point set acquisition module is used to calculate the first distance between the ordinates of each corner point and acquire the corner point set whose first distance is less than a first preset threshold.

[0211] The second row-column relationship determination module is used to calculate the second distance between the horizontal coordinates of each corner point in the corner point set, obtain the corner points whose second distance is less than a second preset threshold, and obtain the row-column relationship of each corner point.

[0212] Example 5

[0213] The following are embodiments of the device described in this application, which can be used to execute the methods described in Embodiment 1 and / or Embodiment 2 of this application. For details not disclosed in the embodiments of the device described in this application, please refer to the methods described in Embodiment 1 and / or Embodiment 2 of this application.

[0214] Please see Figure 9This application also provides an electronic device 500, which may specifically be a computer, mobile phone, tablet computer, interactive flat panel, etc. In an exemplary embodiment of this application, the electronic device 500 is an interactive flat panel, and the electronic device 500 may include: at least one processor 501, at least one memory 502, at least one display 503, at least one network interface 504, user interface 505, and at least one communication bus 506.

[0215] The user interface 505 is primarily used to provide an input interface for the user and to acquire user input data. Optionally, the user interface 505 may also include a standard wired interface or a wireless interface.

[0216] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0217] The communication bus 506 is used to enable communication between these components.

[0218] The processor 501 may include one or more processing cores. The processor 501 connects to various parts within the electronic device 500 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 502, and by calling data stored in the memory 502. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 501.

[0219] The memory 502 may include random access memory (RAM) or read-only memory. Optionally, the memory 502 may include a non-transitory computer-readable storage medium. The memory 502 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 502 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 9 As shown, the memory 502, which serves as a computer storage medium, may include an operating system, a network communication module, and a user.

[0220] The processor 501 can be used to call the application program of the table recognition and reconstruction method stored in the memory 502, and specifically execute the method steps of Embodiment 1 and / or Embodiment 2 shown above. For the specific execution process, please refer to the specific description shown in Embodiment 1 and / or Embodiment 2, which will not be repeated here.

[0221] Example 6

[0222] This application also provides a computer-readable storage medium storing a computer program thereon, the instructions of which are adapted to be loaded by a processor and executed by the method steps of Embodiment 1 and / or Embodiment 2 shown above. For the specific execution process, please refer to the detailed descriptions shown in Embodiment 1 and / or Embodiment 2, which will not be repeated here. The device containing the storage medium can be an electronic device such as a personal computer, laptop computer, smartphone, or tablet computer.

[0223] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0224] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0225] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function selected in one or more boxes.

[0226] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0227] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0228] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0229] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0230] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0231] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A table recognition and reconstruction method, characterized in that, Includes the following steps: When a trigger operation of the selection button on the stylus is detected, the target content selected by the stylus on the touch screen is obtained; the selection button is used to trigger the recognition and reconstruction of the selected content; When the target content includes a table image, corner detection is performed on the table image to obtain each corner of the table image and the position coordinates of the corners; Based on the position coordinates of the corner points, each corner point is classified into rows and columns to obtain the row and column relationships of each corner point; Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell. Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex. If there exists a corner point whose distance is less than a preset threshold, then the corner point whose distance is less than the preset threshold is taken as the fourth vertex of the cell. If there is no distance less than the preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex will be used as the second vertex of the cell again. Using the redefined ordinate of the second vertex and the x-coordinate of the third vertex as the coordinates of the predicted fourth vertex, the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained. The position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex. Select a new corner point as the first vertex of the next cell, and repeat the above steps to obtain the position coordinates of the next cell until all corner points are traversed and the position coordinates of each cell are obtained. In the table image, the images of each cell are obtained by cropping according to the position coordinates of each cell; Perform text recognition on the images of each cell to obtain the text content within each cell; Based on the position coordinates of each cell, reconstruct the table structure of the table image; The text content in each of the cells is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table; On the touchscreen, the reconstructed table replaces the table image.

2. The table recognition and reconstruction method according to claim 1, characterized in that: The step of classifying each corner point into rows and columns based on its position coordinates to obtain the row and column relationships of each corner point includes: Calculate the first distance between the ordinates of each corner point, and obtain a set of corner points whose first distance is less than a first preset threshold. Calculate the second distance between the x-coordinates of each corner point in the corner point set, obtain the corner points whose second distance is less than a second preset threshold, and obtain the row and column relationship of each corner point.

3. The table recognition and reconstruction method according to claim 1, characterized in that: The step of reconstructing the table structure of the table image based on the position coordinates of each cell includes: Based on the position coordinates of each cell, the individual cells are reconstructed; The reconstructed cells are stitched together to reconstruct the table structure of the table image.

4. The table recognition and reconstruction method according to any one of claims 1 to 3, characterized in that: When a trigger operation of the selection button on the stylus is detected, the step of obtaining the target content selected by the stylus on the touchscreen includes: When a trigger operation caused by pressing the selection button on the stylus is detected, the selection operation of the stylus on the touch screen is obtained when the selection button is pressed. When the stylus is lifted or the selection button is no longer pressed, the touch points detected during the selection process are connected to obtain the target content.

5. A table recognition and reconstruction method, characterized in that, Includes the following steps: When a trigger operation of the selection button on the stylus is detected, the table image to be recognized obtained by the stylus on the touch screen is acquired; the selection button is used to trigger the recognition and reconstruction of the selected table image to be recognized. Corner detection is performed on the table image to obtain each corner of the table image and the position coordinates of the corners; Based on the position coordinates of the corner points, each corner point is classified into rows and columns to obtain the row and column relationships of each corner point; Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell. Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex. If there exists a corner point whose distance is less than a preset threshold, then the corner point whose distance is less than the preset threshold is taken as the fourth vertex of the cell. If there is no distance less than the preset threshold, the second corner point located in the same row as the first vertex and to the right of the first vertex will be used as the second vertex of the cell again. Using the redefined ordinate of the second vertex and the x-coordinate of the third vertex as the coordinates of the predicted fourth vertex, the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained. The position coordinates of the cell are obtained based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex. Select a new corner point as the first vertex of the next cell, and repeat the above steps to obtain the position coordinates of the next cell until all corner points are traversed and the position coordinates of each cell are obtained. In the table image, the images of each cell are obtained by cropping according to the position coordinates of each cell; Perform text recognition on the images of each cell to obtain the text content within each cell; Based on the position coordinates of each cell, reconstruct the table structure of the table image; The text content in each of the cells is filled into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

6. A table recognition and reconstruction device, characterized in that, include: The target content acquisition module is used to acquire the target content selected by the stylus on the touch screen when the trigger operation of the selection button on the stylus is detected. The selection button is used to trigger the recognition and reconstruction of the selected content; The first corner information acquisition module is used to perform corner detection on the table image when the target content includes a table image, and obtain each corner of the table image and the position coordinates of the corner; The first position coordinate acquisition module is used to classify each corner point into rows and columns according to the position coordinates of the corner points, and obtain the row and column relationship of each corner point; select a corner point as the first vertex of a cell, take the first corner point located in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell, and take the first corner point located in the same column as the first vertex and located below the first vertex as the third vertex of the cell; Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex; if there is a corner point whose distance is less than a preset threshold, use the corner point whose distance is less than the preset threshold as the fourth vertex of the cell; if there is no corner point whose distance is less than the preset threshold, use the second corner point located in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell again. Using the redefined ordinate of the second vertex and the x-coordinate of the third vertex as the coordinates of the predicted fourth vertex, the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained. Based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex, the position coordinates of the cell are obtained; a new corner point is selected as the first vertex of the next cell, and the position coordinates of the next cell are obtained again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained; The first image cropping module is used to crop the image of each cell in the table image according to the position coordinates of each cell; The first text content acquisition module is used to perform text recognition on the images of each cell to obtain the text content in each cell. The first table structure acquisition module is used to reconstruct the table structure of the table image based on the position coordinates of each cell; The first table reconstruction module is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table. A table image replacement module is used to replace the table image with the reconstructed table on the touch screen.

7. A table recognition and reconstruction device, characterized in that, include: The table image acquisition module is used to acquire the table image to be recognized by the stylus when the trigger operation of the selection button on the stylus is detected; the selection button is used to trigger the recognition and reconstruction of the selected table image to be recognized. The second corner information acquisition module performs corner detection on the table image to obtain each corner of the table image and the position coordinates of the corners; The second position coordinate acquisition module is used to classify each corner point into rows and columns based on the position coordinates of the corner points, and obtain the row and column relationship of each corner point; Select a corner point as the first vertex of a cell, take the first corner point in the same row as the first vertex and to the right of the first vertex as the second vertex of the cell, and take the first corner point in the same column as the first vertex and below the first vertex as the third vertex of the cell. Using the ordinate of the second vertex and the abscissa of the third vertex as the coordinates of the predicted fourth vertex, calculate the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex; if there is a corner point whose distance is less than a preset threshold, use the corner point whose distance is less than the preset threshold as the fourth vertex of the cell; if there is no corner point whose distance is less than the preset threshold, use the second corner point located in the same row as the first vertex and located to the right of the first vertex as the second vertex of the cell again. Using the redefined ordinate of the second vertex and the x-coordinate of the third vertex as the coordinates of the predicted fourth vertex, the distance between the coordinates of each corner point and the coordinates of the predicted fourth vertex is calculated until the fourth vertex of the cell is obtained. Based on the coordinates of the first vertex, the second vertex, the third vertex, and the fourth vertex, the position coordinates of the cell are obtained; a new corner point is selected as the first vertex of the next cell, and the position coordinates of the next cell are obtained again according to the above steps, until all corner points are traversed and the position coordinates of each cell are obtained; The second image cropping module is used to crop the image of each cell in the table image according to the position coordinates of each cell; The second text content acquisition module is used to perform text recognition on the images of each cell to obtain the text content in each cell. The second table structure acquisition module is used to reconstruct the table structure of the table image based on the position coordinates of each cell; The second table reconstruction module is used to fill the text content in each of the cells into the corresponding cells in the reconstructed table structure to obtain the reconstructed table.

8. An interactive flat panel, comprising a processor and a memory; characterized in that, The memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the table recognition and reconstruction method as described in any one of claims 1 to 5.

Images