Table recognition method, device and computer-readable storage medium

Abstract

To provide a table recognition method and apparatus, and a computer-readable recording medium which can improve accuracy in table recognition.SOLUTION: A table recognition method according to an embodiment of the present invention has steps of discriminating a plurality of table elements including at least horizontal and vertical lines in a first table image to acquire a position of each of the table elements in the first table image, acquiring a second table image by subjecting the first table image to projection transformation based on the positions of the first table elements including the horizontal and vertical lines in the first table image so that distortion of the first table image should be canceled, and discriminating the second table image to acquire a cell frame region and character contents in the cell frame region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical fields of image processing and character recognition, and in particular to a table recognition method, apparatus and computer readable storage medium. [Background technology]

[0002] Tables are one of the data formats that can contain rich information in documents. However, most tables are difficult for computers to process data by scanning or photographing them, and the structured information is lost. Therefore, table identification has traditionally been an important technology for document decoding. Advanced table recognition technology allows digital products to accurately grasp the content information of documents and provide reliable services.

[0003] Table recognition aims to recognize the contents of a table based on an image of the table. Specifically, table recognition includes the recognition of the table structure (specifically, the positional area of ​​the cell frame) and the recognition of the text content within the cell frame. Conventional table recognition methods usually identify the table structure by target detection or row / column division methods, but the table image obtained by photographing or scanning may have distortion. It is usually difficult to obtain good results when recognizing such table images. Therefore, a table recognition technology that corrects the distortion of the table image to improve the accuracy of table recognition is required. Summary of the Invention [Problem to be solved by the invention]

[0004] SUMMARY OF THE PRESENTLY PREFERRED EMBODIMENTS The problem to be solved by the embodiments of the present invention is to provide a table recognition method, an apparatus and a computer-readable storage medium to improve the accuracy of table recognition. [Means for solving the problem]

[0005] At least one embodiment of one aspect of the present invention provides a table recognition method comprising: recognizing a plurality of table elements in a first table image and obtaining a position of each table element in the first table image, the plurality of table elements including at least horizontal and vertical lines; performing a projective transformation on the first table image based on the position of the first table element in the first table image to obtain a second table image, the first table elements including horizontal and vertical lines, and the projective transformation is used to remove distortion of the first table image; and recognizing the second table image to obtain a cell frame area and character content in the cell frame area.

[0006] Also, according to at least one embodiment of the present invention, recognizing a plurality of table elements in the first table image includes inputting the first table image into a pre-trained multi-task semantic segmentation model and obtaining a position in the first table image of each table element output from the multi-task semantic segmentation model.

[0007] Also, according to at least one embodiment of the present invention, the plurality of table elements further includes a whole table region and a text region.

[0008] Moreover, according to at least one embodiment of the present invention, the method further includes obtaining a plurality of sample table images and tagging results of table elements of each sample table image, and training the multi-task semantic segmentation model based on the plurality of sample table images and the tagging results, wherein the multi-task semantic segmentation model takes table images as input and table elements and their positions in the table images as output.

[0009] Furthermore, according to at least one embodiment of the present invention, performing projective transformation on the first table image based on the position of the first table element in the first table image to obtain a second table image includes performing fitting based on the position coordinates of points on the horizontal and vertical lines in the first table image to obtain a projective transformation relationship such that the vertical coordinates of points on the same horizontal line projected through the projective transformation are identical and the horizontal coordinates of points on the same vertical line are identical, and performing transformation on the first table image using the projective transformation relationship to obtain a second table image.

[0010] Also, according to at least one embodiment of the present invention, the horizontal lines include displayed horizontal lines and non-displayed horizontal lines, and the vertical lines include displayed vertical lines and non-displayed vertical lines.

[0011] Furthermore, according to at least one embodiment of the present invention, recognizing the second table image and obtaining cell frame areas and character content in the cell frame areas includes obtaining cell frame areas in the second table image based on positions of the horizontal and vertical lines in the second table image, and recognizing character content within each cell frame area in the second table image.

[0012] At least one embodiment of another aspect of the present invention provides a table recognition device including: an element recognition module that recognizes a plurality of table elements in a first table image and obtains a position of each table element in the first table image, the plurality of table elements including at least horizontal and vertical lines; a projection transformation module that performs projection transformation on the first table image based on the positions of the first table elements in the first table image to obtain a second table image, the first table elements including horizontal and vertical lines, and the projection transformation is used to remove distortion of the first table image; and a recognition processing module that recognizes the second table image and obtains a cell frame area and character content in the cell frame area.

[0013] Also, according to at least one embodiment of the present invention, the element recognition module inputs the first table image into a pre-trained multi-task semantic segmentation model and obtains the position in the first table image of each table element output from the multi-task semantic segmentation model.

[0014] Also, according to at least one embodiment of the present invention, the plurality of table elements further includes a whole table region and a text region.

[0015] In addition, according to at least one embodiment of the present invention, the method further includes a model training module that obtains a plurality of sample table images and tagging results of table elements of each sample table image, and trains the multi-task semantic segmentation model based on the plurality of sample table images and the tagging results, wherein the multi-task semantic segmentation model receives a table image as an input and outputs each table element in the table image and its position.

[0016] Furthermore, according to at least one embodiment of the present invention, the projection transformation module further performs fitting based on the position coordinates of points on the horizontal and vertical lines in the first table image to obtain a projection transformation relationship such that the vertical coordinates of points on the same horizontal line after projection transformation are identical and the horizontal coordinates of points on the same vertical line are identical, and performs transformation on the first table image using the projection transformation relationship to obtain a second table image.

[0017] Also, according to at least one embodiment of the present invention, the horizontal lines include displayed horizontal lines and non-displayed horizontal lines, and the vertical lines include displayed vertical lines and non-displayed vertical lines.

[0018] Furthermore, according to at least one embodiment of the present invention, the recognition processing module obtains cell border areas in the second table image based on the positions of the horizontal and vertical lines in the second table image, and recognizes the character content within each cell border area in the second table image.

[0019] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to realize the steps of the above-mentioned table recognition method. Effect of the Invention

[0020] Compared with the prior art, the table identification method, apparatus and computer-readable storage medium provided by the embodiments of the present invention have the effect of eliminating distortion in the table image, thereby improving the accuracy of table identification. [Brief description of the drawings]

[0021] The following briefly introduces the drawings necessary for describing the embodiments of the present invention, in order to more clearly describe the technical solutions of the embodiments of the present invention. To understand the embodiments of the present application, those skilled in the art can also obtain other drawings based on these drawings without exerting creative efforts. [Figure 1] FIG. 1 is a flow chart of a table identification method in an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating table element identification for a first table image in an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a configuration of a table identification device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing another configuration of a table identification device according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing still another configuration of the table identification device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] In order to clarify the problems, technical concepts and effects of the present invention, the present invention will be described in detail below in combination with drawings and specific embodiments. In the following description, specific details of the disclosed specific configurations and elements are only intended to help the reader understand the embodiments of the present invention correctly. Therefore, it is obvious to those skilled in the art that various modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. For the sake of clarity and conciseness, descriptions of known functions and structures are omitted.

[0023] References throughout the specification to "one embodiment" or "one embodiment" should be understood to mean that a particular feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of the present invention. Thus, appearances of "in one embodiment" or "in one embodiment" in the specification do not necessarily refer to the same embodiment. However, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0024] In each embodiment of the present invention, the magnitude of the numbers of the following processes does not mean the order of execution. It should be understood that the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0025] Conventional table recognition methods mainly include identifying table structure and identifying text content in a table. In table images, especially table images obtained by photographing or scanning, there is a large gap between table cells due to image deformation. Therefore, conventional table recognition methods cannot obtain accurate table structure and cannot ensure the accuracy of table recognition.

[0026] In response to the problem that conventional table recognition methods have difficulty in eliminating image deformations, an embodiment of the present invention provides a table recognition method based on a multi-task semantic segmentation model, which can eliminate deformations in table images and improve the accuracy of table recognition.

[0027] In the table recognition method according to the embodiment of the present invention, a table structure and text contents of a first table image including a table image are obtained by performing classification on the first table image. As shown in FIG. 1, the table recognition method includes steps 11 to 13.

[0028] In step 11, a plurality of table elements including at least horizontal and vertical lines in the first table image are identified, and the position of each table element in the first table image is obtained.

[0029] Here, table elements are elements that make up a table. In an embodiment of the present invention, table elements include horizontal and vertical lines that divide a table to form cells. In a table, horizontal and vertical lines may or may not be displayed. In the table shown in Table 1, horizontal and vertical lines are displayed for each cell. In the table shown in Table 2, some cells are divided by horizontal lines that are not displayed and vertical lines that are not displayed. Therefore, in an embodiment of the present invention, horizontal lines are classified into displayed horizontal lines and not displayed horizontal lines, and vertical lines are classified into displayed vertical lines and not displayed vertical lines.

[0030] [Table 1]

[0031] [Table 2] In an embodiment of the present invention, a multi-task semantic segmentation model is trained in advance to identify table elements. The multi-task semantic segmentation model includes a feature extraction layer and multiple output layers. Each output layer is associated with a table element. The feature extraction layer extracts features from the input table image. The feature extraction layer classifies pixels in the table image based on the features extracted for each output layer, classifies whether the pixel belongs to the table element corresponding to the output layer, and further outputs the area where the table element is located. That is, the multi-task is that multiple output layers of the model share the features extracted by one feature extraction layer to perform segmentation of different target areas. By introducing such a multi-task semantic segmentation model, the effect of the features extracted by the feature extraction layer can be enhanced, and each output layer can improve the identification accuracy for table elements. In the embodiment of the present invention, the multi-task semantic segmentation model includes, but is not limited to, models such as UNet and SegFormer.

[0032] Thus, in step 11, the first table image is input into a pre-trained multi-task semantic segmentation model, and the positions of various table elements in the first table image output by the multi-task semantic segmentation model are obtained.

[0033] In step 12, a projective transformation is performed on the first table image to obtain a second table image based on the positions of the first table elements, including the horizontal and vertical lines, in the first table image, so that distortion of the first table image is eliminated.

[0034] Here, in an embodiment of the present invention, fitting is performed based on the position coordinates of the points on the horizontal and vertical lines obtained in step 11 in the first table image so as to form a projective transformation relationship in which the vertical coordinates (e.g., y coordinates on the xy axis) of the points on the same horizontal line after projective transformation are the same, and the horizontal coordinates (e.g., x coordinates on the xy axis) of the points on the same vertical line are the same. In this way, based on the projective transformation relationship, transformation can be performed on the first table image, and the first table image is transformed into a second table image. By performing projective transformation in this way, the horizontal lines in the first table image are transformed into horizontal lines in the second table image, and the vertical coordinates of the points on the horizontal lines are all the same in the second table image, and similarly, the vertical lines in the first table image are transformed into vertical lines in the second table image, and the horizontal coordinates of the points on the vertical lines are all the same in the second table image, thereby eliminating deformation in the table image.

[0035] In step 12, it is assumed that points on the same horizontal line in the table image have the same ordinates, and points on the same vertical line have the same abscissas. Fitting of the projective transformation is to make the ordinates of points on the same horizontal line the same, and the abscissas of points on the same vertical line the same. In fitting of the projective transformation, based on the horizontal lines and vertical lines in the first table image obtained in step 11, the position coordinates of the points on each horizontal line in the first table image and the position coordinates of the points on each vertical line in the second table image are determined. Among them, a plurality of points that are equally spaced on the horizontal line for each horizontal line are selected to form a point group. The position coordinates of the points of the point group in the first table image are known. In addition, the horizontal coordinates of the points of the point group in the second table image and the horizontal coordinates in the first table image are the same, and the average value of the vertical coordinates in the first table image is set to the vertical coordinate in the second table image. Similarly, a plurality of points that are equally spaced on the vertical line for each vertical line are selected to form a point group. The position coordinates of the point group in the first table image are known. In addition, the average value of the horizontal coordinates of the points in the point group in the first table image is set as the horizontal coordinate in the second table image, and the vertical coordinate in the second table image is the same as the vertical coordinate in the first table image. By the above method, the position coordinates of the multiple points in the first table image and the position coordinates in the second table image are obtained. After that, fitting is performed based on the position coordinates of the multiple points in the first table image and the position coordinates in the second table image, and a projective transformation relationship between the first table image and the second table image is obtained.

[0036] In actual table element recognition, the horizontal and vertical lines identified in step 11 may have a line width greater than the width of one pixel point. In this case, the average value of the vertical coordinates of multiple points in the line width direction of the horizontal line may be set as the vertical coordinate of the point on the horizontal line in the first table image, and similarly, the average value of the horizontal coordinates of multiple points in the line width direction of the vertical line may be set as the horizontal coordinate of the point on the vertical line in the first table image.

[0037] In step 13, a second table image is identified to obtain cell border regions and text content within the cell border regions.

[0038] Here, based on the projection transformation relationship, the positions of the horizontal and vertical lines in the first table image in the second table image are obtained, and the cell border areas in the second table image are obtained. For example, the horizontal and vertical lines in the second table image are superimposed to obtain the cell border areas surrounded by the horizontal and vertical lines. The horizontal and vertical coordinate ranges of the cell border areas are used to restore the table structure. After that, the text content in each cell border area in the second table image is identified, for example, by an Optical Character Recognition (OCR) tool to identify the text content in the cell border areas. This completes the identification of the first table image.

[0039] In an embodiment of the present invention, the table element further includes a text area. The text area refers to an area corresponding to the text content in a cell. Thus, in step 11, the position of the text area in the first table image is obtained. Furthermore, in step 12, the position of the text area in the second table image is obtained based on the projective transformation relationship obtained. In step 13, character recognition is performed on the text area based on the position of the text area in the second table image to obtain the text content of the text area. Then, based on the position of the text area in the second table image, a cell frame area to which the text content belongs in the second table image is determined, thereby obtaining a table structure (cell frame) and its text content.

[0040] Through the above steps, an embodiment of the present invention uses a multi-task semantic segmentation model to identify table elements in a first table image, and based on the identified table elements, eliminates the distortion of the first table image through a projective transformation relationship to obtain a second table image, and then performs classification on the second table image, thereby improving the accuracy of table structure classification and further improving the effect of table classification.

[0041] The following describes training a multi-task semantic segmentation model according to an embodiment of the present invention.

[0042] First, a number of sample table images and tagging results of table elements for each sample table image are obtained. Each table element and its position in the sample table image are manually tagged. Then, a multi-task semantic segmentation model is trained based on the sample table images and the tagging results. The multi-task semantic segmentation model takes the table image as input and each table element in the table image and its position as output.

[0043] In addition, in order to improve the effect of identifying table structures by the model, the table elements according to the embodiment of the present invention include the entire table area. The entire table area refers to the area corresponding to the peripheral frame of the table. A multi-task semantic segmentation model is trained by introducing more table elements. By associating one task (one output layer) with each table element, the features extracted by one feature extraction layer are shared among multiple output layers to segment different target areas, thereby enhancing the effect of the features extracted by the feature extraction layer and improving the identification accuracy of table elements by each output layer. FIG. 2 shows the application of the multi-task semantic segmentation model according to the present invention to identify table elements on a first table image. Finally, the positions of the entire table area, horizontal lines, vertical lines and text areas in the first table image are obtained.

[0044] Based on this model structure, different types of table elements share visual information in image feature extraction. Multiple output layers can predict for each pixel whether the pixel belongs to a certain type of table element, and do not affect each other. For example, the intersection of a horizontal line and a vertical line belongs to both a horizontal line and a vertical line, and the prediction results of the horizontal line and the vertical line are output in different output layers to avoid label confusion.

[0045] Further, according to the above method, an embodiment of the present invention provides an apparatus for implementing the method. As shown in Fig. 3, an embodiment of the present invention provides a table recognition apparatus including an element recognition module 31 for identifying a plurality of table elements including at least horizontal and vertical lines in a first table image and acquiring the position of each table element in the first table image, a projection transformation module 32 for performing projection transformation on the first table image based on the positions in the first table image of the first table elements including horizontal and vertical lines so as to eliminate distortion in the first table image, thereby acquiring a second table image, and a recognition processing module 33 for identifying the second table image and acquiring a cell frame region and character content in the cell frame region.

[0046] In the embodiment of the present invention, the accuracy of table identification is improved by the above modules.

[0047] Also, preferably, the element recognition module 31 inputs the first table image into a pre-trained multi-task semantic segmentation model to obtain the position of each table element in the first table image output from the multi-task semantic segmentation model.

[0048] In addition, preferably, as shown in FIG. 4, the table identification apparatus of the embodiment of the present invention further includes a model training module 34, which obtains a plurality of sample table images and tagging results of table elements for each sample table image, and trains the multi-task semantic segmentation model according to the plurality of sample table images and the tagging results.

[0049] Also, preferably, the plurality of table elements further include an entire table region and a text region.

[0050] In addition, preferably, the projection transformation module further performs fitting based on the position coordinates of points on the horizontal and vertical lines in the first table image so as to create a projection transformation relationship in which the vertical coordinates of points on the same horizontal line after projection transformation are the same and the horizontal coordinates of points on the same vertical line are the same, and performs transformation on the first table image using the projection transformation relationship to obtain a second table image.

[0051] Also, preferably, the horizontal lines include displayed and non-displayed horizontal lines, and the vertical lines include displayed and non-displayed vertical lines.

[0052] Also, preferably, a cell border area in the second table image is obtained based on the positions of the horizontal and vertical lines in the second table image, and character content within the cell border area in the second table image is identified.

[0053] Furthermore, Fig. 5 is a block diagram showing a hardware configuration of a table recognition device according to an embodiment of the present invention. As shown in Fig. 5, the table recognition device 500 includes a processor 502 and a memory 504 in which computer program instructions are stored, and the processor 502 executes the computer program instructions to perform the following processes: identify a plurality of table elements including at least horizontal and vertical lines in a first table image and obtain the positions of each table element in the first table image; obtain a second table image by performing a projective transformation on the first table image based on the positions of the first table elements including horizontal and vertical lines in the first table image so that distortion of the first table image is eliminated; and identify the second table image to obtain a cell frame region and character contents within the cell frame region.

[0054] As shown in FIG. 5, the table recognition device 500 also includes a network interface 501 , an input device 503 , a hard disk 505 , and a display device 506 .

[0055] Each of the above interfaces is connected to each device via a bus architecture. The bus architecture is a bus and bridge that can include any number of interconnections. Specifically, various circuits of one or more central processing units (CPUs) represented by processor 502 and one or more memories represented by memory 504 are connected. In addition, various other circuits such as external devices, regulators, and power management circuits are connected by the bus architecture. In this way, these devices are connected so that they can communicate with each other by the bus architecture. In addition to a data bus, the bus architecture includes a power bus, a control bus, and a status signal bus. These are well known techniques in the field of the present invention, and detailed description will be omitted in the present text.

[0056] The network interface 501 is an interface that is connected to a network (eg, the Internet or a LAN), receives related data from the network, and stores it in the hard disk 505 .

[0057] The input device 503 is a means for receiving various commands input by a user, such as a preset threshold value or threshold setting information, and transmitting the commands to the processor 502 for execution. The input device 503 may be a keyboard or a click means (e.g., a mouse, a trackball, a touch board), or a touch panel or a touch screen.

[0058] The display device 506 is a means for displaying the results of the execution of instructions by the processor 502, for example the final recognition results obtained in post-processing.

[0059] The memory 504 is a memory for storing programs and data necessary for running an operating system (OS), as well as data such as intermediate results in the calculation process of the processor 502 .

[0060] In accordance with an embodiment of the present invention, memory 504 may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. Examples of non-volatile memory include ROM, PROM, EPROM, EEPROM, and flash memory. Examples of volatile memory include RAM, which may be used as an external cache. However, memory 504 used in the apparatus and methods described herein is not limited to these types of memory and may be any other suitable type of memory.

[0061] In some embodiments, memory 504 stores an OS 5041 and an application program 5042, which may be executable modules or data structures or submodules or extension modules thereof.

[0062] Among them, the OS 5041 includes various system programs, such as a framework layer, a core library layer, and a drive layer, and is used to realize various core operations and hardware-based tasks. The application programs 5042 include various application programs, such as a web browser, and are used to realize various application operations. The program for executing the method according to this embodiment is included in the application programs 5042.

[0063] The method according to the embodiment of the present invention is applied to or implemented by the processor 502. The processor 502 is an integrated circuit board capable of processing signals. Each step of the method is implemented by an integrated logic circuit in hardware or instructions in software form in the processor 502. The processor 502 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-flash programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing each method, step, and logic box disclosed in the embodiment of the present invention. The general-purpose processor may be a microprocessor or any general processor. Each step of the method according to the embodiment of the present invention may be implemented by being executed by a decoder in hardware, or may be implemented by a combination of hardware and software that can be implemented in the decoder. The software module is stored in a storage medium mature in the field, such as a random memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The processor 502 reads information from the memory 504, which includes a storage medium on which the software is stored, and configures the hardware to implement the steps of the above-mentioned method.

[0064] The above-described embodiments may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof, whereby, for hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units performing the functions of the present invention, or a combination thereof.

[0065] Regarding software implementation, the above technology is realized by modules (e.g., processes, functions, etc.) that realize the functions described above. The software code is stored in a memory and executed by a processor. The memory may be implemented inside or outside the processor.

[0066] Specifically, when the computer program is executed by the processor 502, a step is realized of inputting a first table image into a pre-trained multi-task semantic segmentation model and obtaining the position of each table element in the first table image output from the multi-task semantic segmentation model.

[0067] In particular, the computer program is executed by the processor 502 to further realize the steps of: obtaining a plurality of sample table images and tagging results of table elements for each sample table image; and training the multi-task semantic segmentation model based on the plurality of sample table images and the tagging results. The multi-task semantic segmentation model takes table images as input and table elements in the table images and their positions as output.

[0068] Preferably, the plurality of table elements further includes a whole table region and a text region.

[0069] Specifically, when the computer program is executed by the processor 502, the following steps are realized: fitting is performed based on the position coordinates of points on the horizontal and vertical lines in a first table image so as to form a projection transformation relationship in which the vertical coordinates of points on the same horizontal line after projection transformation are the same, and the horizontal coordinates of points on the same vertical line are the same; and a transformation is performed on the first table image using the projection transformation relationship to obtain a second table image.

[0070] Specifically, the horizontal lines include displayed and non-displayed horizontal lines, and the vertical lines include displayed and non-displayed vertical lines.

[0071] In particular, when the computer program is executed by the processor 502, the steps of obtaining a cell border area in the second table image based on the positions of the horizontal and vertical lines in the second table image; and identifying character content within the cell border area in the second table image are realized.

[0072] Those skilled in the art of the present invention can easily imagine that the units and algorithm steps of each example described in the above disclosed embodiments can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by either hardware or software depends on the specific application and design constraints of the invention. Those skilled in the art can realize the above functions in a way that suits the specific application, but it should not go beyond the scope of the present invention.

[0073] In addition, for convenience and conciseness of explanation, detailed descriptions of the specific operating processes of the above systems, devices and units will be omitted, since it is clear to those skilled in the art that the corresponding processes in the above embodiments can be referred to.

[0074] It is easily conceivable that the method and apparatus disclosed in the multiple embodiments of the present invention can be realized in other forms. For example, the above-described apparatus is merely schematic. For example, the division of the units is merely one example of logical function allocation, and a different division method may be adopted when actually realizing the present invention. For example, a plurality of units or modules may be combined or integrated into another system, or some functions may be omitted or not executed. Note that the mutual connections, direct connections, or communicative connections shown or disclosed above are connections via interfaces. Indirect connections or communicative connections between devices or units may be electrical, mechanical, or other forms of connections.

[0075] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, i.e., they may be in the same location or distributed across multiple network units. Depending on actual needs, some or all of the units may be selected to achieve the objectives of the embodiments of the present invention.

[0076] Note that each functional unit according to the embodiment of the present invention may be integrated into one processing unit, may be physically independent, or may be integrated into one unit consisting of two or more functional units.

[0077] When the above-mentioned functions are realized in the form of a software functional unit and sold or used as an independent product, the functions can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical concept of the present invention or a part that contributes to the prior art or a part of the technical concept can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions, and causes a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The above-mentioned storage medium includes various media that can store program codes, such as a USB memory, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0078] The above description is not intended to limit the scope of the present invention, since it is not intended to limit the scope of the present invention. Any modifications or replacements that are easily conceived by those skilled in the art within the scope of the present invention are included in the scope of the present invention. Therefore, the scope of the present invention should be determined based on the claims.

Claims

1. A table recognition method executed by a table recognition device, comprising: Recognizing a plurality of table elements in a first table image and obtaining a position of each table element in the first table image, the plurality of table elements including at least a horizontal line and a vertical line; performing a projective transformation on the first table image based on a position in the first table image of a first table element of the plurality of table elements, the first table element including horizontal and vertical lines, to obtain a second table image, the projective transformation being used to remove distortion of the first table image; recognizing the second table image to obtain a cell frame area and text content in the cell frame area.

2. Recognizing a plurality of table elements in the first table image includes:

2. The method of claim 1, further comprising: inputting the first table image into a pre-trained multi-task semantic segmentation model; and obtaining a position in the first table image of each table element output from the multi-task semantic segmentation model.

3. The method of claim 2 , wherein the plurality of table elements further includes an entire table region and a text region.

4. obtaining a plurality of sample table images and tagging results for table elements of each sample table image; training the multi-task semantic segmentation model based on the plurality of sample table images and the tagging results; 4. The table recognition method according to claim 2, wherein the multi-task semantic segmentation model receives a table image as an input and outputs table elements and their positions in the table image.

5. performing a projective transformation on the first table image based on a position of the first table element in the first table image to obtain a second table image, performing fitting based on position coordinates of points on the horizontal line and the vertical line in the first table image to obtain a projective transformation relationship such that points on the same horizontal line after projective transformation have the same vertical coordinates and points on the same vertical line have the same horizontal coordinates; 2. The method of claim 1, further comprising: performing a transformation on the first table image according to the projective transformation relationship to obtain a second table image.

6. The horizontal lines include displayed and undisplayed horizontal lines, 6. The method of claim 5, wherein the vertical lines include displayed vertical lines and non-displayed vertical lines.

7. Recognizing the second table image to obtain a cell frame region and character content in the cell frame region includes: obtaining a cell border area in the second table image based on positions of the horizontal and vertical lines in the second table image; 7. The method according to claim 5, further comprising: recognizing character content within each cell frame area in the second table image.

8. an element recognition module for recognizing a plurality of table elements in a first table image and obtaining a position of each table element in the first table image, the plurality of table elements including at least a horizontal line and a vertical line; a projection transformation module for performing a projection transformation on a first table image based on a position of a first table element of the plurality of table elements in the first table image to obtain a second table image, the first table element including horizontal and vertical lines, and the projection transformation being used to remove distortion of the first table image; a recognition processing module that recognizes the second table image and acquires a cell frame area and character content in the cell frame area.

9. The table recognition device of claim 8, wherein the element recognition module inputs the first table image into a pre-trained multi-task semantic segmentation model and obtains a position in the first table image of each table element output from the multi-task semantic segmentation model.

10. The table recognition apparatus of claim 9 , wherein the plurality of table elements further includes an entire table region and a text region.

11. a model training module for obtaining a plurality of sample table images and a tagging result of table elements of each sample table image, and training the multi-task semantic segmentation model based on the plurality of sample table images and the tagging results; 11. The table recognition device according to claim 9, wherein the multi-task semantic segmentation model receives a table image as an input and outputs each table element in the table image and its position.

12. The projection transformation module further comprises: performing fitting based on position coordinates of points on the horizontal line and the vertical line in the first table image to obtain a projective transformation relationship such that the vertical coordinates of points on the same horizontal line after projective transformation are the same and the horizontal coordinates of points on the same vertical line are the same; 9. The table recognition device according to claim 8, further comprising: a transform for the first table image based on the projective transformation relationship to obtain a second table image.

13. The horizontal lines include displayed and undisplayed horizontal lines, The table recognition device according to claim 12 , wherein the vertical lines include displayed vertical lines and non-displayed vertical lines.

14. The recognition processing module includes: obtaining a cell border area in the second table image based on positions of the horizontal and vertical lines in the second table image; 14. The table recognition device according to claim 12, further comprising a step of recognizing character content within each cell frame area in the second table image.

15. 1. A computer-readable storage medium, comprising: A computer program is stored in the storage medium, 7. A storage medium, comprising: a processor that executes the computer program; and a storage medium that realizes the steps of the table recognition method according to any one of claims 1 to 3 and 5 to 6.

Images