Table detection method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a table detection method and device, electronic equipment and storage medium, which relates to the technical field of artificial intelligence, specifically the technical field of deep learning, image processing and computer vision. The method comprises: performing feature extraction on a table image to be detected to obtain table feature information; performing split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a split line and a table vertex in the table image; correcting the table vertex in the table image according to the split line to obtain a corrected table vertex; matching the corrected table vertex and the split line, and obtaining cell information in the table image according to the matching relationship. The above technical solution can improve the accuracy of table detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, image processing and computer vision. Specifically, it relates to a table detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), artificial intelligence has been widely applied in the fields of computer vision technology, speech recognition technology, natural language processing technology, deep learning, big data processing technology, etc.

[0003] The text recognition technology based on artificial intelligence has also been widely applied in different scenarios. Many scenarios often have documents with tables, and the data organization method of the table structure has a lot of structured knowledge. Accurate extraction and restoration of table structured information can provide more direct decision-making information for business applications. Therefore, how to improve the performance of table detection is very important. SUMMARY

[0004] The present disclosure provides a table detection method and device, an electronic device and a storage medium.

[0005] According to an aspect of the present disclosure, a table detection method is provided, comprising:

[0006] performing feature extraction on a table image to be detected to obtain table feature information;

[0007] performing split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a split line and table vertices in the table image;

[0008] correcting the table vertices in the table image according to the split line in the table image to obtain corrected table vertices;

[0009] matching the corrected table vertices and the split line, and obtaining cell information in the table image according to the matching relationship.

[0010] According to another aspect of the present disclosure, a table detection device is provided, comprising:

[0011] a feature extraction module configured to perform feature extraction on a table image to be detected to obtain table feature information;

[0012] a table prediction module configured to perform split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a split line and table vertices in the table image;

[0013] a vertex correction module, configured to correct vertices of a table in the table image according to the split line in the table image, to obtain corrected vertices of the table;

[0014] a vertex matching module, configured to match the corrected vertices of the table and the split line, and obtain cell information in the table image according to a matching relationship.

[0015] According to another aspect of the present disclosure, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected to the at least one processor in communication; wherein,

[0018] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method provided by any of the embodiments of the present disclosure.

[0019] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the method provided by any of the embodiments of the present disclosure.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0022] Figure 1a is a flowchart of a table detection method according to an embodiment of the present disclosure;

[0023] Figure 1b is a schematic diagram of the principle of a table detection process according to an embodiment of the present disclosure;

[0024] Figure 2 is a flowchart of another table detection method according to an embodiment of the present disclosure;

[0025] Figure 3 is a flowchart of another table detection method according to an embodiment of the present disclosure;

[0026] Figure 4 is a structural schematic diagram of a table detection device according to an embodiment of the present disclosure;

[0027] Figure 5is a block diagram of an electronic device for implementing a table detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Figure 1a is a flowchart of a table detection method according to an embodiment of the present disclosure. The method is applicable to the case of content recognition of a table in an image. The method can be performed by a table detection apparatus, which can be implemented in software and / or hardware and can be integrated into an electronic device. As shown in Figure 1a , the table detection method of the present embodiment can include:

[0029] S101, feature extraction is performed on a table image to be detected to obtain table feature information;

[0030] S102, according to the table feature information, respectively, the table image is segmented and the vertex is predicted to obtain the segmentation line and the table vertex in the table image;

[0031] S103, according to the segmentation line in the table image, the table vertex in the table image is corrected to obtain the corrected table vertex;

[0032] S104, the corrected table vertex and the segmentation line are matched, and the cell information in the table image is obtained according to the matching relationship.

[0033] In the present embodiment, the table image to be detected can be any document image with a table, and the table type in the image can be wired, wireless, inclined or irregular, etc. It can be used for Optical Character Recognition (OCR) of document image data with tables in different scenes.

[0034] Figure 1b is a principle schematic diagram of a table detection process according to an embodiment of the present disclosure, referring to Figure 1b , the table image to be detected can be input into a feature extractor to obtain table feature information. The table feature information is input into a segmentation line prediction module and a vertex prediction module, respectively. The segmentation line in the table image is determined through the segmentation line prediction module, which can include row segmentation lines or column segmentation lines, and then the table image is cut into cells according to the row segmentation lines and column segmentation lines. The position of the table vertex of the table cell in the table image is predicted through the vertex prediction module to obtain the pixel coordinates of the table vertex. And the cells and the table vertex are fused through a fusion module to obtain the table recognition result. It should be noted that the position information of the table cell and the position information of the cell can include pixel coordinates and logical coordinates, and the logical coordinates are used to represent the relative position of the table cell and the cell, such as the number of rows and columns.

[0035] The feature extractor can be constructed based on a convolutional neural network (CNN) or a transformer network. The split line prediction module can be constructed based on a fully convolutional network (FCN) or a table splitting and merging method based on a table structure. The vertex prediction module can be constructed based on a regression method such as Faster-RCNN or Casade-RCNN, or a fully convolutional network (FCN).

[0036] In the case of vertex prediction for part of the table type, especially the wireless type table, the pixel coordinates of the table vertex are prone to inaccurate prediction, resulting in a large error in the recognition result of the table. Moreover, the logical coordinates of the table vertex are often obtained by post-processing, which also has the defect of low accuracy. Although the split line prediction method can accurately divide the table image and obtain the logical coordinates of the cells in the table image, for inclined tables or irregular tables, the logical coordinates of the cells also have a large error.

[0037] In the embodiments of the present disclosure, the table vertex in the table image is corrected according to the split line in the table image, that is, the position information of the table vertex is corrected by taking the split line as a reference, which can improve the accuracy of the position information of the table vertex. Moreover, the corrected table vertex and the split line are matched to obtain the matching relationship between the table vertex and the split line, that is, the dependency relationship between the table vertex and the split line, and the position information of the table vertex and the position information of the cell are fused based on the dependency relationship to obtain the logical coordinates of the cell and the pixel coordinates of the cell vertex as the cell information. By correcting the table vertex using the split line in the table image, the accuracy of the position information of the table vertex, the matching relationship between the table vertex and the split line, and the cell information can be improved.

[0038] The technical scheme provided by the embodiments of the present disclosure can improve the accuracy of the position information of the table vertex, the matching relationship between the table vertex and the split line, and the cell information by correcting the table vertex using the split line in the table image, and can accurately identify the table structure in a complex scene.

[0039] Figure 2 is a flowchart of another table detection method according to an embodiment of the present disclosure. Referring to FIG. 9, Figure 2 the table detection method of the present embodiment can include:

[0040] S201, performing feature extraction on a table image to be detected to obtain table feature information;

[0041] S202, performing split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a split line and a table vertex in the table image;

[0042] S203, extracting feature information of the split line from the table feature information according to the split line in the table image;

[0043] S204, extracting feature information of the table vertex from the table feature information according to the table vertex in the table image;

[0044] S205, correcting the feature information of the table vertex according to the feature information of the split line to obtain corrected feature information of the table vertex;

[0045] S206, determining pixel coordinates of the corrected table vertex according to the corrected feature information of the table vertex;

[0046] S207, matching the corrected table vertex and the split line, and obtaining cell information in the table image according to the matching relationship.

[0047] In the present embodiment, the feature information of the split line and the feature information of the table vertex can be determined based on the table feature information. For each split line in the table image, the average pixel coordinates of each point in the split line can be obtained, and the average pixel coordinates of the split line are used to extract the feature information of the split line from the table feature information; the feature information of each split line is combined to obtain a combination result. For each table vertex in the table image, the pixel coordinates of the table vertex are used to extract the feature information of the table vertex from the table feature information.

[0048] The combination result of the feature information of the split line can be taken as a reference basis to correct the feature information of the table vertex to obtain corrected feature information of the table vertex, and the pixel coordinates of the corrected table vertex can be determined according to the corrected feature information of the table vertex, for example, the corrected feature information of the table vertex can be input into the first regression unit to obtain the pixel coordinates of the corrected table vertex. By respectively determining the feature information of the split line and the feature information of the table vertex and correcting the feature information of the table vertex according to the feature information of the split line, the accuracy of the feature expression of the table vertex can be improved, thereby improving the accuracy of the pixel coordinates of the table vertex.

[0049] In an optional implementation, the correcting the feature information of the table vertex according to the feature information of the split line to obtain corrected feature information of the table vertex includes: taking the feature information of the split line as a key vector and a key value vector in an attention model respectively, and taking the feature information of the table vertex as a query vector in the attention model to obtain an output vector of the attention model; and obtaining the corrected feature information of the table vertex according to the output vector of the attention model.

[0050] In the embodiments of the present disclosure, the combination result of the feature information of the split line can be taken as a key vector and a key value vector of an attention model respectively, and the feature information of the table vertex can be taken as a query vector of the attention model to obtain an output vector of the attention model, and the corrected feature information of the table vertex can be obtained according to the output vector. The attention model can be based on a Transformer structure and pre-trained by supervised training based on a table image including split line annotation information and table vertex annotation information.

[0051] Specifically, taking M row split lines, N column split lines, K table vertices, and a single feature information of D dimensions as an example, the combination result of the split line feature information is (M+N)xD, and the feature information of the table vertex is KxD, where M, N, K, and D are natural numbers. The (M+N)xD split line feature information can be taken as a key vector and a key value vector of an attention model respectively, and the KxD feature information of the table vertex can be taken as a query vector of the attention model to obtain an output vector KxD' of the attention model, and the output vector D' is taken as the corrected feature information of the table vertex. In the correction process of the table vertex, the attention distribution of the feature information of the table vertex is learned based on the attention mechanism, and the corrected feature information of the table vertex is determined based on the attention distribution, which realizes the accuracy correction of the feature information of the table vertex, thereby improving the accuracy of the table recognition based on the corrected feature information of the table vertex.

[0052] In an optional implementation, after the feature information of the modified table vertex is obtained, the method further includes: determining a confidence of the modified table vertex according to the feature information of the modified table vertex; and filtering the modified table vertex according to the confidence.

[0053] After the feature information of the table vertex is modified, the confidence of the modified table vertex can be obtained again according to the feature information of the modified table vertex, for example, the feature information of the modified table vertex can be input into a second regression unit to obtain the confidence of the modified table vertex; and the table vertex with a confidence lower than a confidence threshold is filtered out, and only the table vertex with a confidence equal to or higher than the confidence threshold is reserved.

[0054] For example, in a case where the table has X table vertices, Y table vertices can be reserved according to the vertex prediction result, and after the Y table vertices are modified, the Y table vertices can be filtered according to the feature information of the modified table vertices. Y is a natural number greater than X. After the feature information of the table vertex is modified, the table vertex is further filtered according to the feature information of the modified table vertex, which can further improve the accuracy of table vertex prediction, thereby improving the accuracy of the fusion result between the subsequent table vertex and the division line.

[0055] The technical scheme provided by the embodiments of the present disclosure can further improve the accuracy of table structure recognition by determining the feature information of the division line and the feature information of the table vertex according to the table feature information, respectively; modifying the feature information of the table vertex according to the feature information of the division line as a reference; and further determining the pixel coordinates of the table vertex and filtering the table vertex according to the feature information of the modified table vertex.

[0056] Figure 3 is a flowchart of another table detection method provided by the embodiments of the present disclosure. The present embodiment is an optional implementation based on the above-mentioned embodiments. Referring to Figure 3 , the table detection method of the present embodiment can include:

[0057] S301, performing feature extraction on a table image to be detected to obtain table feature information;

[0058] S302, performing division line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a division line and a table vertex in the table image;

[0059] S303, modifying the table vertex in the table image according to the division line in the table image to obtain a modified table vertex;

[0060] S304, dividing the table image according to the division line to obtain logical coordinates of the cells and cell vertices in the table image;

[0061] S305, matching the corrected table vertex and the division line according to the feature information of the corrected table vertex and the feature information of the division line to obtain the logical coordinates of the corrected table vertex;

[0062] S306, determining the cell vertex matched with the corrected table vertex according to the logical coordinates of the cell vertex and the logical coordinates of the corrected table vertex, and taking the pixel coordinates of the corrected table vertex as the pixel coordinates of the matched cell vertex.

[0063] In the embodiments of the present disclosure, the table image can be divided into cells according to the row division line and the column division line in the table image to obtain the logical coordinates of the cells and cell vertices. The feature information of the division line and the feature information of the table vertex can be determined based on the table feature information. The feature information of the table vertex can be corrected according to the feature information of the division line to obtain the feature information of the corrected table vertex. It should be noted that after the cells are obtained by dividing the table image according to the division line, the cells can be merged according to the overlapping relationship between adjacent cells, feature information and the like, so as to improve the accuracy of cell division.

[0064] For each corrected table vertex, the feature information of the corrected table vertex can be matched with the feature information of the row division line and the feature information of the column division line respectively, and the logical coordinates of the corrected table vertex can be obtained according to the matching relationship. If the logical coordinates of any corrected table vertex are the same as the logical coordinates of any cell vertex, the pixel coordinates of the corrected table vertex are taken as the pixel coordinates of the cell vertex. By respectively determining the logical coordinates of the corrected table vertex and the logical coordinates of the cell vertex, the matching relationship between the corrected table vertex and the cell vertex is obtained based on the logical coordinates, and the pixel coordinates of the corrected table vertex are taken as the pixel coordinates of the matched cell vertex, which can improve the prediction accuracy of the cell vertex, especially for the inclined table and the irregular table, which can improve the accuracy of the position of the cell vertex.

[0065] In an optional implementation, the matching of the corrected table vertex and the division line according to the feature information of the corrected table vertex and the feature information of the division line to obtain the logical coordinates of the corrected table vertex comprises: determining the row division line matched with the corrected table vertex and the column division line matched with the corrected table vertex according to the similarity between the feature information of the corrected table vertex and the feature information of each division line; and determining the logical coordinates of the corrected table vertex according to the sequence number of the row division line and the sequence number of the column division line.

[0066] For each table vertex, a row similarity between the feature information of the table vertex and the feature information of each row split line can be determined, and a row split line to which the table vertex belongs can be selected according to the row similarity; a column similarity between the feature information of the table vertex and the feature information of each column split line can be determined, and a column split line to which the table vertex belongs can be selected according to the column similarity; and a logical coordinate of the table vertex can be obtained according to the row split line and the column split line to which the table vertex belongs. By respectively determining the similarity between the feature information of the corrected table vertex and the feature information of each split line, obtaining the row split line and the column split line to which the corrected table vertex belongs, and obtaining the logical coordinate of the table vertex according to the row split line and the column split line, the accuracy of the logical coordinate of the table vertex can be improved, thereby further improving the accuracy of the matching relationship between the table vertex and the cell vertex.

[0067] The technical scheme provided in the embodiments of the present disclosure can improve the accuracy of the pixel coordinates of the cells by respectively determining the logical coordinates of the cell vertex and the logical coordinates of the corrected table vertex, obtaining the matching relationship between the cell vertex and the corrected table vertex based on the logical coordinates of both, and taking the pixel coordinates of the corrected table vertex as the pixel coordinates of the corresponding cell vertex, thereby enabling accurate identification of the table structure in a complex scene.

[0068] Figure 4 FIG. 1 is a structural schematic diagram of a table detection device according to an embodiment of the present disclosure. The present embodiment is applicable to the case of content identification of a table in an image. The device can be implemented in the form of software and / or hardware. As shown in FIG. 1, the table detection device 400 of the present embodiment can include: Figure 4

[0069] A feature extraction module 410 is configured to perform feature extraction on a table image to be detected to obtain table feature information.

[0070] A table prediction module 420 is configured to perform split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain split lines and table vertices in the table image.

[0071] A vertex correction module 430 is configured to correct table vertices in the table image according to the split lines in the table image to obtain corrected table vertices.

[0072] A vertex matching module 440 is configured to match the corrected table vertices and the split lines, and obtain cell information in the table image according to the matching relationship.

[0073] In an optional implementation, the vertex correction module 430 includes: ​

[0074] a line feature extraction unit configured to extract, from the table feature information, feature information of a split line according to the split line in the table image;

[0075] a point feature extraction unit configured to extract, from the table feature information, feature information of a table vertex according to the table vertex in the table image;

[0076] a point feature correction unit configured to correct the feature information of the table vertex according to the feature information of the split line to obtain corrected feature information of the table vertex;

[0077] a point coordinate correction unit configured to determine pixel coordinates of the corrected table vertex according to the corrected feature information of the table vertex.

[0078] In an optional implementation, the point feature correction unit includes:

[0079] an attention subunit configured to take the feature information of the split line as a key vector and a key value vector in an attention model respectively, and take the feature information of the table vertex as a query vector in the attention model to obtain an output vector of the attention model;

[0080] a point feature correction subunit configured to obtain the corrected feature information of the table vertex according to the output vector of the attention model.

[0081] In an optional implementation, the vertex correction module 430 further includes a vertex filtering unit, and the vertex filtering unit includes:

[0082] a confidence subunit configured to determine a confidence of the corrected table vertex according to the corrected feature information of the table vertex;

[0083] a vertex correction subunit configured to filter the corrected table vertex according to the confidence.

[0084] In an optional implementation, the vertex matching module 440 includes:

[0085] a table division unit configured to divide the table image according to the split line to obtain logical coordinates of a cell and a cell vertex in the table image;

[0086] a table matching unit configured to match the corrected table vertex and the split line according to the corrected feature information of the table vertex and the feature information of the split line to obtain logical coordinates of the corrected table vertex;

[0087] The pixel coordinate unit is configured to determine a cell vertex matched with the corrected table vertex according to the logical coordinates of the cell vertex and the logical coordinates of the corrected table vertex, and determine the pixel coordinates of the corrected table vertex as the pixel coordinates of the matched cell vertex.

[0088] In an optional implementation, the table matching unit comprises:

[0089] The similarity sub-unit is configured to determine the row split line matched with the corrected table vertex and the column split line matched with the corrected table vertex according to the similarity between the feature information of the corrected table vertex and the feature information of each split line.

[0090] The table matching sub-unit is configured to determine the logical coordinates of the corrected table vertex according to the sequence number of the row split line and the sequence number of the column split line.

[0091] The technical solution of the embodiments of the present disclosure can improve the position accuracy of the table vertex by correcting the table vertex by using the split lines in the table image, can improve the accuracy of the position information of the cell by matching the corrected table vertex and the split line and fusing the position information of the corrected table vertex and the position information of the cell according to the matching relationship, and thus can accurately identify the table structure in a complex scene and is suitable for various OCR identification scenes such as finance, verification, medical treatment, insurance, office, and government affairs.

[0092] In the technical solution of the present disclosure, the acquisition, storage, and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0093] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0094] Figure 5 is a block diagram of an electronic device for implementing the table detection method of the embodiments of the present disclosure.

[0095] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0096] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0097] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the table detection method. For example, in some embodiments, the table detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the table detection method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the table detection method by any other suitable means (e.g., by means of firmware).

[0099] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0100] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0101] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0103] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0104] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0105] Artificial intelligence is a discipline that studies enabling computers to simulate some human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both hardware and software technologies. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning technology, big data processing technology, knowledge graph technology, etc. several major directions.

[0106] Cloud computing refers to a technology system that accesses an elastic scalable shared physical or virtual resource pool through a network, the resources can include servers, operating systems, networks, software, applications and storage devices, etc., and the resources can be deployed and managed in a demand self-service manner. Through cloud computing technology, efficient and powerful data processing capabilities can be provided for artificial intelligence, blockchain and other technical applications and model training.

[0107] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0108] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A table detection method, comprising: performing feature extraction on a table image to be detected to obtain table feature information; performing split line prediction and vertex prediction on the table image according to the table feature information, respectively, to obtain a split line and table vertices in the table image; correcting the table vertices in the table image according to the split line to obtain corrected table vertices; matching the corrected table vertices and the split line, and obtaining cell information in the table image according to a matching relationship; wherein the correcting the table vertices in the table image according to the split line to obtain corrected table vertices comprises: averaging pixel coordinates of each point in the split line in the table image to obtain average pixel coordinates of the split line; extracting feature information of the split line from the table feature information according to the average pixel coordinates; extracting feature information of the table vertices from the table feature information according to the table vertices in the table image; correcting the feature information of the table vertices according to the feature information of the split line to obtain feature information of the corrected table vertices; determining pixel coordinates of the corrected table vertices according to the feature information of the corrected table vertices.

2. The method of claim 1, wherein, the correcting the feature information of the table vertices according to the feature information of the split line to obtain feature information of the corrected table vertices comprises: taking the feature information of the split line as a key vector and a key value vector in an attention model, respectively, and taking the feature information of the table vertices as a query vector in the attention model to obtain an output vector of the attention model; obtaining the feature information of the corrected table vertices according to the output vector of the attention model.

3. The method of claim 1, after the obtaining the feature information of the corrected table vertices, further comprising: determining a confidence of the corrected table vertices according to the feature information of the corrected table vertices; filtering the corrected table vertices according to the confidence.

4. The method of any one of claims 1-3, wherein, the matching the corrected table vertices and the split line, and obtaining the cell information in the table image according to a matching relationship comprises: dividing the table image according to the split line to obtain logical coordinates of cells and cell vertices in the table image; matching the corrected table vertices and the split line according to the feature information of the corrected table vertices and the feature information of the split line to obtain logical coordinates of the corrected table vertices; determining a cell vertex matched by the corrected table vertices according to the logical coordinates of the cell vertices and the logical coordinates of the corrected table vertices, and taking pixel coordinates of the corrected table vertices as pixel coordinates of the matched cell vertex.

5. The method of claim 4, wherein, the matching the corrected table vertices and the split line according to the feature information of the corrected table vertices and the feature information of the split line to obtain logical coordinates of the corrected table vertices comprises: determine a row split line matched with the modified table vertex and a column split line matched with the modified table vertex according to similarity between feature information of the modified table vertex and feature information of each split line; determine a logical coordinate of the modified table vertex according to the sequence number of the row split line and the sequence number of the column split line.

6. A table detection device, comprising: a feature extraction module configured to perform feature extraction on a table image to be detected to obtain table feature information; a table prediction module configured to perform split line prediction and vertex prediction on the table image respectively according to the table feature information to obtain split lines and table vertices in the table image; a vertex correction module configured to correct table vertices in the table image according to the split lines to obtain modified table vertices; a vertex matching module configured to match the modified table vertices and the split lines, and obtain cell information in the table image according to a matching relationship; wherein the vertex correction module comprises: a line feature extraction unit configured to average pixel coordinates of each point in the split lines in the table image to obtain average pixel coordinates of the split lines, and extract feature information of the split lines from the table feature information according to the average pixel coordinates; a point feature extraction unit configured to extract feature information of the table vertices from the table feature information according to the table vertices in the table image; a point feature correction unit configured to correct the feature information of the table vertices according to the feature information of the split lines to obtain feature information of the modified table vertices; a point coordinate correction unit configured to determine pixel coordinates of the modified table vertices according to the feature information of the modified table vertices.

7. The apparatus of claim 6, wherein, The point feature correction unit comprises: an attention subunit configured to take the feature information of the split lines as a key vector and a key value vector in an attention model respectively, and take the feature information of the table vertices as a query vector in the attention model to obtain an output vector of the attention model; a point feature correction subunit configured to obtain the feature information of the modified table vertices according to the output vector of the attention model.

8. The device of claim 6, wherein the vertex correction module further comprises a vertex filtering unit, and the vertex filtering unit comprises: a confidence subunit configured to determine a confidence of the modified table vertices according to the feature information of the modified table vertices; a vertex correction subunit configured to filter the modified table vertices according to the confidence.

9. The apparatus of any one of claims 6-8, wherein, The vertex matching module comprises: a table division unit configured to divide the table image according to the split lines to obtain logical coordinates of cells and cell vertices in the table image; a table matching unit configured to match the modified table vertices and the split lines according to the feature information of the modified table vertices and the feature information of the split lines to obtain logical coordinates of the modified table vertices. The pixel coordinate unit is configured to determine a cell vertex matched with the corrected table vertex according to the logical coordinates of the cell vertex and the logical coordinates of the corrected table vertex, and determine the pixel coordinates of the corrected table vertex as the pixel coordinates of the matched cell vertex.

10. The apparatus of claim 9, wherein, The table matching unit comprises: a similarity sub-unit configured to determine a row split line matched with the corrected table vertex and a column split line matched with the corrected table vertex according to the similarity between the feature information of the corrected table vertex and the feature information of each split line; a table matching sub-unit configured to determine the logical coordinates of the corrected table vertex according to the sequence number of the row split line and the sequence number of the column split line. 11.An electronic device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method according to any one of claims 1-5. 13.A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-5.

Images