Text table recognition method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a text table recognition method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a to-be-processed image, and determining a table region and a plurality of text regions from the to-be-processed image through a region segmentation model; for each text region, converting the image content of the text region into text data, and taking the coordinate position of the text region in the image coordinate system as the first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the to-be-processed image; determining a plurality of cell regions in the table region, and determining a second coordinate position of each cell region in the image coordinate system; based on the first coordinate position and the second coordinate position, determining target data corresponding to each cell region from the plurality of pieces of text data; and obtaining table data based on each cell and the target data corresponding to each cell. By adopting the method, the accuracy of table recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text table recognition method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] With the widespread application of digital images, the positioning of table areas in images and the recognition of table texts are becoming increasingly important in the fields of image processing and computer vision. At present, some systems have used optical character recognition (OCR) technology and table recognition technology to extract table areas in images and recognize table texts, but this solution can only handle tables in simple situations. For tables in complex situations, such as tables with complex background images and tables covered with occluded images, in this case, the table area will overlap with the text area in the background image, or the table area will overlap with the text area in the occluded image, resulting in low table recognition accuracy. Summary of the invention

[0003] Based on this, it is necessary to provide a text table recognition method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of table recognition in order to solve the above technical problems.

[0004] In a first aspect, the present application provides a method for identifying a table in a text, comprising:

[0005] Acquire an image to be processed, and determine a table area and a plurality of text areas from the image to be processed by a region segmentation model; the region segmentation model is obtained by training a first image sample with a text area mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text area mark is obtained by marking the text area in the image body;

[0006] For each text area, convert the image content of the text area into text data, and use the coordinate position of the text area in the image coordinate system as the first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed;

[0007] Determine a plurality of cell regions in the table region, and determine a second coordinate position of each cell region in the image coordinate system;

[0008] Based on the first coordinate position and the second coordinate position, determining target data corresponding to each cell area from a plurality of text data;

[0009] Based on each cell and the target data corresponding to each cell, table data is obtained.

[0010] In one embodiment, after converting the image content of the text area into text data, the method further comprises:

[0011] For each text data, input the current text data into a deep learning model, learn the semantic features of the current text data through the deep learning model, and determine the key data in the text data based on the semantic features;

[0012] The first coordinate position of each text data is used as the key coordinate position of the corresponding key data;

[0013] Correspondingly, determining the target data corresponding to each cell area from the plurality of text data based on the first coordinate position and the second coordinate position includes:

[0014] Based on the key coordinate position and the second coordinate position, target key data corresponding to each cell area is determined from a plurality of key data.

[0015] In one embodiment, determining target key data corresponding to each cell area from a plurality of key data based on the key coordinate position and the second coordinate position includes:

[0016] For each cell area, the matching degree between the second coordinate position corresponding to the current cell area and multiple key coordinate positions is calculated; when the matching degree between the second coordinate position corresponding to the current cell area and the key coordinate position is greater than a preset matching degree threshold, the key data corresponding to the key coordinate position is used as the target key data corresponding to the cell area.

[0017] In one embodiment, determining a plurality of cell regions in the table region includes:

[0018] Inputting the table area into a deep learning model, and determining contour-related information of each cell in the table area through the deep learning model; the contour-related information includes at least one of a cell contour or a text field contour;

[0019] Based on the outline-related information, a plurality of cell areas are determined from the table area.

[0020] In one embodiment, the training process of the deep learning model includes:

[0021] Acquire a plurality of second image samples, each of which includes a table area sample, and the table area sample has a contour-related mark related to a cell contour;

[0022] Inputting each second image sample into an initial model, and determining contour-related information of each cell in the table area sample through the initial model;

[0023] Based on the contour-related information of each cell in the table area sample and the corresponding contour-related marks, the parameters of the initial model are adjusted, and training is continued based on the initial model with adjusted parameters until the predicted contour-related information of each cell in the table area sample and the corresponding contour-related marks are within a preset difference range, thereby obtaining a deep learning model.

[0024] In a second aspect, the present application provides a table data comparison method, the method comprising:

[0025] Taking the initial image as the image to be processed, executing the above-mentioned text table recognition method to obtain the table data corresponding to the initial image;

[0026] The updated image is used as the image to be processed, and the above-mentioned text table recognition method is executed to obtain table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image;

[0027] The table data corresponding to the initial text image and the table data corresponding to the updated text image are compared to determine data update information of the cell.

[0028] In a third aspect, the present application further provides a text table recognition device, the device comprising:

[0029] An acquisition module is used to acquire an image to be processed, and determine a table area and a plurality of text areas from the image to be processed by using a region segmentation model; the region segmentation model is obtained by training a first image sample with a text area mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text area mark is obtained by marking a text area in the image body;

[0030] A conversion module, for each text region, converting the image content of the text region into text data, and using the coordinate position of the text region in an image coordinate system as a first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed;

[0031] A first determining module, used for determining a plurality of cell regions in the table region, and determining a second coordinate position of each cell region in the image coordinate system;

[0032] A second determining module, configured to determine target data corresponding to each cell area from a plurality of text data based on the first coordinate position and the second coordinate position;

[0033] The table data output module is used to obtain the table data based on each cell and the target data corresponding to each cell.

[0034] In a third aspect, the present application further provides a table data comparison device, the device comprising:

[0035] A first input module, used for inputting an initial image as the image to be processed into the text table recognition device of claim 7 for processing, so as to obtain table data corresponding to the initial image;

[0036] A second input module is used to input the updated image as the image to be processed into the text table recognition device of claim 7 for processing to obtain table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image;

[0037] The comparison module is used to compare the table data corresponding to the initial text image and the table data corresponding to the updated text image to determine the data update information of the cell.

[0038] In a fifth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of each embodiment when executing the computer program.

[0039] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method of each embodiment when executed by a processor.

[0040] In a seventh aspect, the present application further provides a computer program product, including a computer program, which implements the steps of the methods of each embodiment when executed by a processor.

[0041] The above-mentioned text table recognition method, device, computer equipment, computer-readable storage medium and computer program product, the region segmentation model is obtained by training a first image sample with a text region mark, the first image sample is an image formed by an image body and an overlapping image of the image body, the text region mark is obtained by marking the text region in the image body, so that the image to be processed is obtained, and the table region and multiple text regions are determined from the image to be processed through the region segmentation model, and the text in the overlapping image in the image to be processed will not be used as the text region, only the text in the image body in the image to be processed will be used as the text region, so that for each text region, after the image content of the text region is converted into text data, the coordinate position of the text region in the image coordinate system is used as the first A coordinate position is determined, and then multiple cell areas in the table area are determined, and a second coordinate position of each cell area in the image coordinate system is determined. Based on the first coordinate position and the second coordinate position, target data corresponding to each cell area is determined from multiple text data, so that each cell area in the table and the target data corresponding to each cell area can be found, so that the cell area of ​​the table area and the corresponding target data can be identified from the image to be processed to obtain the table data. Compared with the existing scheme, for the case where the table condition in the text is complex, the text in the overlapping image in the image to be processed will not be regarded as the text area, and only the text in the image body in the image to be processed will be regarded as the text area, so that the table data can be identified from the image to be processed, thereby improving the recognition accuracy of the table data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a flow chart of a text table recognition method in one embodiment;

[0044] Figure 2 A schematic diagram of a process flow of a table data comparison method in one embodiment;

[0045] Figure 3 It is a structural block diagram of a text table recognition device in one embodiment;

[0046] Figure 4 is a structural block diagram of a table data comparison device in one embodiment;

[0047] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The present application can be applied to the scenario of electronic contract signing. In the field of smart contracts, the convenience and accuracy of electronic contract signing are of key significance to the effective execution of the contract. However, as mentioned above, for tables in complex situations, for example, tables with complex background images and tables covered with occluded images, in this case, the table area will overlap with the text area in the background image, or the table area will overlap with the text area in the occluded image. There is a problem of low table recognition accuracy. At the same time, it is impossible to recognize the content of tables in complex situations, resulting in lack of generalization ability and poor adaptability.

[0050] To solve these problems, in one embodiment, Figure 1 As shown, a text table recognition method is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0051] Step 102: obtaining an image to be processed, and determining a table region and a plurality of text regions from the image to be processed by using a region segmentation model; the region segmentation model is obtained by training a first image sample with a text region marker, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text region marker is obtained by marking a text region in the image body;

[0052] In this embodiment, the image to be processed may be an image of an electronic contract.

[0053] In the region segmentation model training stage, for each first image sample, only the text area of ​​the image body in the first image sample is marked, and the text area of ​​the overlapping image (such as a watermark image or a stamp overlapping on the image body) in the image sample is not marked. The marked first image samples are input into the initial region segmentation model, and the initial region segmentation model is trained to obtain the region segmentation model.

[0054] In the application stage of the region segmentation model, after the image to be processed that is blocked or has a watermark background is input into the region segmentation model, the model region segmentation model will only mark the text in the image body of the image to be processed as the text region, but will not mark the text in the overlapping image as the text region, so that multiple text regions can be determined from the image to be processed;

[0055] The table outline in the image to be processed may also be identified through the region segmentation model, and the table region may be obtained based on the table outline.

[0056] Step 104: for each text region, convert the image content of the text region into text data, and use the coordinate position of the text region in the image coordinate system as the first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed;

[0057] For each identified text area, converting the image content in the text area into text data, for example, for an image containing text, identifying the text to obtain text data;

[0058] An image coordinate system can be constructed for the image to be processed, and it can be determined that the text area in the image to be processed has a corresponding coordinate position in the image coordinate system. The text area is generally a rectangle, and the coordinate position of the text area in the image coordinate system refers to the coordinates of the four vertices of the rectangle;

[0059] Then, for each text region, the coordinate position corresponding to the current text region in the image coordinate system is used as the first coordinate position of the corresponding text data.

[0060] Step 106: determine multiple cell areas in the table area, and determine the second coordinate position of each cell area in the image coordinate system;

[0061] Step 108: determining target data corresponding to each cell area from the plurality of text data based on the first coordinate position and the second coordinate position;

[0062] For each cell area, the matching degree between the second coordinate position of the current cell area and each first coordinate position is calculated, and the matching degree greater than the first matching degree threshold is determined from the calculated matching degrees, and the first coordinate position corresponding to the matching degree greater than the first matching degree threshold is used as the first coordinate position matching the second coordinate position, and the text data corresponding to the matched first coordinate position is used as the target data corresponding to the current cell area, that is, the target data is the text data located in the corresponding cell area. In this way, the target data corresponding to each cell area can be determined.

[0063] Step 110: Obtain table data based on each cell and the target data corresponding to each cell.

[0064] In this embodiment, the region segmentation model is obtained by training a first image sample with a text region mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text region mark is obtained by marking the text region in the image body, so that the image to be processed is obtained, and the table region and multiple text regions are determined from the image to be processed through the region segmentation model, and the text in the overlapping image in the image to be processed will not be marked as a text region, and only the text in the image body in the image to be processed will be marked as a text region, so that for each text region, after the image content of the text region is converted into text data, the coordinate position of the text region in the image coordinate system is used as the first coordinate position of the text data, and then the text in the table region is determined. Multiple cell areas of the table are identified, and the second coordinate position of each cell area in the image coordinate system is determined. Based on the first coordinate position and the second coordinate position, the target data corresponding to each cell area is determined from the multiple text data. In this way, each cell area in the table and the target data corresponding to each cell area can be found, so that the cell area of ​​the table area and the corresponding target data can be identified from the image to be processed to obtain the table data. Compared with the existing scheme, for the case where the table condition in the text is complex, the text in the overlapping image in the image to be processed will not be regarded as the text area, and only the text in the image body in the image to be processed will be regarded as the text area, so that the table data can be identified from the image to be processed, thereby improving the recognition accuracy of the table data.

[0065] At the same time, the present application can identify table contents under complex conditions, thus improving the generalization and adaptability of table recognition.

[0066] In one embodiment, after acquiring the image to be processed, and before determining the table region and the plurality of text regions from the image to be processed by using a region segmentation model, the method further comprises: preprocessing the image to be processed;

[0067] The preprocessing of the image to be processed specifically includes: performing denoising on the image to be processed, adjusting the image brightness and contrast. The preprocessing of the image to be processed can improve the accuracy of subsequent processing.

[0068] In one embodiment, the method further comprises:

[0069] Visual features are extracted from the image to be processed through a deep learning model, including features such as the font, color, size, and layout features and arrangement features of the table in the image. Then, based on the visual features, computer vision techniques such as object detection and segmentation algorithms, or other visual feature extraction methods are used to identify the illustration area, header area, and footer area in the image to be processed.

[0070] Through the deep learning model, accurate positioning of text areas, table areas, illustration areas, header areas, and footer areas in the processed images can be achieved, adapting to contract images of different formats and layouts.

[0071] In one of the embodiments, after converting the image content of the text area into text data, the method also includes: for each text data, inputting the current text data into a deep learning model, learning the semantic features of the current text data through the deep learning model, and determining the key data in the text data based on the semantic features; using the first coordinate position of each text data as the key coordinate position of the corresponding key data; accordingly, based on the first coordinate position and the second coordinate position, determining the target data corresponding to each cell area from multiple text data, including: based on the key coordinate position and the second coordinate position, determining the target key data corresponding to each cell area from multiple key data.

[0072] For each text data, natural language processing technology, including text analysis and semantic understanding, is used to identify the key information in the text data. Specifically:

[0073] Perform OCR recognition to convert text data into processable text data, input the text data into the deep learning model (or select other text recognition models to replace the deep learning model as needed, and input the text data into other text recognition models), learn the semantic features of the text data through the deep learning model, and then determine the key data in the text data based on the learned semantic features.

[0074] In one of the embodiments, based on the key coordinate position and the second coordinate position, target key data corresponding to each cell area is determined from multiple key data, including: for each cell area, calculating the matching degree between the second coordinate position corresponding to the current cell area and multiple key coordinate positions respectively; when the matching degree between the second coordinate position corresponding to the current cell area and the key coordinate position is greater than a preset matching degree threshold, the key data corresponding to the key coordinate position is used as the target key data corresponding to the cell area.

[0075] For each cell area, the matching degree between the second coordinate position corresponding to the current cell area and multiple key coordinate positions is calculated to obtain multiple matching degrees, and a matching degree greater than a preset second matching degree threshold is determined from the multiple matching degrees, and the key coordinate position corresponding to the matching degree greater than the second matching degree threshold is determined, and the key data corresponding to the key coordinate position is used as the target key data corresponding to the cell area.

[0076] In this embodiment, natural language processing technology can be used to automatically identify key data in the text data corresponding to the cell, and finally obtain table data, thereby reducing the workload of users manually entering key data. As described below, the present application can determine the table update content based on the table area before the update and the table area after the update to sign the electronic contract. Therefore, the table data of the present application includes the key data automatically identified, and there is no need to wait for the user to manually enter the key data, thereby improving the signing efficiency when signing the electronic contract.

[0077] For the image to be processed, the deep learning model (or other modules) can be used to segment the located table area, and the table recognition processing technology is used to identify the line contours of the table. After identification, the outlines are reorganized to form a cell sequence, and the cells are serialized and sorted to return a structured table. Specifically:

[0078] In one embodiment, determining multiple cell areas in a table area includes: inputting the table area into a deep learning model, and determining contour-related information of each cell in the table area through the deep learning model; the contour-related information includes at least one of a cell contour or a text field contour; and determining multiple cell areas from the table area based on the contour-related information.

[0079] If the cells in the table area are cells with lines, the application scheme is specifically as follows:

[0080] In the model training stage, for cells with lines, the cell contours are pre-marked, the deep learning model learns the features of the marked cell contours, and the deep learning model is trained based on the learned features; in the model application stage, the table area is input into the deep learning model, and the contour-related information of each cell in the table area is determined by the deep learning model. The contour-related information can be the contour lines of the cells, so that based on the contour lines of the cells, multiple cell areas can be determined from the table area.

[0081] If the cells in the table area are cells without lines, or if some of the cell lines in the table area are missing, or if some of the cell lines in the table area are covered and cannot be seen clearly, for the table areas in these situations, the application scheme is:

[0082] In the model training stage, the text domain outline of the text in the cell is pre-marked, and the deep learning model learns the features of the marked text domain outline, and the deep learning model is trained based on the learned features; in the model application stage, the table area is input into the deep learning model, and the outline-related information of each cell in the table area is determined by the deep learning model. The outline-related information can be the text domain outline corresponding to the cell, so that the cell outline of the table area is restored based on the text domain outline corresponding to the cell, and multiple cell areas are determined from the table area based on the cell outline.

[0083] If the colors of cells in the table area are diverse, for example, some cells are red, some are black, and some are blue, the present application also includes:

[0084] In the model training stage, the table area samples include cells of various colors. The color marks and cell outlines of the cells of various colors are pre-marked. The color outlines of cells of different colors are different. The table area samples are input into the model, and the color and cell outline of each cell in the table area samples are predicted. The predicted results are compared with the marked colors and cell outlines, and the model parameters are adjusted to train a deep learning model. In the model application stage, the table area is input into the deep learning model, and the color and cell outline of each cell in the table area are determined by the deep learning model.

[0085] It can be seen that the above describes several situations that may occur in the form area of ​​this application, so that the cell area can be identified for different types of forms, ensuring the diversity of application scenarios of the solution.

[0086] In smart contracts, this application can accurately locate the cell area and automatically match the text data corresponding to the cell area, so that the text in the cell area can also be automatically identified.

[0087] Specifically, the training process of the deep learning model is as follows:

[0088] In one embodiment, the training process of the deep learning model includes: obtaining multiple second image samples, each second image sample includes a table area sample, and the table area sample has a contour-related mark related to the cell contour; inputting each second image sample into an initial model, and determining the contour-related information of each cell in the table area sample through the initial model; adjusting the parameters of the initial model based on the contour-related information of each cell in the table area sample and the corresponding contour-related mark, and continuing training based on the initial model with adjusted parameters until the difference in the predicted contour-related information of each cell in the table area sample and the corresponding contour-related mark is within a preset difference range, thereby obtaining a deep learning model.

[0089] After obtaining a plurality of second image samples, for each second image sample, a table area is determined from the second image sample by using a region segmentation model, and the determined table area is used as a table area sample. The outline of the cells in the table area sample is manually marked to obtain outline-related marks, and the outline-related marks include outline line marks of the cells (which have the same meaning as the above-mentioned cell outlines) or outline marks of text fields corresponding to the cells; the table area sample also has color marks of each cell;

[0090] Input each second image sample into the initial model, and determine the outline-related information of each cell in the table area sample through the initial model, such as the outline-related information including the outline line of the cell, or the outline of the text field corresponding to the cell; and also determine the color of each cell in the second image sample through the initial model;

[0091] A possible situation of this embodiment is: comparing the contour line marks of each cell in the table area sample and the corresponding contour line, adjusting the parameters of the first initial model, and continuing training based on the initial model after the parameter adjustment until the difference between the contour line marks of each cell in the table area sample and the corresponding contour line is within a preset difference range, thereby obtaining a first deep learning model;

[0092] Another possible situation of this embodiment is: comparing the text field contour mark of each cell in the table area sample and the corresponding text field contour, adjusting the parameters of the second initial model, and continuing training based on the initial model after the parameter adjustment until the difference between the text field contour mark of each cell in the table area sample and the corresponding text field contour is within a preset difference range, thereby obtaining a second deep learning model;

[0093] Another possible situation of this embodiment is: comparing the color mark and the corresponding color of each cell in the table area sample, adjusting the parameters of the third initial model, and continuing to train based on the initial model after the parameter adjustment until the difference between the color mark and the corresponding color of each cell in the table area sample obtained by prediction is within a preset difference range, thereby obtaining a third deep learning model;

[0094] In this embodiment, the first deep learning model, the second deep learning model, and the third deep learning model can be integrated into one model.

[0095] One application scenario of this application is to compare the differences between a table before editing and a table after editing. Specifically:

[0096] See also Figure 2The present application also provides a text table recognition method. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0097] Step 202: using the initial image as the image to be processed, executing the above text table recognition method to obtain table data corresponding to the initial image;

[0098] In this embodiment, the initial image includes a table area, and the cells in the table area are in an unedited state, for example, the cells are blank cells;

[0099] Step 204: using the updated image as the image to be processed, executing the above-mentioned text table recognition method to obtain table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image;

[0100] In this embodiment, after updating the data of the cells in the table area in the initial image, an updated image is obtained. Wherein, updating the data of the cells in the table area in the initial image includes: adding data to blank cells in the table area in the initial image, modifying the cell data in the table area in the initial image, or deleting at least one of the cell data in the table area in the initial image.

[0101] Step 206: Compare the table data corresponding to the initial text image and the table data corresponding to the updated text image to determine data update information of the cell.

[0102] The table data corresponding to the initial text image includes the first cell data of the table area in the initial text image. Updating the table data corresponding to the text image includes updating the second cell data of the table area in the text image. For each first cell data, the second cell data corresponding to the same cell as the current first cell data is determined from the second cell data, and the matching degree of the current first cell data and the second cell data corresponding to the same cell is calculated to determine whether there is any deletion, addition or modification of the cell data.

[0103] The matching degree between the current first cell data and the second cell data corresponding to the same cell is calculated, including using text similarity and semantic matching algorithms to determine the matching degree between the current first cell data and the corresponding second cell data.

[0104] In this embodiment, the initial image is taken as the image to be processed, the above-mentioned text table recognition method is executed, and the table data corresponding to the initial image is obtained. Then, the updated image is taken as the image to be processed, the above-mentioned text table recognition method is executed, and the table data corresponding to the updated text image is obtained. Since the updated image is an image obtained by updating the data of the cells in the table area in the initial image, the table data corresponding to the initial text image and the table data corresponding to the updated text image are compared to determine the data update information of the cells. The changes of the cells in the table area in the updated image relative to the initial image can be automatically determined without manual confirmation.

[0105] When this application can be applied to the smart contract signing process, the application solution can facilitate and automate the smart contract signing process, thereby improving the efficiency and accuracy of electronic contract signing.

[0106] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0107] Based on the same inventive concept, the embodiment of the present application also provides a table data comparison device for implementing the table data comparison method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more table data comparison device embodiments provided below can refer to the limitations of the table data comparison method above, and will not be repeated here.

[0108] In an exemplary embodiment, Figure 3 As shown, a text table recognition device 300 is provided, comprising:

[0109] The acquisition module 301 is used to acquire an image to be processed, and determine a table area and a plurality of text areas from the image to be processed by using a region segmentation model; the region segmentation model is obtained by training a first image sample with a text area mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text area mark is obtained by marking a text area in the image body;

[0110] A conversion module 302 is used for converting the image content of each text region into text data, and taking the coordinate position of the text region in an image coordinate system as a first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed;

[0111] A first determination module 303 is used to determine a plurality of cell regions in the table region, and determine a second coordinate position of each cell region in the image coordinate system;

[0112] A second determination module 304, configured to determine target data corresponding to each cell area from a plurality of text data based on the first coordinate position and the second coordinate position;

[0113] The table data output module 305 is used to obtain table data based on each cell and the target data corresponding to each cell.

[0114] In one embodiment, the device further includes a key data determination module, which is used after the conversion module 302 converts the image content of the text area into text data; the key data determination module is used for inputting the current text data into a deep learning model for each text data, learning the semantic features of the current text data through the deep learning model, and determining the key data in the text data based on the semantic features; using the first coordinate position of each text data as the key coordinate position of the corresponding key data;

[0115] Accordingly, the second determining module 304 is specifically configured to:

[0116] Based on the key coordinate position and the second coordinate position, target key data corresponding to each cell area is determined from a plurality of key data.

[0117] In one embodiment, the second determining module 304 is specifically configured to:

[0118] For each cell area, the matching degree between the second coordinate position corresponding to the current cell area and multiple key coordinate positions is calculated; when the matching degree between the second coordinate position corresponding to the current cell area and the key coordinate position is greater than a preset matching degree threshold, the key data corresponding to the key coordinate position is used as the target key data corresponding to the cell area.

[0119] In one embodiment, when determining the multiple cell areas in the table area, the first determination module 303 is specifically used to:

[0120] Inputting the table area into a deep learning model, and determining contour-related information of each cell in the table area through the deep learning model; the contour-related information includes at least one of a cell contour or a text field contour;

[0121] Based on the outline-related information, a plurality of cell areas are determined from the table area.

[0122] In one embodiment, the training process of the deep learning model includes:

[0123] Acquire a plurality of second image samples, each of which includes a table area sample, and the table area sample has a contour-related mark related to a cell contour;

[0124] Inputting each second image sample into an initial model, and determining contour-related information of each cell in the table area sample through the initial model;

[0125] Based on the contour-related information of each cell in the table area sample and the corresponding contour-related marks, the parameters of the initial model are adjusted, and training is continued based on the initial model with adjusted parameters until the predicted contour-related information of each cell in the table area sample and the corresponding contour-related marks are within a preset difference range, thereby obtaining a deep learning model.

[0126] In an exemplary embodiment, Figure 4 As shown, a table data comparison device 400 is provided, comprising:

[0127] The first input module 401 is used to input the initial image as the image to be processed into the table recognition device for processing to obtain the table data corresponding to the initial image;

[0128] The second input module 402 is used to input the updated image as the image to be processed into the text table recognition device for processing to obtain the table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image;

[0129] The comparison module 403 is used to compare the table data corresponding to the initial text image and the table data corresponding to the updated text image to determine the data update information of the cell.

[0130] Each module in the above-mentioned text table recognition device and table data comparison device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0131] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of the table recognition method of the text and data of the table data comparison method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the above method.

[0132] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned methods when executing the computer program.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods of the above embodiments are implemented.

[0135] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned methods when executed by a processor.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0137] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0139] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for recognizing a table in a text, characterized in that: The method comprises: Acquire an image to be processed, and determine a table area and a plurality of text areas from the image to be processed by a region segmentation model; the region segmentation model is obtained by training a first image sample with a text area mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text area mark is obtained by marking the text area in the image body; For each text area, convert the image content of the text area into text data, and use the coordinate position of the text area in the image coordinate system as the first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed; Determine a plurality of cell regions in the table region, and determine a second coordinate position of each cell region in the image coordinate system; Based on the first coordinate position and the second coordinate position, determining target data corresponding to each cell area from a plurality of text data; Based on each cell and the target data corresponding to each cell, table data is obtained.

2. The method according to claim 1, characterized in that After converting the image content of the text area into text data, the method further comprises: For each text data, input the current text data into a deep learning model, learn the semantic features of the current text data through the deep learning model, and determine the key data in the text data based on the semantic features; The first coordinate position of each text data is used as the key coordinate position of the corresponding key data; Correspondingly, determining the target data corresponding to each cell area from the plurality of text data based on the first coordinate position and the second coordinate position includes: Based on the key coordinate position and the second coordinate position, target key data corresponding to each cell area is determined from a plurality of key data.

3. The method according to claim 2, characterized in that The determining, based on the key coordinate position and the second coordinate position, target key data corresponding to each cell area from a plurality of key data includes: For each cell area, the matching degree between the second coordinate position corresponding to the current cell area and multiple key coordinate positions is calculated; when the matching degree between the second coordinate position corresponding to the current cell area and the key coordinate position is greater than a preset matching degree threshold, the key data corresponding to the key coordinate position is used as the target key data corresponding to the cell area.

4. The method according to any one of claims 1 to 3, characterized in that The determining of the plurality of cell areas in the table area comprises: Inputting the table area into a deep learning model, and determining contour-related information of each cell in the table area through the deep learning model; the contour-related information includes at least one of a cell contour or a text field contour; Based on the outline-related information, a plurality of cell areas are determined from the table area.

5. The method according to claim 4, characterized in that The training process of the deep learning model includes: Acquire a plurality of second image samples, each of which includes a table area sample, and the table area sample has a contour-related mark related to a cell contour; Inputting each second image sample into an initial model, and determining contour-related information of each cell in the table area sample through the initial model; Based on the contour-related information of each cell in the table area sample and the corresponding contour-related marks, the parameters of the initial model are adjusted, and training is continued based on the initial model with adjusted parameters until the predicted contour-related information of each cell in the table area sample and the corresponding contour-related marks are within a preset difference range, thereby obtaining a deep learning model.

6. A table data comparison method, characterized in that: The method comprises: Taking the initial image as the image to be processed, executing the method described in any one of claims 1 to 5 to obtain the table data corresponding to the initial image; The updated image is used as the image to be processed, and the method described in any one of claims 1 to 5 is executed to obtain the table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image; The table data corresponding to the initial text image and the table data corresponding to the updated text image are compared to determine data update information of the cell.

7. A text table recognition device, characterized in that: The device comprises: An acquisition module is used to acquire an image to be processed, and determine a table area and a plurality of text areas from the image to be processed by using a region segmentation model; the region segmentation model is obtained by training a first image sample with a text area mark, the first image sample is an image formed by an image body and an overlapping image of the image body, and the text area mark is obtained by marking a text area in the image body; A conversion module, for each text region, converting the image content of the text region into text data, and using the coordinate position of the text region in an image coordinate system as a first coordinate position of the text data; the image coordinate system is a coordinate system constructed for the image to be processed; A first determining module, used for determining a plurality of cell regions in the table region, and determining a second coordinate position of each cell region in the image coordinate system; A second determining module, configured to determine target data corresponding to each cell area from a plurality of text data based on the first coordinate position and the second coordinate position; The table data output module is used to obtain the table data based on each cell and the target data corresponding to each cell.

8. A table data comparison device, characterized in that: The device comprises: A first input module, used to input an initial image as the image to be processed into the text table recognition device of claim 7 for processing, so as to obtain table data corresponding to the initial image; A second input module is used to input the updated image as the image to be processed into the text table recognition device of claim 7 for processing to obtain table data corresponding to the updated text image; the updated image is an image obtained by updating the data of the cells in the table area of ​​the initial image; The comparison module is used to compare the table data corresponding to the initial text image and the table data corresponding to the updated text image to determine the data update information of the cell.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images