Table recognition method and device and related equipment

Abstract

The application discloses a table recognition method and device and related equipment, and relates to the technical field of artificial intelligence, to solve the problem of low accuracy of table recognition results. The method comprises the following steps: obtaining a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image based on a pre-trained semantic segmentation model; obtaining table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image and the fourth target image; the first target image comprises image information representing horizontal frame lines of the first table image, the second target image comprises image information representing vertical frame lines of the first table image, the third target image comprises image information representing horizontal edges of text of the first table image, and the fourth target image comprises image information representing vertical edges of text of the first table image. The embodiment of the application can improve the accuracy of table recognition structure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a table recognition method and device and related equipment. BACKGROUND

[0002] A table is a special information expression structure, which is usually composed of multiple horizontal frame lines and multiple vertical frame lines, and the cells in the table can contain text information. When arranging a table file stored in an unstructured digital file (such as a picture format), the structure and text content of the table usually need to be recognized.

[0003] The prior art usually directly recognizes all line segments in the table in units of line segments. Since the frame lines and text in the table include various line segments, directly recognizing all line segments in the table will result in greater interference between the text line segments and the table frame lines, and the accuracy of the table recognition result is low. SUMMARY

[0004] Embodiments of the present application provide a table recognition method, device and related equipment to solve the problem of low accuracy of table recognition results.

[0005] In a first aspect, embodiments of the present application provide a table recognition method, comprising:

[0006] obtaining a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image based on a pre-trained semantic segmentation model;

[0007] obtaining table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image and the fourth target image;

[0008] The first target image includes image information representing the horizontal frame lines of the first table image, the second target image includes image information representing the vertical frame lines of the first table image, the third target image includes image information representing the horizontal edges of the text of the first table image, and the fourth target image includes image information representing the vertical edges of the text of the first table image.

[0009] In a second aspect, embodiments of the present application also provide a table recognition device, comprising:

[0010] a first obtaining module configured to obtain a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image based on a pre-trained semantic segmentation model;

[0011] The second acquisition module is configured to acquire table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image and the fourth target image.

[0012] The first target image includes image information representing horizontal frame lines of the first table image, the second target image includes image information representing vertical frame lines of the first table image, the third target image includes image information representing horizontal edges of text of the first table image, and the fourth target image includes image information representing vertical edges of text of the first table image.

[0013] In a third aspect, an embodiment of the present application also provides an electronic device, including a memory, a processor and a program stored in the memory and executable on the processor.

[0014] The processor is configured to read the program in the memory to implement steps in the method according to the first aspect.

[0015] In a fourth aspect, an embodiment of the present application also provides a readable storage medium for storing a program, which is executable by a processor to implement steps in the method according to the first aspect.

[0016] In the embodiment of the present application, the first target image, the second target image, the third target image and the fourth target image corresponding to the first table image are acquired based on a pre-trained semantic segmentation model, and the table structure information and the text information corresponding to the first table image are acquired based on the first target image, the second target image, the third target image and the fourth target image. Through the above method, the text edges and the target frame lines are respectively extracted, and then the table structure information and the text information are respectively acquired, which can avoid mutual interference between the frame lines and the text in the table and improve the recognition accuracy of the table. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is one of the flowcharts of the table recognition method provided by the embodiments of the present application;

[0019] Figure 2 is one of the structural schematic diagrams of the semantic segmentation model provided by the embodiments of the present application;

[0020] Figure 3is a schematic diagram of a table block diagram provided by an embodiment of the present application;

[0021] Figure 4 is a flowchart of text information recognition provided by an embodiment of the present application;

[0022] Figure 5 is a flowchart of a table recognition method provided by an embodiment of the present application;

[0023] Figure 6 is a sample image labeling flowchart provided by an embodiment of the present application;

[0024] Figure 7a is a structural schematic diagram of a semantic segmentation model provided by an embodiment of the present application;

[0025] Figure 7b is Figure 7a a structural schematic diagram of a convolution stage provided by the present application;

[0026] Figure 7c is Figure 7a a structural schematic diagram of an up-sampling stage provided by the present application;

[0027] Figure 8 is a flowchart of table structure information determination provided by an embodiment of the present application;

[0028] Figure 9 is a structural schematic diagram of a table recognition device provided by an embodiment of the present application;

[0029] Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0031] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by “first”, “second” are generally a category, not limited to the number of objects, for example, the first object can be one or more. In addition, “and / or” in the specification and claims means at least one of the connected objects, and the character “ / ” generally represents an “or” relationship between the front and rear associated objects.

[0032] Referring to Figure 1 , Figure 1 One of flowcharts of the table recognition method provided by the embodiments of the present application is shown in FIG. 1, which specifically includes the following steps: Figure 1

[0033] In step 101, based on a pre-trained semantic segmentation model, a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image are obtained.

[0034] In step 102, based on the first target image, the second target image, the third target image and the fourth target image, table structure information and text information corresponding to the first table image are obtained.

[0035] The first target image includes image information representing horizontal frame lines of the first table image, the second target image includes image information representing vertical frame lines of the first table image, the third target image includes image information representing text horizontal edges of the first table image, and the fourth target image includes image information representing text vertical edges of the first table image.

[0036] It should be understood that the first table image is an image including a first table. For example, in some embodiments, the first table image is a photo or a printed table file of the first table.

[0037] In the present embodiment, the horizontal frame lines and the vertical frame lines are mainly used to distinguish different frame lines in the table, and do not represent the limitation of the direction of the frame lines. In some embodiments, the horizontal frame lines can also be referred to as horizontal frame lines in the table or table horizontal frame, and the vertical frame lines can also be referred to as column frame lines in the table or table column frame. Since the table is composed of horizontal frame lines and vertical frame lines, the table structure information can be determined based on the first target image and the second target image.

[0038] In some embodiments, the first target image only includes the horizontal frame lines in the first table image, the second target image only includes the vertical frame lines in the first table image, the third target image only includes the text horizontal edges in the first table image, and the fourth target image only includes the text vertical edges in the first table image. Generally, a table is composed of two parts, one part is the frame line structure of the table, which can divide the table into a plurality of different cells, and the other part is the text information located in the cells. Since the frame lines and the text both include line segments, in the case that the frame lines and the text are relatively close, there is interference between the line segments of the frame lines and the line segments of the text.

[0039] ​In the embodiment of the present application, based on the pre-trained semantic segmentation model, the first target image, the second target image, the third target image and the fourth target image corresponding to the first table image are obtained, and the table structure information and the text information corresponding to the first table image are obtained based on the first target image, the second target image, the third target image and the fourth target image. Through the above method, the features of the text edge and the target frame line are extracted respectively, and then the table structure information and the text information are obtained respectively, which can avoid the mutual interference between the frame line and the text in the table, and improve the recognition accuracy of the table.

[0040] It should be noted that in the specific implementation, based on the table structure information and the text information, the table file in different formats can be displayed according to actual needs.

[0041] Optionally, in some embodiments, the step 101 comprises:

[0042] Rotating the first table image by a target angle along a first direction to obtain a second table image;

[0043] Based on the semantic segmentation model, the first segmentation map and the second segmentation map corresponding to the first table image are obtained, and the third segmentation map and the fourth segmentation map corresponding to the second table image are obtained;

[0044] Based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map, the first target image, the second target image, the third target image and the fourth target image corresponding to the first table image are determined;

[0045] The first segmentation map comprises image information representing the target frame line of the first table image, the second segmentation map comprises image information representing the text edge of the first table image, the third segmentation map comprises image information representing the target frame line of the second table image, and the fourth segmentation map comprises image information representing the text edge of the second table image;

[0046] Wherein, the target frame line comprises the horizontal frame line or the vertical frame line, the horizontal frame line is parallel to the vertical frame line after rotating the target angle along the first direction, when the target frame line is the horizontal frame line, the text edge is the horizontal text edge, and when the target frame line is the vertical frame line, the text edge is the vertical text edge.

[0047] It should be noted that in some embodiments, the horizontal frame line and the vertical frame line are in a vertical relationship, and the target angle is 90°. Of course, according to the actual situation, the horizontal frame line and the vertical frame line may not be strictly vertical, and the range of the target angle is not strictly limited here.

[0048] The horizontal frame line is parallel to the vertical frame line after rotating the target angle along the first direction, and thus, after rotating the first table image by the target angle along the first direction to obtain a second table image, the horizontal frame line in the first table image is parallel to the vertical frame line in the second table image, and the vertical frame line in the first table image is parallel to the horizontal frame line in the second table image.

[0049] The semantic segmentation model can realize the function of edge detection. In some embodiments, by pre-training the semantic segmentation model, the semantic segmentation model can extract the features of horizontal edges, and the horizontal edges are divided into horizontal frame lines and horizontal edges of text. In other embodiments, by pre-training the semantic segmentation model, the semantic segmentation model can extract the features of vertical edges, and the vertical edges are divided into vertical frame lines and vertical edges of text.

[0050] For the convenience of understanding, the following will take the semantic segmentation model extracting the features of horizontal edges as an example for description.

[0051] The first table image is input into the semantic segmentation model, and the semantic segmentation model extracts the features of the horizontal edges in the first table image, so that the image information of the horizontal frame lines of the first table image and the image information of the horizontal edges of text of the first table image can be obtained.

[0052] The second table image is input into the semantic segmentation model, and the semantic segmentation model extracts the features of the horizontal edges in the second table image, so that the image information of the horizontal frame lines of the second table image and the image information of the horizontal edges of text of the second table image can be obtained.

[0053] Since the second table image is obtained by rotating the first table image by the target angle along the first direction, the image information of the horizontal frame lines of the second table image is actually the image information of the vertical frame lines of the first table image, and the image information of the horizontal edges of text of the second table image is actually the image information of the vertical edges of text of the first table image.

[0054] Therefore, after obtaining the second table image by rotating the first table image by the target angle along the first direction, and inputting the first table image and the second table image into the semantic segmentation model, the image information of the horizontal frame lines, the vertical frame lines, the horizontal edges of text and the vertical edges of text in the first table image can be obtained.

[0055] The embodiments of the semantic segmentation model extracting the features of vertical edges can refer to the description of the above embodiments, and will not be repeated here. In the embodiments of the semantic segmentation model extracting the features of vertical edges, the image information of the horizontal frame lines, the vertical frame lines, the horizontal edges of text and the vertical edges of text in the first table image can also be finally obtained.

[0056] In the embodiments of the present application, the semantic segmentation model only extracts features of horizontal edges or features of vertical edges. By rotating the first table image by a target angle in a first direction to obtain a second table image, image information of horizontal frame lines, vertical frame lines, horizontal edges of text and vertical edges of text in the first table image can be obtained based on the semantic segmentation model. Compared with the case of using the semantic segmentation model to simultaneously extract features of horizontal edges and features of vertical edges of the first table image, the method provided in the present application can reduce the processing amount of filtering processing, improve the efficiency and accuracy of feature extraction, and further improve the efficiency of table recognition.

[0057] Optionally, in some embodiments, in a case where the target frame line is the horizontal frame line, the determining the first target image, the second target image, the third target image and the fourth target image based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map comprises:

[0058] determining the first segmentation map as the first target image, determining the second segmentation map as the third target image, rotating the third segmentation map by the target angle in a second direction to obtain the second target image, and rotating the fourth segmentation map by the target angle in the second direction to obtain the fourth target image, the second direction being opposite to the first direction.

[0059] Optionally, in some embodiments, in a case where the target frame line is the vertical frame line, the determining the first target image, the second target image, the third target image and the fourth target image based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map comprises:

[0060] rotating the first segmentation map by the target angle in a second direction to obtain the first target image, rotating the second segmentation map by the target angle in the second direction to obtain the third target image, determining the third segmentation map as the second target image, and determining the fourth segmentation map as the fourth target image, the second direction being opposite to the first direction.

[0061] In the embodiments of the present application, in a case where the target frame line is a horizontal frame line or a vertical frame line, only two of the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map need to be rotated to obtain the first target image, the second target image, the third target image and the fourth target image. By rotating the image, the processing amount of filtering processing can be reduced, the process of table processing can be simplified, and the efficiency of table recognition can be improved.

[0062] Optionally, in some embodiments, the obtaining, based on the semantic segmentation model, a first segmentation map and a second segmentation map corresponding to the first table image, and a third segmentation map and a fourth segmentation map corresponding to the second table image comprises:

[0063] inputting the first table image and the second table image into the semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image;

[0064] obtaining, based on the first semantic segmentation map, a first segmentation map and a second segmentation map corresponding to the first table image, and obtaining, based on the second semantic segmentation map, a third segmentation map and a fourth segmentation map corresponding to the second table image.

[0065] inputting the first table image into the semantic segmentation model to output a first semantic segmentation map corresponding to the first table image. Each pixel (also referred to as a coordinate position or a pixel point) in the first semantic segmentation map corresponds to class label information, which represents the class of the pixel as background, a target frame line, or a character edge.

[0066] Based on the class label information corresponding to each pixel, the pixels of the class of the target frame line can be extracted separately to obtain a first segmentation map corresponding to the first table image, and the pixels of the class of the character edge can be extracted separately to obtain a second segmentation map corresponding to the first table image.

[0067] For example, in some embodiments, the first table image is input into the semantic segmentation model to obtain a first semantic segmentation map. Each pixel in the first semantic segmentation map corresponds to class label information, wherein the class label information of 0 represents the class of the pixel as background, the class label information of 1 represents the class of the pixel as a target frame line, and the class label information of 2 represents the class of the pixel as a character edge.

[0068] The pixels in the first semantic segmentation map with the class label information of 1 are determined, and the value of the pixel is set to 255. At the same time, the values of the other pixels (pixels with the class label information of 0 and 2) in the first semantic segmentation map are all set to 0 to obtain a first segmentation map.

[0069] The pixels in the first semantic segmentation map with the class label information of 2 are determined, and the value of the pixel is set to 255. At the same time, the values of the other pixels (pixels with the class label information of 0 and 1) in the first semantic segmentation map are all set to 0 to obtain a second segmentation map.

[0070] The specific process of obtaining the third segmentation map and the fourth segmentation map based on the second semantic segmentation map can be referred to the foregoing content, and will not be repeated here.

[0071] In the embodiments of the present application, the first table image and the second table image are input into a semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image; a first segmentation map and a second segmentation map are obtained based on the first semantic segmentation map, and a third segmentation map and a fourth segmentation map are obtained based on the second semantic segmentation map. Through the above method, the target frame line, the character edge and the background can be distinguished by performing semantic segmentation on the first table image and the second table image, the mutual interference between the character edge and the target frame line is reduced, and the accuracy of table recognition is improved.

[0072] Optionally, as shown in some embodiments, the semantic segmentation model comprises a first down-sampling network, a second down-sampling network and an up-sampling network: Figure 2

[0073] The first down-sampling network is configured to down-sample the input image in a third direction and a fourth direction to obtain a first feature map, the third direction is parallel to the target frame line, and the fourth direction is perpendicular to the target frame line; the second down-sampling network is configured to down-sample the first feature map in the third direction to obtain a second feature map; and the up-sampling network is configured to up-sample the second feature map to obtain an output image, the size of the output image is the same as that of the input image.

[0074] In a case where the input image comprises the first table image, the output image comprises the first semantic segmentation map; and in a case where the input image comprises the second table image, the output image comprises the second semantic segmentation map.

[0075] The specific structure of the first down-sampling network is not limited herein. For example, in some embodiments, the first down-sampling network comprises one down-sampling convolutional layer. In other embodiments, the first down-sampling network comprises at least two down-sampling convolutional layers connected in sequence.

[0076] The specific structure of the second down-sampling network is not limited herein. For example, in some embodiments, the second down-sampling network comprises one down-sampling convolutional layer. In other embodiments, the second down-sampling network comprises at least two down-sampling convolutional layers connected in sequence.

[0077] It should be noted that the first down-sampling network and the second down-sampling network are both configured to down-sample the input image, but the down-sampling directions of the first down-sampling network and the second down-sampling network are different. The first down-sampling network is configured to down-sample the input image in the third direction and the fourth direction, and the second down-sampling network is configured to down-sample the first feature map in the third direction.

[0078] ​The upsampling network is configured to upsample the second feature map, so that the size of the output image is restored to the same size as the input image. The specific structure of the upsampling network is not limited here, and in actual use, the structure of the upsampling network can be set and adjusted according to the structures of the first downsampling network and the second downsampling network.

[0079] For the convenience of understanding, specific examples will be used for illustration in the following.

[0080] In the case where the target frame line includes a horizontal frame line, the semantic segmentation model is configured to extract features of the horizontal edge. The third direction can be understood as the horizontal direction, and the fourth direction can be understood as the vertical direction. The first downsampling network is configured to downsample the input image in the horizontal direction and the vertical direction at the same time, so as to reduce the size of the input image, thereby improving the calculation efficiency of the semantic segmentation model. The second downsampling network is configured to downsample the first feature map in the horizontal direction while keeping the scale of the first feature map in the vertical direction unchanged, so as to avoid the features in the vertical direction being excessively compressed, expand the receptive field in the horizontal direction, improve the extraction effect of the features in the horizontal direction, and thereby guarantee the extraction performance of the semantic segmentation model for the horizontal edge.

[0081] In the case where the target frame line includes a vertical frame line, the semantic segmentation model is configured to extract features of the vertical edge. The third direction can be understood as the vertical direction, and the fourth direction can be understood as the horizontal direction. The first downsampling network is configured to downsample the input image in the horizontal direction and the vertical direction at the same time, so as to reduce the size of the input image, thereby improving the calculation efficiency of the semantic segmentation model. The second downsampling network is configured to downsample the first feature map in the vertical direction while keeping the scale of the first feature map in the horizontal direction unchanged, so as to avoid the features in the horizontal direction being excessively compressed, expand the receptive field in the vertical direction, improve the extraction effect of the features in the vertical direction, and thereby guarantee the extraction performance of the semantic segmentation model for the vertical edge.

[0082] In the embodiments of the present application, the semantic segmentation model includes the first downsampling network, the second downsampling network and the upsampling network. Through the setting of the first downsampling network, the size of the input image can be reduced, the calculation amount of the semantic segmentation model can be reduced, and the calculation efficiency of the semantic segmentation model can be improved. Through the setting of the second downsampling network, the extraction performance of the semantic segmentation model for the features of the horizontal edge or the vertical edge can be guaranteed. Through the first downsampling network and the second downsampling network, different sampling rates can be set for different directions, so as to guarantee the extraction performance of the semantic segmentation model for the features of the horizontal edge or the vertical edge.

[0083] In some embodiments, the first table image and the second table image can be input into the semantic segmentation model simultaneously. In this embodiment, the input image corresponding to the first downsampling network includes the first table image and the second table image; the output image corresponding to the upsampling network includes the first semantic segmentation map and the second semantic segmentation map.

[0084] In other embodiments, the first table image and the second table image can be input into the semantic segmentation model respectively. In this embodiment, the input image corresponding to the first downsampling network includes either the first table image or the second table image; the output image corresponding to the upsampling network includes either the first semantic segmentation map or the second semantic segmentation map.

[0085] Optionally, in some embodiments, the step of inputting the first table image and the second table image into the semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image includes:

[0086] Image augmentation is performed on the first table image and the second table image to obtain a first augmented image and a second augmented image, wherein the size of the first augmented image is the same as the size of the second augmented image.

[0087] The first augmented image and the second augmented image are input into the semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image.

[0088] It should be understood that the specific method of image amplification of the first and second table images to obtain the first and second amplified images is not limited here. For ease of description, the dimensions of the first table image are denoted as (w1, h1), and the dimensions of the second table image are denoted as (w2, h2). The dimensions of both the first and second amplified images are (w1, h1). y ,h y ).

[0089] Exemplarily, in some embodiments, w y It is the preset value, and w y Greater than w1 and w2; h y It is a preset value, and h y Greater than h1 and h2. In other embodiments, w y h is the maximum value among w1 and w2. y It is the maximum value among h1 and h2.

[0090] In the embodiment of the present application, the first table image and the second table image are image augmented to obtain a first augmented image and a second augmented image of the same size, and the first augmented image and the second augmented image are simultaneously input into a semantic segmentation model to obtain a first semantic segmentation map and a second semantic segmentation map. Through the above method, the first semantic segmentation map and the second semantic segmentation map can be obtained simultaneously, and the efficiency of table recognition is improved.

[0091] Optionally, in some embodiments, the table frame diagram includes a plurality of horizontal frame lines and a plurality of vertical frame lines, and the plurality of horizontal frame lines and the plurality of vertical frame lines enclose a plurality of cells, and the table structure information includes structure information of the plurality of cells.

[0092] The table structure information corresponding to the first table image is determined based on the table frame diagram, including:

[0093] The first serial number of each horizontal frame line is determined based on the coordinate value of the plurality of horizontal frame lines in a pre-established rectangular coordinate system, and the second serial number of each vertical frame line is determined based on the coordinate value of the plurality of vertical frame lines in the rectangular coordinate system.

[0094] The structure information of each cell in the plurality of cells is determined, and the structure information includes a first sub-serial number, a second sub-serial number, a third sub-serial number and a fourth sub-serial number. The first sub-serial number and the second sub-serial number are the first serial numbers of the horizontal frame lines enclosing the cell, and the third sub-serial number and the fourth sub-serial number are the second serial numbers of the vertical frame lines enclosing the cell.

[0095] The plurality of horizontal frame lines and the plurality of vertical frame lines enclose a plurality of cells. For any one cell, its periphery is formed by two horizontal frame lines and two vertical frame lines. Therefore, the position of the cell and the size of the cell can be determined by the two horizontal frame lines and the two vertical frame lines enclosing the cell.

[0096] For the convenience of understanding, a specific embodiment will be described below.

[0097] A Cartesian coordinate system is pre-established. Based on the coordinates of each horizontal frame line in this system, a first index is determined for each horizontal frame line, and a second index is determined for each vertical frame line. For example, the first index is denoted as `row`, and the second index as `col`. The value of `row` for each horizontal frame line is sequentially set according to its coordinates along the vertical axis of the Cartesian coordinate system, where the values ​​of `row` are 1, 2, ..., n. Similarly, the value of `col` for each vertical frame line is sequentially set according to its coordinates along the horizontal axis of the Cartesian coordinate system, where the values ​​of `col` are 1, 2, ..., m. Here, both n and m are positive integers.

[0098] For ease of description, the structure information of the cell is denoted as (span_cols, span_rows), where span_cols is the number of rows in the cell. cols =[col left ,col right ], span_rows = [row top ,row bottom ]. col left The col represents the second index of the left vertical border of the cell. right The row represents the second index of the vertical border on the right side of the cell. top The row represents the first index of the top horizontal border of the cell. bottom This represents the first index of the horizontal border at the bottom of the cell. In this embodiment, row top and row bottom This can be understood as the first sub-sequence number and the second sub-sequence number, col left and col right This can be understood as the third sub-sequence number and the fourth sub-sequence number.

[0099] Please see Figure 3 Let's take cell A and cell B as examples. The structure of cell A is as follows: span cols =[1,2], span_rows=[3,2]. The structure information of cell B is as follows: span cols =[2,3], span_rows=[3,1]. Based on the structural information of cells A and B, we know that cell A is located to the left of cell B, and the size of cell A is smaller than the size of cell B. Cell B can be a merged cell.

[0100] The method shown in the embodiments of this application can determine the relative position and size of each cell in a table by using the structural information of the cells. It can also identify cells of different sizes, thereby improving the accuracy and convenience of table structure recognition.

[0101] Optionally, in some embodiments, the step 102 comprises:

[0102] performing merging processing on the first target image and the second target image to obtain a table frame diagram;

[0103] determining table structure information corresponding to the first table image based on the table frame diagram;

[0104] determining text information corresponding to the first table image based on the table structure information, the third target image and the fourth target image.

[0105] The table frame diagram is obtained by merging the first target image and the second target image, and thus the table frame diagram includes information in the first target image and information in the second target image. Specifically, the table frame diagram includes horizontal frame lines in the first table image and vertical frame lines in the first table image, and a plurality of cells can be divided by the intersection of the horizontal frame lines and the vertical frame lines.

[0106] The table structure information corresponding to the first table image can be determined based on the table frame diagram. The specific content of the table structure information is not limited herein. Exemplarily, in some embodiments, the table structure information includes the number of horizontal frame lines, the number of vertical frame lines, the number of cells and the size of each cell, etc.

[0107] In the embodiments of the present application, the first target image and the second target image are merged to obtain a table frame diagram, the table structure information corresponding to the first table image is determined based on the table frame diagram, and the text information corresponding to the first table image is determined based on the table structure information, the third target image and the fourth target image. Through the above setting, after obtaining the table structure information, the text information is determined based on the table structure information, which can improve the convenience of positioning the text information and improve the accuracy of the text information.

[0108] Optionally, in some embodiments, the determination of the text information corresponding to the first table image based on the table structure information, the third target image and the fourth target image comprises:

[0109] performing merging processing on the third target image and the fourth target image to obtain a text feature diagram, the text feature diagram including a text region;

[0110] obtaining a text field region based on the table structure information and the coordinate values of the text region in the rectangular coordinate system, the text field region including text regions located in the same cell and in the same row;

[0111] determining the text information corresponding to the first table image based on the coordinate values of the text field region in the rectangular coordinate system and the text field region in the first table image. Figure 5Obtain the text field image corresponding to the text field region from the image;

[0112] The text field image is input into a pre-trained character recognition model to obtain a character recognition result. The text information includes the character recognition result and the coordinate values ​​of the text field region in the Cartesian coordinate system.

[0113] It should be understood that, based on table structure information and the coordinate values ​​of the text area, the text area can be divided into multiple text sub-areas, each located within a single cell. Furthermore, based on the coordinate values ​​of the text sub-areas, multiple lines of text located within the same cell can be divided into text field areas.

[0114] Based on the coordinate values ​​of the text field region, determine the text field image corresponding to the coordinate values ​​in the first table image. Input the text field image into the pre-trained character recognition model to obtain the character recognition result.

[0115] To make it easier to understand, examples will be given below. Please refer to [link / reference]. Figure 3 and Figure 4 .

[0116] By merging the third target image and the fourth target image, a text feature map is obtained, which includes text regions. The text feature map is then used to map the text regions in a Cartesian coordinate system.

[0117] Scale value and basis Figure 3 By comparing the obtained table structure information, it can be determined that both cell B and cell C0 contain a portion of text area.

[0118] Based on the coordinates of the text area in cell B, we know that cell B contains two lines of text. Therefore, we can further split the text area in cell B to obtain text field area 1 and text field area 2. Based on the coordinates of the text area in cell C, we know that cell C contains only one line of text. Therefore, we can obtain text field area 3.

[0119] 5. Based on the coordinate values ​​of text field region 1, text field region 2 and text field region 3, find the regions with the same coordinate values ​​in the first table image, and then obtain text field image 1 corresponding to text field region 1, text field image 2 corresponding to text field region 2 and text field image 3 corresponding to text field region 3.

[0120] By inputting text field image 1, text field image 2, and text field image 3 into a pre-trained text recognition model, the corresponding text recognition results can be obtained.

[0121] In the embodiment of the present application, the text field region is obtained based on the table structure information and the coordinate values of the text regions in the rectangular coordinate system, the text field region includes text regions located in the same cell and in the same row, and the text field image corresponding to the text field region in the first table image is obtained based on the coordinate values of the text field region in the rectangular coordinate system; and the text field image is input into the pre-trained text recognition model to obtain the text recognition result. Through the above method, the text detection function is realized in combination with the table structure information, the processing efficiency can be improved, the mutual interference between the table frame lines and the text information is reduced, and the accuracy of table recognition is improved.

[0122] The specific process of the table recognition method provided in the present application will be described below with reference to a specific embodiment. Please refer to Figures 5-8 . Figure 5 The flowchart of the table recognition method provided in the embodiment of the present application.

[0123] The semantic segmentation model is pre-trained. In some embodiments, as shown in Figure 6 , the horizontal edges of the sample image are detected by using the edge detection operator (for example, the canny algorithm) in the related art, the data pre-labeling is realized by horizontal filtering, and finally the pre-labeled results are verified by manual operation to obtain the final labeled sample image. Through the above method, the sample image is labeled, the training data set is constructed, and the semantic segmentation model is trained by using the training data set to obtain the final semantic segmentation model. Through the above method, the labeling efficiency can be improved, and the data set can be expanded. In addition, in the specific implementation, in order to improve the model generalization, various table styles can be supplemented by data generation, and the specific method is not described here.

[0124] In the embodiment, the structure of the semantic segmentation model is as shown in Figures 7a-7c . Among them, Figure 7b is a structural diagram of a convolution stage (conv stage), Figure 7c is a structural diagram of an up-sampling stage (upconv stage). The input image size of the semantic segmentation model is 768*768, the edge segmentation result line_mask is extracted from the depth feature map of 1 / 1 original size, line_mask is a 3-channel image with a size of 768*768, and the value of each position line_mask i,j represents the class probability distribution of this position (or called pixel), and the corresponding semantic segmentation graph is obtained after argmax. Among them, line_mask i,j 0 represents that this position is background, 1 represents that this position is horizontal frame line, and 2 represents that this position is text horizontal edge.

[0125] To ensure the performance of the model in extracting horizontal edges, the size of the convolution kernel is modified so that the input image is down-sampled twice in the y direction in the first down-sampling network and is down-sampled only in the x direction in the second down-sampling network, while keeping the scale of the feature map in the y direction unchanged.

[0126] See Figure 8 , the first table image is rotated by 90° to obtain a second table image, and the first table image and the second table image are scale augmented to obtain a first augmented image and a second augmented image. The size of the first table image is denoted as (w1, h1), the size of the second table image is denoted as (w2, h2), and the sizes of the first augmented image and the second augmented image are both (w y ,h y ). w y is equal to h y , and is the maximum value among w1, w2, h1 and h2.

[0127] The first augmented image and the second augmented image are input into a semantic segmentation model as shown in Figure 6 , to obtain a first semantic segmentation map and a second semantic segmentation map. For the first semantic segmentation map, positions where line_mask i,j is 1 are extracted, and the values of the positions are set to 255, and the values of other positions are set to 0, to obtain a first segmentation map; for the second semantic segmentation map, positions where line_mask i,j is 1 are extracted, and the values of the positions are set to 255, and the values of other positions are set to 0, to obtain a third segmentation map.

[0128] The first segmentation map is a binary image, and each connected domain in the binary image represents a horizontal frame line in the first table image. The average longitudinal coordinate value of the connected domain in a pre-established rectangular coordinate system is calculated, and the corresponding serial number is the first serial number of the horizontal frame line, denoted as row, according to the longitudinal coordinate sorting. The third segmentation map is a binary image, and each connected domain in the binary image represents a vertical frame line in the first table image. The average transverse coordinate value of the connected domain in the pre-established rectangular coordinate system is calculated, and the corresponding serial number is the second serial number of the vertical frame line, denoted as col, according to the transverse coordinate sorting.

[0129] The third segmentation map is rotated by -90°, and then the first segmentation map and the third segmentation map are taken as a union to obtain a table frame map. The inner contour of the table frame map is searched to obtain the position of each cell and the range serial number of the horizontal frame line and the vertical frame line crossed by the cell, and table structure information (or referred to as cell information) is obtained. The cells are sorted based on the cell information, and html structured information is output.

[0130] For the first semantic segmentation map, positions where line_mask i,jThe position with a value of 2 is obtained, and the value of the position is set to 255, and the values of other positions are set to 0, to obtain a second segmentation map; for the second semantic segmentation map, a line_mask is extracted i,j The position with a value of 2 is obtained, and the value of the position is set to 255, and the values of other positions are set to 0, to obtain a fourth segmentation map.

[0131] The second segmentation map and the fourth segmentation map are both binary maps, and the second segmentation map and the fourth segmentation map are merged to obtain a text feature map. For each cell region, a horizontal projection of the binary map is performed to obtain a text field region; and an enclosing rectangle of each text field region is calculated to obtain coordinate values of the enclosing rectangle. A corresponding text field image is obtained in the first table image based on the coordinate values of the enclosing rectangle. A text field image is recognized based on a convolutional recurrent neural network (CRNN) to obtain a text recognition result. The text information includes the text recognition result and the coordinate values of the text field region in the rectangular coordinate system. After obtaining the text recognition result, the text can be output in a structured manner, that is, the table structure information and the text recognition structure are combined to output the text information in a table form.

[0132] In some embodiments, more fonts and richer corpus information can be obtained through a generation algorithm based on a public data set to supplement the data, and a printed text recognition model with better performance is trained.

[0133] In this embodiment, for a table image with a frame line, the semantic segmentation model can realize extraction of horizontal edge features, and distinguish two categories of horizontal frame lines and horizontal edges of text. At the same time, considering that the semantic segmentation model is used to realize detection of horizontal edge features, different sampling rates are designed for the horizontal direction and the vertical direction during downsampling to ensure the accuracy of horizontal edge detection.

[0134] The embodiment of the present application also provides a table recognition device. Referring to Figure 9 , Figure 9 is a structural diagram of the table recognition device provided by the embodiment of the present application. Since the principle of solving the problem of the table recognition device is similar to that of the table recognition method in the embodiment of the present application, the implementation of the table recognition device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0135] As shown in Figure 9 , the table recognition device 900 includes:

[0136] The first acquisition module 901 is configured to acquire, based on a pre-trained semantic segmentation model, a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image;

[0137] The second acquisition module 902 is configured to acquire table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image, and the fourth target image.

[0138] The first target image includes image information representing horizontal frame lines of the first table image, the second target image includes image information representing vertical frame lines of the first table image, the third target image includes image information representing horizontal edges of text of the first table image, and the fourth target image includes image information representing vertical edges of text of the first table image.

[0139] Optionally, the second acquisition module 902 includes:

[0140] The merging processing unit is configured to perform merging processing on the first target image and the second target image to obtain a table frame diagram.

[0141] The first determination unit is configured to determine the table structure information corresponding to the first table image based on the table frame diagram.

[0142] The second determination unit is configured to determine the text information corresponding to the first table image based on the table structure information, the third target image, and the fourth target image.

[0143] Optionally, the first acquisition module 901 includes:

[0144] The rotating unit is configured to rotate the first table image by a target angle along a first direction to obtain a second table image.

[0145] The acquisition unit is configured to acquire a first segmentation diagram and a second segmentation diagram corresponding to the first table image and acquire a third segmentation diagram and a fourth segmentation diagram corresponding to the second table image based on the semantic segmentation model.

[0146] The third determination unit is configured to determine the first target image, the second target image, the third target image, and the fourth target image corresponding to the first table image based on the first segmentation diagram, the second segmentation diagram, the third segmentation diagram, and the fourth segmentation diagram.

[0147] The first segmentation diagram includes image information representing target frame lines of the first table image, the second segmentation diagram includes image information representing text edges of the first table image, the third segmentation diagram includes image information representing target frame lines of the second table image, and the fourth segmentation diagram includes image information representing text edges of the second table image.

[0148] The target frame line includes the horizontal frame line or the vertical frame line, the horizontal frame line is parallel to the vertical frame line after being rotated by the target angle along the first direction, the text edge is the horizontal text edge when the target frame line is the horizontal frame line, and the text edge is the vertical text edge when the target frame line is the vertical frame line.

[0149] Optionally, in a case where the target frame line is the horizontal frame line, the third determination unit is specifically configured to:

[0150] The first segmentation map is determined as the first target image, the second segmentation map is determined as the third target image, the third segmentation map is rotated by the target angle along a second direction to obtain the second target image, and the fourth segmentation map is rotated by the target angle along the second direction to obtain the fourth target image, the second direction being opposite to the first direction.

[0151] Optionally, the acquisition unit includes:

[0152] The first input subunit is configured to input the first table image and the second table image into the semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image.

[0153] The first acquisition subunit is configured to acquire a first segmentation map and a second segmentation map corresponding to the first table image based on the first semantic segmentation map, and acquire a third segmentation map and a fourth segmentation map corresponding to the second table image based on the second semantic segmentation map.

[0154] Optionally, the semantic segmentation model includes a first down-sampling network, a second down-sampling network, and an up-sampling network.

[0155] The first down-sampling network is configured to down-sample an input image in a third direction and a fourth direction to obtain a first feature map, the third direction being parallel to the target frame line, and the fourth direction being perpendicular to the target frame line, the second down-sampling network is configured to down-sample the first feature map in the third direction to obtain a second feature map, and the up-sampling network is configured to up-sample the second feature map to obtain an output image, the output image having the same size as the input image.

[0156] In a case where the input image includes the first table image, the output image includes the first semantic segmentation map, and in a case where the input image includes the second table image, the output image includes the second semantic segmentation map.

[0157] Optionally, the first input subunit is specifically configured to:

[0158] perform image augmentation on the first table image and the second table image to obtain a first augmented image and a second augmented image, the first augmented image having a same size as the second augmented image;

[0159] input the first augmented image and the second augmented image into the semantic segmentation model to obtain a first semantic segmentation map corresponding to the first table image and a second semantic segmentation map corresponding to the second table image.

[0160] Optionally, the table frame diagram includes a plurality of horizontal frame lines and a plurality of vertical frame lines, and the plurality of horizontal frame lines and the plurality of vertical frame lines enclose a plurality of cells, and the table structure information includes structure information of the plurality of cells.

[0161] The first determining unit includes:

[0162] a first determining subunit configured to determine a first serial number of each of the horizontal frame lines based on coordinate values of the plurality of horizontal frame lines in a pre-established rectangular coordinate system, and determine a second serial number of each of the vertical frame lines based on coordinate values of the plurality of vertical frame lines in the rectangular coordinate system.

[0163] a second determining subunit configured to determine structure information of each of the plurality of cells, the structure information including a first subserial number, a second subserial number, a third subserial number and a fourth subserial number, the first subserial number and the second subserial number being the first serial numbers of the horizontal frame lines enclosing the cell, and the third subserial number and the fourth subserial number being the second serial numbers of the vertical frame lines enclosing the cell.

[0164] Optionally, the second determining unit includes:

[0165] a merging subprocessing unit configured to perform merging processing on the third target image and the fourth target image to obtain a character feature map, the character feature map including a character region.

[0166] a second acquiring subunit configured to acquire a text field region based on the table structure information and coordinate values of the character region in the rectangular coordinate system, the text field region including character regions located in a same cell and in a same row.

[0167] a third acquiring subunit configured to acquire a text field image corresponding to the text field region from the first table image based on coordinate values of the text field region in the rectangular coordinate system.

[0168] The second input subunit is configured to input the text field image into a pre-trained character recognition model to obtain a character recognition result, and the character information includes the character recognition result and coordinate values of the text field region in the rectangular coordinate system.

[0169] The table recognition device 900 provided by the embodiment of the present application can execute the method embodiments described above, and has similar implementation principles and technical effects. Details are not described herein again.

[0170] As shown in Figure 10 The embodiment of the present application further provides an electronic device 1000, which comprises a processor 1001, a memory 1002, a program or instruction stored in the memory 1002 and executable on the processor 1001. When the program or instruction is executed by the processor 1001, each process of the method embodiment shown in Figure 1 can be implemented, and the same technical effects can be achieved. To avoid repetition, details are not described herein again.

[0171] The embodiment of the present application further provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, each process of the method embodiment shown in Figure 1 can be implemented, and the same technical effects can be achieved. To avoid repetition, details are not described herein again.

[0172] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented by other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0173] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can be physically included separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function unit.

[0174] The integrated unit in the form of software function unit can be stored in a computer readable storage medium. The software function unit is stored in a storage medium, and includes a plurality of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform part of the steps of the transceiving method according to the embodiments of the present application. The storage medium mentioned above includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0175] The above describes the preferred embodiments of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A form recognition method characterized by, The method comprises: obtaining, based on a pre-trained semantic segmentation model, a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image; obtaining, based on the first target image, the second target image, the third target image and the fourth target image, table structure information and text information corresponding to the first table image; wherein the first target image comprises image information representing horizontal frame lines of the first table image, the second target image comprises image information representing vertical frame lines of the first table image, the third target image comprises image information representing text horizontal edges of the first table image, and the fourth target image comprises image information representing text vertical edges of the first table image; the obtaining, based on the pre-trained semantic segmentation model, the first target image, the second target image, the third target image and the fourth target image corresponding to the first table image comprises: rotating the first table image by a target angle in a first direction to obtain a second table image; obtaining, based on the semantic segmentation model, a first segmentation map and a second segmentation map corresponding to the first table image, and obtaining a third segmentation map and a fourth segmentation map corresponding to the second table image; determining, based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map, the first target image, the second target image, the third target image and the fourth target image corresponding to the first table image; the first segmentation map comprises image information representing target frame lines of the first table image, the second segmentation map comprises image information representing text edges of the first table image, the third segmentation map comprises image information representing target frame lines of the second table image, and the fourth segmentation map comprises image information representing text edges of the second table image; wherein the target frame lines comprise the horizontal frame lines or the vertical frame lines, the horizontal frame lines are parallel to the vertical frame lines after being rotated by the target angle in the first direction, in the case that the target frame lines are the horizontal frame lines, the text edges are the text horizontal edges, and in the case that the target frame lines are the vertical frame lines, the text edges are the text vertical edges.

2. The method of claim 1, wherein, the obtaining, based on the first target image, the second target image, the third target image and the fourth target image, the table structure information and the text information corresponding to the first table image comprises: performing merging processing on the first target image and the second target image to obtain a table frame map; determining, based on the table frame map, table structure information corresponding to the first table image; determining, based on the table structure information, the third target image and the fourth target image, text information corresponding to the first table image.

3. The method of claim 1, wherein, in the case that the target frame lines are the horizontal frame lines, the determining, based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map, the first target image, the second target image, the third target image and the fourth target image comprises: The first segmentation map is determined as the first target image, the second segmentation map is determined as the third target image, the third segmentation map is rotated by the target angle along a second direction to obtain the second target image, and the fourth segmentation map is rotated by the target angle along the second direction to obtain the fourth target image, the second direction being opposite to the first direction.

4. The method of claim 1, wherein, The first segmentation map corresponding to the first table image and the second segmentation map are obtained based on the semantic segmentation model, and the third segmentation map corresponding to the second table image and the fourth segmentation map are obtained. The first table image and the second table image are input into the semantic segmentation model to obtain the first semantic segmentation map corresponding to the first table image and the second semantic segmentation map corresponding to the second table image. The first semantic segmentation model includes a first downsampling network, a second downsampling network, and an upsampling network.

5. The method of claim 4, wherein, The first downsampling network is used for downsampling an input image in a third direction and a fourth direction to obtain a first feature map, the third direction being parallel to the target frame line, and the fourth direction being perpendicular to the target frame line. The first table image and the second table image are input into the semantic segmentation model to obtain the first semantic segmentation map corresponding to the first table image and the second semantic segmentation map corresponding to the second table image. The first table image and the second table image are input into the semantic segmentation model to obtain the first semantic segmentation map corresponding to the first table image and the second semantic segmentation map corresponding to the second table image.

6. The method of claim 4, wherein, The first table image and the second table image are input into the semantic segmentation model to obtain the first semantic segmentation map corresponding to the first table image and the second semantic segmentation map corresponding to the second table image. The first table image and the second table image are input into the semantic segmentation model to obtain the first semantic segmentation map corresponding to the first table image and the second semantic segmentation map corresponding to the second table image. The table frame map includes a plurality of horizontal frame lines and a plurality of vertical frame lines, and a plurality of unit cells are enclosed by the plurality of horizontal frame lines and the plurality of vertical frame lines.

7. The method of claim 2, wherein, The table structure information corresponding to the first table image is determined based on the table frame map, and the table structure information corresponding to the second table image is determined based on the table frame map. The table structure information corresponding to the first table image is determined based on the table frame map, and the table structure information corresponding to the second table image is determined based on the table frame map. determine a first serial number of each of the horizontal frame lines based on coordinate values of the horizontal frame lines in a pre-established rectangular coordinate system, and determine a second serial number of each of the vertical frame lines based on coordinate values of the vertical frame lines in the rectangular coordinate system; determine structure information of each of the plurality of cells, the structure information comprising a first sub-serial number, a second sub-serial number, a third sub-serial number and a fourth sub-serial number, the first sub-serial number and the second sub-serial number being the first serial numbers of the horizontal frame lines enclosing the cell, and the third sub-serial number and the fourth sub-serial number being the second serial numbers of the vertical frame lines enclosing the cell.

8. The method of claim 7, wherein, determine the text information corresponding to the first table image based on the table structure information, the third target image and the fourth target image, comprising: performing merging processing on the third target image and the fourth target image to obtain a text feature map, the text feature map comprising a text region; obtain a text field region based on the table structure information and coordinate values of the text region in the rectangular coordinate system, the text field region comprising text regions located in a same cell and in a same row; obtain a text field image corresponding to the text field region from the first table image based on coordinate values of the text field region in the rectangular coordinate system; input the text field image into a pre-trained text recognition model to obtain a text recognition result, the text information comprising the text recognition result and the coordinate values of the text field region in the rectangular coordinate system.

9. A form recognition apparatus, characterized by comprising: comprising: a first obtaining module, configured to obtain a first target image, a second target image, a third target image and a fourth target image corresponding to a first table image based on a pre-trained semantic segmentation model; a second obtaining module, configured to obtain table structure information and text information corresponding to the first table image based on the first target image, the second target image, the third target image and the fourth target image; wherein the first target image comprises image information representing horizontal frame lines of the first table image, the second target image comprises image information representing vertical frame lines of the first table image, the third target image comprises image information representing horizontal edges of text of the first table image, and the fourth target image comprises image information representing vertical edges of text of the first table image; the first obtaining module comprises: a rotating unit, configured to rotate a first table image by a target angle along a first direction to obtain a second table image; an obtaining unit, configured to obtain a first segmentation map and a second segmentation map corresponding to the first table image, and obtain a third segmentation map and a fourth segmentation map corresponding to the second table image based on the semantic segmentation model; a third determining unit, configured to determine a first target image, a second target image, a third target image and a fourth target image corresponding to the first table image based on the first segmentation map, the second segmentation map, the third segmentation map and the fourth segmentation map; The first segmentation map comprises image information representing a target frame line of the first table image, the second segmentation map comprises image information representing a character edge of the first table image, the third segmentation map comprises image information representing a target frame line of the second table image, and the fourth segmentation map comprises image information representing a character edge of the second table image. The target frame line comprises the horizontal frame line or the vertical frame line, the horizontal frame line is parallel to the vertical frame line after being rotated by the target angle along the first direction, the character edge is the character horizontal edge when the target frame line is the horizontal frame line, and the character edge is the character vertical edge when the target frame line is the vertical frame line.

10. An electronic device comprising: A memory, a processor, and a program stored in the memory and capable of running on the processor; characterized in that, The processor is configured to read the program in the memory to implement the steps in the method according to any one of claims 1 to 8.

11. A readable storage medium for storing a program, characterized in that, The program, when executed by the processor, implements the steps in the method according to any one of claims 1 to 8.

Images