A table image processing method, device, electronic equipment and medium

Abstract

The present invention provides a table image processing method, device, electronic device and medium, the method comprising: determining that a target table image is a borderless table, processing the target table image through a table recognition structure to obtain a first border frame, and adding the content in the target table image to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image into an editable table, the table information obtained based on the table image itself is more comprehensive and clear, the processing speed of the table image is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a table image processing method, device, electronic equipment and medium. Background Art

[0002] With the continuous development of office software, people's requirements for office software processing are getting higher and higher. They hope that while meeting normal office requirements, they can also meet different application scenarios.

[0003] At present, in practical applications, users may only get table images, such as table images searched from the Internet, but not editable tables. In this case, it is necessary to convert the table images into editable tables and process the tables accordingly.

[0004] In the prior art, when a table image is converted into an editable table document, the table lines are segmented through image segmentation to obtain the horizontal and vertical lines of the table, thereby generating an editable table. However, this processing method has the defects of poor table image segmentation effect and incomplete lines, resulting in poor processing effect and affecting user experience. Summary of the invention

[0005] Based on the problems existing in the prior art, the present invention proposes a table line generation method, device, electronic device and medium for a borderless table, which can meet the needs of any form of table image and has the advantages of improving the table image processing effect and enhancing the user experience.

[0006] In a first aspect, the present invention provides a method for processing a table image, comprising:

[0007] Determine that the target table image is a borderless table;

[0008] Processing the target table image by a table recognition structure to obtain a first border frame;

[0009] Add the content in the target table image into the first border frame.

[0010] Furthermore, according to the method for processing a table image provided by the present invention, the method further includes:

[0011] The table recognition structure is a line, a sliding window or a graph convolutional neural network model.

[0012] Furthermore, according to the method for processing a table image provided by the present invention, the method further includes:

[0013] In the case where the table recognition structure is the line or the sliding window, the target table image is processed by the table recognition structure to obtain a first border frame, including:

[0014] Moving the table recognition structure along a first direction of the target table image to record a first area not covering the content;

[0015] and / or, moving the table recognition structure along a second direction of the target table image to record a second area not covered by the content;

[0016] A table line is generated in at least one of the first area and the second area to obtain a first border frame.

[0017] Furthermore, according to the method for processing a table image provided by the present invention, the method further includes:

[0018] In the case where the table identification structure is the line, the table identification structure includes a first direction line and a second direction line;

[0019] The process of processing the target table image by using the table recognition structure to obtain a first border frame includes:

[0020] Based on the content distribution in the target table image, translating the first direction line and the second direction line in the target table image to determine a non-intersection area in the target table image;

[0021] Target points are determined in the non-intersection area, and the target points are connected to obtain a first frame.

[0022] Further, according to the table image processing method provided by the present invention, based on the content distribution in the target table image, translating the first direction line and the second direction line in the target table image to determine the non-intersection area in the target table image includes:

[0023] Binarizing the target table image to obtain a first binary image; wherein the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the second value in the first binary image is the pixel corresponding to the non-content distribution area in the target table image;

[0024] Processing pixels in the first binary image according to a first direction to obtain lines in a first direction;

[0025] Processing pixels in the first binary image according to a second direction to obtain lines in a second direction;

[0026] The first direction lines are crossed with the second direction lines, and the area after the crossing that is not covered by the first direction lines or the second direction lines is used as a non-crossing area in the target table image.

[0027] Further, according to the table image processing method provided by the present invention, the step of determining the target point in the non-intersection area includes:

[0028] Determine the contour of the non-intersection area by contour searching;

[0029] According to the outline of the non-intersection area, a coordinate point of the non-intersection area is used as the target point.

[0030] Furthermore, according to the method for processing a table image provided by the present invention, the method further includes:

[0031] In the case where the table identification structure is the sliding window, the table identification structure includes a first direction sliding window and a second direction sliding window;

[0032] The process of processing the target table image by using the table recognition structure to obtain a first border frame includes:

[0033] Based on the content distribution in the target table image, translating the first direction sliding window in the target table image to obtain a first direction frame; translating the second direction sliding window in the target table image to obtain a second direction frame;

[0034] A first direction table line is set based on the first direction frame, and a second direction table line is set based on the second direction frame to obtain a first border frame.

[0035] Furthermore, according to the method for processing a table image provided by the present invention, the method further includes:

[0036] In the case where the table recognition structure is the graph convolutional neural network model, a first border frame is determined according to a positional relationship between a first text box and a second text box in the target table image.

[0037] Furthermore, according to the table image processing method provided by the present invention, before the target table image is processed by the table recognition structure to obtain the first border frame, the method further includes:

[0038] Binarizing the target table image to obtain a second binary image; wherein the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the first value in the second binary image is the pixel in the target table image other than the text character;

[0039] A table identification structure is set for the second binary image.

[0040] Further, according to the table image processing method provided by the present invention, the step of adding the content in the target table image to the first border frame includes:

[0041] Performing text detection on the target table image, and obtaining a text box in the target table image according to the text detection result;

[0042] Setting a text box in the first border frame;

[0043] Fill the content in the text box in the target table image into the text box of the first border frame.

[0044] In a second aspect, the present invention further provides a table image processing device, comprising:

[0045] A determination module, used for determining that the target table image is a borderless table;

[0046] A processing module, used for processing the target table image through a table recognition structure to obtain a first border frame;

[0047] An adding module is used to add the content in the target table image into the first border frame.

[0048] In a third aspect, the present invention further provides an electronic device, comprising: a processor, a memory and a bus, wherein:

[0049] The processor and the memory communicate with each other via the bus;

[0050] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the steps of any of the above table image processing methods.

[0051] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the steps of the method for processing a table image as described in any one of the above items.

[0052] The present invention provides a table image processing method, device, electronic device and medium, the method comprising: determining that a target table image is a borderless table; processing the target table image through a table recognition structure to obtain a first border frame; adding the content in the target table image to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image into an editable table, the table information obtained based on the table image itself is more comprehensive and clear, the processing speed of the table image is improved, and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 It is a flow chart of a method for generating table lines of a borderless table provided by the present invention;

[0055] Figure 2 It is an example diagram of a borderless table provided by the present invention;

[0056] Figure 3 is an example diagram of the first binary image provided by the present invention;

[0057] Figure 4 It is an example diagram of the vertical table line area provided by the present invention;

[0058] Figure 5 It is an example diagram of the horizontal table line area provided by the present invention;

[0059] Figure 6 This is an example diagram of the horizontal table line area and the vertical table line area provided by the present invention after being superimposed;

[0060] Figure 7 is an exemplary diagram of determining the center point of a non-intersection area according to the present invention;

[0061] Figure 8 This is an example diagram obtained by connecting the vertical center points in the present invention;

[0062] Fig. 9 This is an example diagram obtained by connecting the horizontal center points in the present invention;

[0063] Fig.10 This is an exemplary diagram of the present invention in which the center points in both the horizontal and vertical directions are connected;

[0064] Fig.11 is an example diagram of a text detection result provided by the present invention;

[0065] Fig.12 is an example diagram of the second binary image provided by the present invention;

[0066] Fig.13 is an example image after morphological processing provided by the present invention;

[0067] Fig.14 This is an example diagram of a text box after being split provided by the present invention;

[0068] Fig.15 It is a structural schematic diagram of a table line generating device for a borderless table provided by the present invention;

[0069] Fig.16 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Figure 1 A schematic diagram of a process flow of a table image processing method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the table image processing method provided by the present invention comprises the following steps:

[0072] Step 101: Determine that the target table image is a borderless table.

[0073] In this embodiment, the target table image needs to be processed. When it is determined that the target table image is a borderless table, the table image processing method provided by the present invention is used to process the table lines in the borderless table to achieve table line generation for the borderless table.

[0074] It should be noted that the target table image refers to the table image to be processed by the method of the present invention. As the name implies, a table image is a table in an image format, for example, a table is saved in an image format such as jpg, png, etc. A borderless table refers to a table in which all cell lines do not exist, such as Figure 2 The table shown has no table lines or frame lines in all cells, and is a borderless table. It should be noted that a borderless table can be a separate table image or a table image formed by converting a table in a PDF document, which is not specifically limited here.

[0075] It should be noted that in Figure 2 In the borderless table shown, each cell is filled with text content, but in other embodiments, some cells in the borderless table may be blank. For such borderless tables, the method of the present invention may also be used to generate table lines.

[0076] Step 102: Process the target table image through the table recognition structure to obtain a first border frame.

[0077] In this embodiment, the first border frame refers to a frame formed by table lines obtained by restoring the borderless table. In this embodiment, the target table image needs to be processed by a table recognition structure to obtain the first border frame. The specific processing flow may be: the target table image is processed using the table recognition result, and then the table lines corresponding to the borderless table in the target table image are determined according to the processing result to obtain the first border frame. The specific processing flow can be seen in the following embodiment, which will not be described in detail here.

[0078] It should be noted that there is no text data in the first border frame, which is just a frame composed of table lines. Moreover, the first border frame is identified and determined on the target table image, and can accurately restore the table information in the target table image.

[0079] Step 103: Add the content in the target table image to the first border frame.

[0080] In this embodiment, the content in the target table image needs to be added to the first border frame to realize the conversion between the target table image and the editable table. The first border frame is a table frame with table lines. Text recognition is performed on each cell in the target table image, and the text recognition results are filled into the corresponding cells of the first border frame to obtain an editable table.

[0081] It should be noted that after successfully converting the target table image into an editable table with table lines, the text results recognized in each cell in the target table image need to be filled into the cells corresponding to the first border frame to restore the data in the target table image.

[0082] According to the table image processing method provided by the present invention, it is determined that the target table image is a borderless table, the target table image is processed by the table recognition structure to obtain a first border frame, and then the content in the target table image is added to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image into an editable table, and the table information obtained based on the table image itself is more comprehensive and clear, which improves the processing speed of the table image and enhances the user experience.

[0083] Based on any of the above embodiments, in this embodiment, the method also includes: the table recognition structure is a line, a sliding window or a graph convolutional neural network model.

[0084] In this embodiment, the table recognition structure is a line, a sliding window or a graph convolutional neural network model. The line refers to processing the target table image in the form of horizontal lines or vertical lines, the sliding window refers to processing the target table image in the form of closed figures of different shapes, and the graph convolutional neural network model is a processing model obtained after training based on sample data to recognize and process the target table image.

[0085] It should be noted that the input of the graph convolutional neural network model is the location information of each text box (such as the coordinate information of the four points of the text box) and the target table image, and the output is the information of whether a text box and its adjacent text boxes are in the same row or column. For details, please see the detailed introduction of the following embodiment.

[0086] According to the table image processing method provided by the present invention, the table recognition structure can be set to a line, a sliding window or a graph convolutional neural network model, which can realize multiple processing of the target table image, ensure the processing effect of the table image, and improve the user experience.

[0087] Based on any of the above embodiments, in this embodiment, the method further includes: when the table recognition structure is a line or a sliding window, processing the target table image by using the table recognition structure to obtain a first border frame, including:

[0088] Moving the table recognition structure along a first direction of the target table image to record a first area of ​​uncovered content;

[0089] and / or, moving the table recognition structure along a second direction of the target table image to record a second area of ​​uncovered content;

[0090] A table line is generated in at least one of the first area and the second area to obtain a first border frame.

[0091] In this embodiment, it is necessary to move the table recognition structure along the first direction of the target table image to record the first area of ​​uncovered content; it is also necessary to move the table recognition structure along the second direction of the target table image, and then record the second area of ​​uncovered content, determine an area from the first area or the second area to generate a table line, and obtain a first border frame. It should be noted that the first direction refers to the longitudinal direction of the target table image, that is, moving from the top of the target table image to the bottom, or from the bottom to the top; the second direction refers to the lateral direction of the target table image, that is, moving from the left side of the target table image to the right, or from the right to the left; accordingly, the first area is the area obtained by moving along the longitudinal direction, and the second area is the area obtained by moving along the lateral direction.

[0092] It should be noted that, in other embodiments, the first direction and the second direction can also be the arc direction or the diameter direction in the fan, or the direction of a 45-degree angle, etc., and are not limited to the horizontal and vertical directions. Movement in any direction can be achieved according to the actual needs of the user, and no specific limitation is made here.

[0093] It should be noted that in this embodiment, the table recognition structure that moves vertically or horizontally along the target table image can be a line or a sliding window, and the area that needs to be recorded refers to the area without covered content, and the area without covered content refers to a blank area where there is no text data.

[0094] According to the method for processing table images provided by the present invention, by moving the table recognition structure along the first direction of the target table image to record the first area of ​​uncovered content, and / or, moving the table recognition structure along the second direction of the target table image to record the second area of ​​uncovered content, and then generating table lines in at least one of the first area and the second area to obtain a first border frame, it is possible to achieve the purpose of generating table lines in areas in different directions of the target table image and obtaining the first border frame, thereby improving the speed of table image processing.

[0095] Based on any of the above embodiments, in this embodiment, the method further includes: when the table recognition structure is a line, the table recognition structure includes a first direction line and a second direction line;

[0096] The target table image is processed by the table recognition structure to obtain a first border frame, including:

[0097] Based on the content distribution in the target table image, the first direction line and the second direction line are translated in the target table image to determine a non-intersection area in the target table image;

[0098] Target points are determined in each non-intersection area, and the target points are connected to obtain a first frame.

[0099] In this embodiment, the first direction lines are horizontal lines, and the second direction lines are vertical lines. In this embodiment, based on the content distribution in the target table image, the obtained horizontal lines and vertical lines need to be translated to determine the non-intersecting areas in the target table image, and then the target points are determined in each non-intersecting area, and the first border frame can be obtained by connecting all the obtained target points. Among them, the target point can be the center point, or the other two-thirds of the points, etc., and the center point is preferred in this embodiment. It should be noted that the non-intersecting area refers to a blank area, which is an area where there is no text content distribution and is not covered by horizontal lines or vertical lines.

[0100] It should be noted that when the table recognition structure is a line, it is necessary to detect and process the text content in the target table image to obtain multiple text boxes, and then perform corresponding processing on the text boxes to obtain target points and determine the first border frame.

[0101] It should be noted that, in the present embodiment, text detection processing is performed on the target table image to obtain a plurality of text boxes, and then the vertical table line area and the horizontal table line area in the target table image need to be determined based on the plurality of text boxes, and then the vertical table line area and the horizontal table line area obtained are translated to obtain the table line area after translation, and a plurality of non-intersection areas are determined from the table line area, and then the center points of the plurality of non-intersection areas are determined, and a first border frame corresponding to the target table image is generated based on the plurality of center points.

[0102] It should be noted that, in the present embodiment, the longitudinal table line area refers to the area formed by moving the table recognition structure longitudinally along the target table image, that is, the second area in the above embodiment, and the transverse table line area refers to the area formed by moving the table recognition structure horizontally along the target table image, that is, the first area in the above embodiment; multiple non-intersection areas refer to areas that are not covered by the transverse table line area and the longitudinal table line area, nor are they overlapped.

[0103] According to the method for processing table images provided by the present invention, based on the text distribution in the target table image, the first direction lines and the second direction lines obtained by translating in the target table image are used to determine the non-intersection area in the target table image, and then the target points are determined in each non-intersection area, and the target points are connected to obtain a first border frame. The operation is simple and can ensure the accuracy of the restoration processing of the table image.

[0104] Based on any of the above embodiments, in this embodiment, based on the content distribution in the target table image, translating the first direction lines and the second direction lines in the target table image to determine the non-intersection area in the target table image includes:

[0105] Binarizing the target table image to obtain a first binary image; wherein the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the second value in the first binary image is the pixel corresponding to the non-content distribution area in the target table image;

[0106] Processing pixels in the first binary image according to a first direction to obtain lines in a first direction;

[0107] Processing pixels in the first binary image according to a second direction to obtain lines in a second direction;

[0108] The first direction lines are crossed with the second direction lines, and the area after the crossing that is not covered by the first direction lines or the second direction lines is used as a non-crossing area in the target table image.

[0109] In this embodiment, it is necessary to perform binarization processing on the target table image after the text box is determined, and obtain a first binary image in which the pixels corresponding to the text box are the first value and the pixels outside the text box are the second value, such as Figure 3 The first binary image shown in FIG. 1 , wherein the first value refers to the value of a pixel being 0, the second value refers to the value of a pixel being 255, and the first binary image refers to the binary image generated by binarizing the text box area.

[0110] In this embodiment, the first direction may refer to the longitudinal direction of the target table image. To determine the longitudinal lines, a pixel column of a certain width needs to be selected, and the pixel column and the text box are closed, and the pixel points corresponding to the text box in the longitudinal direction are connected to form longitudinal lines, thereby obtaining a longitudinal table line area, such as Figure 4 The vertical table line area shown in the figure, where the pixel value corresponding to the vertical table line is 0. It should be noted that the closing operation is a morphological calculation method, specifically a calculation method of dilation followed by erosion, where dilation is the process of merging all background points in contact with the object into the object to expand the boundary outward to fill the holes in the object. Erosion is a process of eliminating boundary points and shrinking the boundary inward, which can be used to eliminate small and meaningless objects.

[0111] In this embodiment, the second direction may refer to the horizontal direction of the target table image. For determining the horizontal lines, pixel rows of a certain width are selected and closed with the text box to obtain horizontal lines. All horizontal lines constitute the horizontal table area, such as Figure 5 The horizontal table line area shown.

[0112] It should be noted that in this embodiment, the obtained horizontal table line area and vertical table line area need to be translated to obtain the translated table line area. The translated table line area can be as follows: Figure 6 As shown, from Figure 6 It can be determined that the area that is not overlapped and not covered by the vertical table line area or the horizontal table line area is determined as the non-intersection area, that is, Figure 6 The blank area shown in , that is, the blank area is a non-intersection area.

[0113] According to the method for processing a table image provided by the present invention, a target table image is binarized to obtain a first binary image, and then pixels in the first binary image are processed in a first direction to obtain first direction lines, and pixels in the first binary image are processed in a second direction to obtain second direction lines, and the first direction lines and the second direction lines are translated, and areas that are not crossed and not covered by the first direction lines or the second direction lines are used as non-crossing areas in the target table image, thereby improving the processing speed of the table image.

[0114] Based on any of the above embodiments, in this embodiment, determining a target point in each non-intersection area includes:

[0115] Determine the contours of each non-intersecting area by contour finding;

[0116] According to the contours of each non-intersection area, the target point in each non-intersection area is determined.

[0117] In this embodiment, the coordinates of the target points of each non-intersecting area need to be determined by contour search. First, the contours of multiple non-intersecting areas are determined, and then the coordinates of the contour target points of the multiple non-intersecting areas are determined according to the contours of the multiple non-intersecting areas, and the first border frame is obtained according to the coordinate information of the target points. It should be noted that the target point in this embodiment is set as the center point. Since the text box is generally a quadrilateral, the center point of the text box can be determined by determining the coordinates of the four vertices, such as Figure 7 As shown, each small circle is the center point of each determined text box.

[0118] For example, if a non-intersection area is obtained, the coordinates of the four vertices of the non-intersection area are found through contour search, which are (2, 6) and (2, 9), (8, 6) and (8, 9), and the coordinates of the center point (5, 7.5) are calculated. Then, the multiple center points determined are connected through the above search method to obtain the table line corresponding to the target table image.

[0119] It should be noted that connecting the center points in the longitudinal direction can obtain the following Figure 8 The area formed by the vertical lines shown in the figure is obtained by connecting multiple center points in the horizontal direction. Fig. 9 The area formed by the horizontal lines shown in the figure is obtained by translating the obtained longitudinal connection line area and the transverse connection line area. Fig.10 The table lines shown obtain the first border frame of the borderless table.

[0120] It should be noted that, in this embodiment, the first border frame is generated on the original target table image. In other embodiments, mapping points of multiple center points may be connected vertically and / or horizontally in a preset area according to the coordinates of the determined center point, and all mapping points may be connected to generate a table line corresponding to the target table image.

[0121] According to the table image processing method provided by the present invention, the contour of each non-intersection area is determined by contour search, and then the target point in each non-intersection area is determined according to the contour of each non-intersection area. The first border frame of the target table image can be generated by means of the center point, thereby improving the accuracy of the generation of the first border frame.

[0122] Based on any of the above embodiments, in this embodiment, when the table recognition structure is a sliding window, the table recognition structure includes a first direction sliding window and a second direction sliding window;

[0123] The target table image is processed by the table recognition structure to obtain a first border frame, including:

[0124] Based on the content distribution in the target table image, the sliding window in the first direction is translated in the target table image to obtain a first direction frame; the sliding window in the second direction is translated in the target table image to obtain a second direction frame;

[0125] A first direction table line is set based on the first direction frame, and a second direction table line is set based on the second direction frame to obtain a first border frame.

[0126] In this embodiment, it is necessary to translate the first sliding window and the second sliding window based on the distribution of text content in the target table image to obtain a first direction frame and a second direction frame, and then set a first direction table line based on the first direction frame, set a second direction table line based on the second direction frame, and determine the first border frame based on the table lines obtained after the translation.

[0127] It should be noted that, in the present embodiment, the first direction frame can be a frame along the longitudinal direction of the target table image, and can be a frame of any shape, such as a rectangle, square, etc.; the second direction frame is a frame along the transverse direction of the target table image, and can also be a frame of any shape; in other embodiments, the first direction frame and the second direction frame can also be frames in any direction on the target table image, such as a frame in the direction of 45 degrees, and can be set according to the actual needs of the user, and no specific limitation is made here.

[0128] It should be noted that the first direction frame is a frame obtained by sliding the first direction sliding window along the first direction of the target table image; the second direction frame is a frame obtained by sliding the second sliding window along the second direction of the target table image; the first direction frame and the second direction frame in this embodiment refer to blank areas where there is no text data content, that is, the first border frame obtained after translation according to the first direction frame and the second direction frame is an area where there is no text data content.

[0129] It should be noted that the first direction frame and the second direction frame obtained in this embodiment can generate table lines. The first direction table lines are set based on the first direction frame, and the second direction table lines are set based on the second direction frame. After completing the table line generation of the first direction frame and the second direction frame, the first border frame can be accurately obtained.

[0130] According to the method for processing a table image provided by the present invention, based on the content distribution in the target table image, a first direction sliding window is translated in the target table image to obtain a first direction frame, a second direction sliding window is translated in the target table image to obtain a second direction frame, a first direction table line is set based on the first direction frame, and a second direction table line is set based on the second direction frame to obtain a first border frame. The first border frame can be accurately obtained, thereby improving the processing speed of the table image.

[0131] Based on any of the above embodiments, in this embodiment, based on the content distribution in the target table image, translating the first direction sliding window in the target table image to obtain the first direction frame; translating the second direction sliding window in the target table image to obtain the second direction frame includes:

[0132] In the process of sliding the first-direction sliding window along a direction perpendicular to the first direction, determining a sliding range of the first-direction sliding window that does not intersect with the content according to the content distribution in the target table image, and determining a first-direction frame according to the shape of the first-direction sliding window and the sliding range of the first-direction sliding window that does not intersect with the content;

[0133] During the process of the second-direction sliding window sliding along a direction perpendicular to the second direction, the sliding range of the second-direction sliding window that does not intersect with the content is determined according to the text distribution in the target table image, and the second direction frame is determined according to the shape of the second-direction sliding window and the sliding range of the second-direction sliding window that does not intersect with the content.

[0134] In this embodiment, a first direction sliding window needs to be set on the target table image. When the first direction sliding window slides in a direction perpendicular to the first direction, the sliding range of the first direction sliding window that does not intersect with the text content is determined according to the distribution of the text content in the target table image, and then the first direction frame is determined according to the shape and sliding range of the first direction sliding window. The first direction sliding window refers to a sliding window set in the longitudinal direction of the target table image, which can be a rectangular sliding window or a square sliding window, which is not specifically limited here.

[0135] It should be noted that in this embodiment, the first direction is the longitudinal direction of the target table image. When the first direction sliding window is set to slide in a direction perpendicular to the first direction, the sliding range that does not intersect with the text is determined to be the horizontal range area, that is, the first direction frame obtained is a horizontal frame. In other embodiments, the first direction sliding window can also slide along the first direction, and the first direction frame obtained is a vertical frame. It can be set according to the actual needs of the user and is not specifically limited here.

[0136] In this embodiment, a second direction sliding window is also required to be set on the target table image. When the second direction sliding window slides in a direction perpendicular to the second direction, the sliding range of the second direction sliding window that does not intersect with the text content is determined according to the distribution of the text content in the target table image, and the second direction frame is determined according to the shape of the second direction sliding window and the sliding range of the second direction sliding window that does not intersect with the text content. The second direction sliding window refers to a sliding window set in the horizontal direction of the target table image, which can be a rectangular sliding window or a square sliding window, which is not specifically limited here.

[0137] It should be noted that, in this embodiment, the second direction is the horizontal direction of the target table image. When the second direction sliding window is set to slide in a direction perpendicular to the second direction, the sliding range that does not intersect with the text content is determined to be the vertical range area, that is, the second direction frame obtained is a vertical frame. In other embodiments, the second direction sliding window can also slide along the second direction, and the second direction frame obtained is a horizontal frame. It can be set according to the actual needs of the user and is not specifically limited here.

[0138] It should be noted that the setting of the first direction sliding window and the second direction sliding window can be calculated and designed using a more mature sliding window algorithm in the prior art, and the specific setting method will not be introduced in detail here.

[0139] According to the method for processing table images provided by the present invention, a sliding window is set on the target table image, and then a sliding range that does not intersect with the text content is determined during the sliding of the sliding window. A first direction frame and a second direction frame are determined according to the shape of the sliding window and the sliding range. The operation is simple and the processing speed of the table image is improved.

[0140] Based on any of the above embodiments, in this embodiment, when the table recognition structure is a graph convolutional neural network model, a first border frame is determined based on the positional relationship between text boxes and text boxes in the target table image.

[0141] In this embodiment, when the table recognition structure is a graph convolutional neural network model, it is necessary to determine the first border frame based on the positional relationship between the text boxes in the target table image. It should be noted that the positional relationship between the text boxes can be confirmed based on the coordinate information of the center point of the text box.

[0142] It should be noted that according to the output of the graph convolutional neural network model provided in this embodiment, the text boxes in the same row and the text boxes in the same column can be determined, and the height of the cell can be determined according to the height value of the text boxes in the same row, and the width of the cell can be determined according to the width value of the text boxes in the same column, that is, the table lines of the borderless table are drawn. A detailed introduction can be seen in the following embodiment.

[0143] According to the table image processing method provided by the present invention, when the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between text boxes in the target table image. The trained graph convolutional neural network model can accurately and quickly determine the positional relationship between the text boxes in the target table image to obtain the first border frame.

[0144] Based on any of the above embodiments, in this embodiment, when the table recognition structure is a graph convolutional neural network model, determining a first border frame according to a positional relationship between text boxes in a target table image includes:

[0145] Inputting the position information of multiple text boxes in the pre-acquired target table image and the target table image into a pre-trained graph convolutional neural network model to obtain the positional relationship between any text box in the multiple text boxes and its adjacent text boxes; wherein the graph convolutional neural network model is trained based on the position information of multiple text boxes in the sample table, the sample table image and the label information of the sample table;

[0146] Determine the text boxes of each row and each column in the target table image according to the positional relationship between any text box in the plurality of text boxes and its adjacent text box;

[0147] A first border frame of the target table image is generated according to the text boxes of each row and each column in the target table image.

[0148] In this embodiment, the position information of multiple text boxes in the target table image and the target table image need to be input into a pre-trained graph convolutional neural network model to obtain the positional relationship between any text box and its adjacent text box, and then determine the text boxes of each row and each column in the target table image based on the obtained positional relationship, and generate a first border frame corresponding to the target table image. The graph convolutional neural network model is an artificial intelligence model, which is trained based on the position information of multiple text boxes in the sample table, the sample table image, and the label information of the sample table.

[0149] In this embodiment, the input of the graph convolutional neural network model is the position information of each text box (such as the coordinate information of the four points of the text box) and the target table image, and the output is the information of whether a text box and its adjacent text boxes are in the same row or column.

[0150] For example, if the determined position information of text box a is (1, 1), the position information of the adjacent text boxes is calculated and analyzed as follows: a1(0, 1), a2(0, 1), a3(1, 0), a4(1, 0). The first number in the position information indicates whether it is in the same row as text box a. If 1 indicates the same row and 0 indicates a different row, then the adjacent text boxes a1 and a2 are in the same column as text box a. The second number indicates whether it is in the same column as text box a. If 1 indicates the same column and 0 indicates a different column, the adjacent text boxes a1 and a2 are in the same column as text box a, while the adjacent text boxes a3 and a4 are in the same row as text box a. The specific setting can be based on the actual needs of the user and is not specifically limited here.

[0151] It should be noted that according to the output of the graph convolutional neural network model provided in this embodiment, the text boxes in the same row and the text boxes in the same column can be determined, and the height of the cell can be determined according to the height value of the text boxes in the same row, and the width of the cell can be determined according to the width value of the text boxes in the same column, that is, the table lines of the borderless table can be drawn.

[0152] According to the table image processing method provided by the present invention, the position information of multiple text boxes and the target table image are input into a pre-trained rare graph convolutional neural network model to obtain the positional relationship between any text box and its adjacent text box, and then the text rows of each row and the text boxes of each column are determined according to the positional relationship, and the first border frame corresponding to the target table image is generated according to the text boxes of each row and column. By generating the first border frame corresponding to the target table image through the positional relationship of multiple text boxes, the accuracy of table line generation can be improved.

[0153] Based on any of the above embodiments, in this embodiment, generating a first border frame of the target table image according to the text boxes of each row and each column in the target table image includes:

[0154] According to the height target values ​​of each text box in the target row of the target table image, the height value of the target row is set; wherein the target row is any row of the target table image;

[0155] According to the width target values ​​of each text box in the target column of the target table image, the width value of the target column is set; wherein the target column is any column of the target table image;

[0156] Draw the table line of the target row according to the height value of the target row and the position of the target row in the target table image; and / or draw the table line of the target column according to the width value of the target column and the position of the target column in the target table image to obtain the first border frame of the target table image.

[0157] In this embodiment, it is necessary to set the height value of the target row according to the height target value of each text box in the target row, set the width value of the target column according to the width target value of each text box in the target column, and then draw the table line of the row corresponding to the target row according to the height value of the target row and the position information of the target row in the target table image, draw the table line of the column corresponding to the target column according to the width value of the target column and the position information of the target column in the target table image, and obtain the first border frame according to the table lines generated for each row and each column.

[0158] It should be noted that, in this embodiment, the maximum height value of each text box in the target row is determined as the height value of the target row, or the maximum width corresponding to each text box in the target column is set as the width value of the target column. In other embodiments, the average height value of each text box in the target row can be selected as the height value of the target row, and the average width value of each text box in the target column can be selected as the width value of the target column. Specific users can also set it according to actual needs, which is not specifically limited here.

[0159] It should be noted that the position information of the target row or target column may be its coordinate information, or may refer to the positional relationship with any other adjacent rows or columns, which may be set according to actual needs and is not specifically limited here.

[0160] For example, if the height values ​​of multiple text boxes in the target row are 4cm, 3cm, and 5cm respectively, the maximum height value of the multiple text boxes in the target row is determined as the height value of the target row, that is, 5cm is determined as the height value of the target row; similarly, the maximum width value in the target column is determined as the width value of the target column, and the above method is used to confirm the height value or width value for other rows or columns in turn, which will not be introduced in detail here.

[0161] It should be noted that the pixel of each text box can also be used as a unit of measurement to determine the height value of the target row or the width value of the target column. For example, if the maximum pixel of the target text box in a target row is 1024, and the value corresponding to the pixel is 8.67 cm through calculation and analysis, then the 8.67 cm corresponding to the pixel is determined as the height value of the target row. In other embodiments, other numerical determination methods can also be used. Different determination methods can be selected according to the specific needs of the user, and are not specifically limited here.

[0162] According to the table image processing method provided by the present invention, the height target value in each text box in the target row is determined as the height value of the target row, and the width target value of each text box in the target column is determined as the width value of the target column. Then, according to the height value of the target row and the position of the target row in the target table image, the width value of the target column, and the position of the target column in the target table image, the table line corresponding to the target row or target column is drawn, and the first border frame is obtained, which can ensure the accuracy of the table line generation and make the generated table line complete and clear.

[0163] Based on any of the above embodiments, in this embodiment, before the target table image is processed by the table recognition structure to obtain the first border frame, the method further includes:

[0164] The target table image is binarized to obtain a second binary image; wherein the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the first value in the second binary image is the pixel in the target table image except the text character.

[0165] A table recognition structure is set for the second binary image.

[0166] In this embodiment, it is necessary to perform binarization processing on the target table image containing the text detection result to obtain a second binary image in which the pixels corresponding to the text characters are the second value and the pixels excluding the text characters are the first value, wherein the first value refers to the value of the pixel being 0 and the second value refers to the value of the pixel being 255. Then, morphological processing is performed on the obtained second binary image to obtain a morphological processing result, and a text box with a gap in the middle is determined in the processing result, and the text box is split to obtain multiple text boxes in the target table image.

[0167] It should be noted that, in this embodiment, it is also necessary to set a table recognition structure for the obtained second binary image, wherein the table recognition structure can be a line, a frame or a graph convolutional neural network model, which can be set according to user needs and is not specifically limited here.

[0168] According to the table image processing method provided by the present invention, by binarizing the target table image containing the text detection results, a second binary image is obtained in which the pixels corresponding to the text characters are the first value and the pixels other than the text characters in the target table image are the second value. Then, a table recognition structure is set for the second binary image, which provides support for the subsequent determination of the first border frame according to the table recognition structure.

[0169] Based on any of the above embodiments, in this embodiment, adding the content in the target table image to the first border frame includes:

[0170] Performing text detection on the target table image, and obtaining a text box in the target table image according to the text detection result;

[0171] Setting a text box in the first border frame;

[0172] Fill the content in the text box in the target table image into the text box of the first border frame.

[0173] In this embodiment, it is necessary to perform text detection on the target table image, and then obtain the text box in the target table image based on the text detection result, set multiple text boxes in the first border frame, and fill the data content in the multiple text boxes in the target table image into the corresponding text boxes in the first border frame to realize data restoration in the target table image.

[0174] In this embodiment, if Fig.11 As shown, a text detection unit is used to perform text detection processing on a borderless table, multiple text contents are identified, and multiple text boxes in the target table image are determined according to the file detection result, wherein a rectangular frame represents a text box. Fig.11 There are multiple text boxes in .

[0175] Specifically, after detecting the text in the target table image, the text detection results need to be binarized and morphologically processed to obtain the text box in the target table image. Image binarization is the process of setting the grayscale value of the pixel points on the image to 0 or 255, which is actually the process of making the entire image appear in obvious black and white. Fig.12 The figure is a schematic diagram of the binarization of multiple text boxes obtained by the above text detection. Morphology, mathematical morphology, is one of the most widely used technologies in image processing. It is mainly used to extract image components that are meaningful for expressing and depicting the shape of the region from the image, so that the subsequent recognition work can grasp the most essential shape features of the target object. Among them, the most essential refers to the most discriminative, such as boundaries and connected regions, etc. Fig.13 As shown, Fig.13 This is a schematic diagram of a table image after binarization and morphological processing. The specific processing process is shown in the following embodiment and will not be described in detail here.

[0176] It should be noted that, in this embodiment, after determining multiple text boxes in the target table image, the text boxes with gaps in the middle are split, such as Fig.14 This is a schematic diagram obtained after splitting a text box with a gap in the middle. For example, a single text box "Zhang San, General Manager 123456937" is split into three text boxes according to the existing gap, namely "Zhang San", "General Manager", and "123456937".

[0177] In this embodiment, it is necessary to perform binarization processing on the multiple text boxes obtained after splitting, confirm the horizontal table line area and the vertical table line area, and then determine the center point of each non-intersection area based on the determined table line area, and connect each center point to determine the first border frame corresponding to the target table image. The details can be found in the following embodiments and will not be introduced in detail in this embodiment.

[0178] It should be noted that in the present embodiment, the horizontal table area and the vertical table area are determined, and then the center point of the translated table line area is determined, and the table lines corresponding to the target table image are generated by connecting the center points. In other embodiments, other generation methods may be used, such as generating table lines based on the positional relationship of text boxes, or other generation methods, which are not specifically limited here.

[0179] It should be noted that, in this embodiment, it is necessary to set up multiple text boxes in the obtained first border frame, and then add the text content in the multiple text boxes in the target table image to the corresponding text boxes in the first border frame, wherein the more mature text recognition method in the prior art is used to recognize the text content in the multiple text boxes in the target table image, and then fill the recognition results into the corresponding text boxes in the first border frame. The specific process of text recognition processing will not be introduced in detail here.

[0180] According to the table image processing method provided by the present invention, by performing text detection on the target table image, a text box in the target table image is obtained according to the text detection result, and then a text box is set in the first border frame, and the content in the text box in the target table image is filled into the text box of the first border frame. The text data in the table image can be accurately restored, and the processing speed of the table image is improved.

[0181] Based on any of the above embodiments, in this embodiment, determining that the target table image is a borderless table includes:

[0182] Input the target table image into the pre-trained table classification model;

[0183] According to the output result of the table classification model, it is determined that the target table image is a borderless table;

[0184] Among them, the table classification model is trained based on the sample table and the category labels of the sample table.

[0185] In this embodiment, the target table image needs to be input into a pre-trained table classification model, and then the target table image is determined to be a borderless table based on the output result of the table classification model, wherein the table classification model is trained based on a sample table and the category labels of the sample table, and there are two output results, one is a borderless table and the other is a bordered table. The target table image is determined to be a borderless table only when there are no table lines in each cell in the target table image.

[0186] It should be noted that the table classification model can be a VGG model or a Resnet model. The table classification model is obtained by training the selected model to identify the type of the target table image. Among them, the VGG model is a model that can be used for identification and confirmation in multiple transfer learning tasks. The VGG model is the preferred algorithm for extracting CNN features from images. The Resnet model refers to a deep residual network model that achieves very high accuracy image recognition, speech recognition and other capabilities through a very deep network.

[0187] For example, if the acquired target table image 1 and target table image 2 are input into a pre-trained table classification model, the model output result is that there are no border lines in the target table image 1, and there are table frame lines in some cells of the target table image 2. The target table image 1 is determined as a borderless table, and the target table image 2 is determined as a semi-border table.

[0188] According to the table image processing method provided by the present invention, the type of the target table image can be determined through a pre-trained table classification model, and the target table image can be determined as a borderless table according to the output result of the table classification model. The type of the target table image can be accurately identified, thereby improving the subsequent processing speed of the borderless table.

[0189] Based on any of the above embodiments, in this embodiment, the content in the first border frame is in an editable state.

[0190] In this embodiment, the content in the first border frame obtained by converting the target table image is in an editable state. According to the table image processing method provided above, the target table image of the borderless table type is converted and processed to obtain a first border frame with table lines, and then text recognition is required for each cell in the target table image, and the text recognition results are filled into the corresponding cells of the first border frame to obtain an editable table.

[0191] According to the method for processing a table image provided by the present invention, the content within the first border frame is in an editable state, which meets the editing needs of the user and improves the user experience.

[0192] Based on any of the above embodiments, in this embodiment, a table classification model is first used to confirm that the target table image is a borderless table, and then the borderless table is subjected to text detection processing according to a text detection unit to obtain multiple text boxes, and the obtained target table image with multiple text boxes is binarized and morphologically processed, and then according to the processing results, the text boxes with intermediate spaces are split to obtain a target table image with multiple text boxes after splitting.

[0193] In this embodiment, it is also necessary to generate table lines according to multiple text boxes of the target table image, first perform binarization processing to obtain a first binary image, extract the vertical table line area in the vertical direction and the horizontal table line area in the horizontal direction from the first binary image, wherein the vertical table line and the horizontal table line are given templates. Then perform translation processing on the two obtained areas, and set the white area on the translated area as the non-intersection area.

[0194] In this embodiment, the contour of each non-intersecting area is determined from the translated area by contour search, the center point of the contour is calculated, the coordinate information of the non-intersecting area is determined, and the coordinate of the center point is determined according to the average value of the coordinate values ​​of the vertices in each direction. The center points are connected horizontally and vertically to generate the table lines of the target table image, and an editable first border frame is obtained.

[0195] It should be noted that in this embodiment, the positional relationship between text boxes can also be used to determine the table lines. For example, a graph convolutional network model is constructed based on the positional relationship between text boxes, and the trained graph convolutional network model is used to determine the text boxes in the same column and the same row. After the determination, the table lines that completely surround all the text boxes in the target table image are determined based on the positional relationship between each row and each column.

[0196] Fig.15 A table image processing device provided by the present invention, such as Fig.15 As shown, the table image processing device provided by the present invention includes:

[0197] A determination module 1501 is used to determine that the target table image is a borderless table;

[0198] A processing module 1502 is used to process the target table image through a table recognition structure to obtain a first border frame;

[0199] The adding module 1503 is used to add the content in the target table image into the first border frame.

[0200] According to the table image processing device provided by the present invention, it is determined that the target table image is a borderless table, the target table image is processed by the table recognition structure to obtain a first border frame, and then the content in the target table image is added to the first border frame. The table image processing method provided by the present invention can quickly and effectively obtain the first border frame, accurately realize the conversion of the table image to an editable table, and the table information obtained based on the table image itself is more comprehensive and clear, which improves the processing speed of the table image and enhances the user experience.

[0201] Furthermore, the table recognition structure is a line, a sliding window or a graph convolutional neural network model.

[0202] According to the table image processing device provided by the present invention, the table recognition structure can be set to a line, a sliding window or a graph convolutional neural network model, which can realize multiple processing of the target table image, ensure the processing effect of the table image, and improve the user experience.

[0203] Further, the processing module 1502 is also used for:

[0204] Moving the table recognition structure along a first direction of the target table image to record a first area of ​​uncovered content;

[0205] and / or, moving the table recognition structure along a second direction of the target table image to record a second area of ​​uncovered content;

[0206] A table line is generated in at least one of the first area and the second area to obtain a first border frame.

[0207] According to the table image processing device provided by the present invention, by moving the table recognition structure along the first direction of the target table image to record the first area of ​​uncovered content, and / or, moving the table recognition structure along the second direction of the target table image to record the second area of ​​uncovered content, and then generating table lines in at least one of the first area and the second area to obtain a first border frame, it is possible to achieve the purpose of generating table lines in areas in different directions of the target table image and obtaining the first border frame, thereby improving the speed of table image processing.

[0208] Furthermore, in the case where the table recognition structure is a line, the table recognition structure includes a horizontal line and a vertical line, and the processing module 1502 is further used for:

[0209] Based on the content distribution in the target table image, translating the first direction line and the second direction line in the target table image to determine a non-intersection area in the target table image;

[0210] Target points are determined in the non-intersection area, and the target points are connected to obtain a first bounding box frame.

[0211] According to the table image processing device provided by the present invention, based on the text distribution in the target table image, the first direction lines and the second direction lines are superimposed in the target table image to determine the non-intersection area in the target table image, and then the target points are determined in each non-intersection area, and the target points are connected to obtain a first border frame. The operation is simple and can ensure the accuracy of the restoration processing of the table image.

[0212] Further, the processing module 1502 is also used for:

[0213] Binarizing the target table image to obtain a first binary image; wherein the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the second value in the first binary image is the pixel corresponding to the non-content distribution area in the target table image;

[0214] Processing pixels in the first binary image according to a first direction to obtain lines in a first direction;

[0215] Processing pixels in the first binary image according to a second direction to obtain lines in a second direction;

[0216] The first direction lines are crossed with the second direction lines, and the area after the crossing that is not covered by the first direction lines or the second direction lines is used as a non-crossing area in the target table image.

[0217] According to the table image processing device provided by the present invention, a first binary image is obtained by binarizing the target table image, and then pixels in the first binary image are processed according to a first direction to obtain first direction lines, and pixels in the first binary image are processed according to a second direction to obtain second direction lines, and the first direction lines are crossed with the second direction lines, and the crossed area not covered by the first direction lines or the second direction lines is used as a non-crossing area in the target table image, thereby improving the processing speed of the table image.

[0218] Further, the processing module 1502 is also used for:

[0219] Determine the contour of the non-intersection area by contour searching;

[0220] According to the outline of the non-intersection area, a coordinate point of the non-intersection area is used as the target point.

[0221] According to the table image processing device provided by the present invention, the contours of each non-overlapping area are determined by contour search, and then the target points in each non-intersecting area are determined according to the contours of each non-intersecting area. The first border frame of the target table image can be generated by means of the center point, thereby improving the accuracy of the generation of the first border frame.

[0222] Further, in the case where the table identification structure is a frame, the table identification structure includes a first direction frame and a second direction frame;

[0223] Accordingly, the processing module 1502 is further used for:

[0224] Based on the content distribution in the target table image, the sliding window in the first direction is translated in the target table image to obtain a first direction frame; the sliding window in the second direction is translated in the target table image to obtain a second direction frame;

[0225] A first direction table line is set based on the first direction frame, and a second direction table line is set based on the second direction frame to obtain a first border frame.

[0226] According to the processing device of the table image provided by the present invention, based on the content distribution in the target table image, a first direction sliding window is translated in the target table image to obtain a first direction frame; a second direction sliding window is translated in the target table image to obtain a second direction frame; a first direction table line is set based on the first direction frame, and a second direction table line is set based on the second direction frame to obtain a first border frame. The first border frame can be obtained accurately, thereby improving the processing speed of the table image.

[0227] Further, the processing module 1502 is also used for:

[0228] In the process of sliding the first-direction sliding window along a direction perpendicular to the first direction, determining a sliding range of the first-direction sliding window that does not intersect with the text according to the content distribution in the target table image, and determining a first-direction frame according to the shape of the first-direction sliding window and the sliding range of the first sliding window that does not intersect with the text;

[0229] During the process of the second-direction sliding window sliding along a direction perpendicular to the second direction, the sliding range of the second-direction sliding window that does not intersect with the text is determined according to the text distribution in the target table image, and the second direction frame is determined according to the shape of the second-direction sliding window and the sliding range of the second sliding window that does not intersect with the text.

[0230] According to the table image processing device provided by the present invention, a sliding window is set on the target table image, and then a sliding range that does not intersect with the text content is determined during the sliding of the sliding window. The first direction frame and the second direction frame are determined according to the shape and sliding range of the sliding window. The operation is simple and the processing speed of the table image is improved.

[0231] Further, in the case where the table recognition structure is a graph convolutional neural network model, a first border frame is determined based on a positional relationship between text boxes in the target table image.

[0232] According to the table image processing device provided by the present invention, when the table recognition structure is a graph convolutional neural network model, the first border frame is determined according to the positional relationship between text boxes in the target table image. The trained graph convolutional neural network model can accurately and quickly determine the positional relationship between the text boxes in the target table image to obtain the first border frame.

[0233] Further, the processing module 1502 is also used for:

[0234] Inputting the position information of multiple text boxes in the pre-acquired target table image and the target table image into a pre-trained graph convolutional neural network model to obtain the positional relationship between any text box in the multiple text boxes and its adjacent text boxes; wherein the graph convolutional neural network model is trained based on the position information of multiple text boxes in the sample table, the sample table image and the label information of the sample table;

[0235] Determine the text boxes of each row and each column in the target table image according to the positional relationship between any text box in the plurality of text boxes and its adjacent text box;

[0236] A first border frame of the target table image is generated according to the text boxes of each row and each column in the target table image.

[0237] According to the table image processing device provided by the present invention, the position information of multiple text boxes and the target table image are input into a pre-trained rare graph convolutional neural network model to obtain the positional relationship between any text box and its adjacent text box, and then the text rows of each row and the text boxes of each column are determined according to the positional relationship, and the first border frame corresponding to the target table image is generated according to the determined text boxes of each row and column. By generating the first border frame corresponding to the target table image through the positional relationship of multiple text boxes, the accuracy of table line generation can be improved.

[0238] Further, the processing module 1502 is also used for:

[0239] According to the height target values ​​of each text box in the target row of the target table image, the height value of the target row is set; wherein the target row is any row of the target table image;

[0240] According to the width target values ​​of each text box in the target column of the target table image, setting the width value of the target column; wherein the target column is any column of the target table image;

[0241] Draw the table line of the target row according to the height value of the target row and the position of the target row in the target table image; and / or draw the table line of the target column according to the width value of the target column and the position of the target column in the target table image to obtain the first border frame of the target table image.

[0242] According to the table image processing device provided by the present invention, the height target value in each text box in the target row is determined as the height value of the target row, and the width target value of each text box in the target column is determined as the width value of the target column. Then, according to the height value of the target row and the position of the target row in the target table image, the width value of the target column, and the position of the target column in the target table image, the table line corresponding to the target row or target column is drawn, and the first border frame is obtained, which can ensure the accuracy of the table line generation and make the generated table line complete and clear.

[0243] Furthermore, the table image processing device is also used for:

[0244] The target table image is binarized to obtain a second binary image; wherein the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the first value in the second binary image is the pixel in the target table image except the text character.

[0245] A table identification structure is set for the second binary image.

[0246] According to the table image processing device provided by the present invention, by binarizing the target table image containing the text detection result, a second binary image is obtained in which the pixels corresponding to the text characters are the first value and the pixels other than the text characters in the target table image are the second value, and then a table recognition structure is set for the second binary image, which provides support for the subsequent determination of the first border frame according to the table recognition structure.

[0247] Further, the adding module 1503 is also used for:

[0248] Performing text detection on the target table image, and obtaining a text box in the target table image according to the text detection result;

[0249] Setting a text box in the first border frame;

[0250] Fill the content in the text box in the target table image into the text box of the first border frame.

[0251] According to the table image processing device provided by the present invention, by performing text detection on the target table image, multiple text boxes in the target table image are obtained according to the text detection result, multiple text boxes are set in the first border frame, and the contents of the multiple text boxes in the target table image are filled into the multiple text boxes in the first border frame. The text data in the table image can be accurately restored, and the processing speed of the table image is improved.

[0252] Further, the determination module 1501 is further used for:

[0253] Input the target table image into the pre-trained table classification model;

[0254] According to the output result of the table classification model, determining that the target table image is a borderless table;

[0255] The table classification model is obtained by training based on sample tables and category labels of the sample tables.

[0256] According to the table image processing device provided by the present invention, the type of the target table image can be determined through a pre-trained table classification model, and the target table image can be determined as a borderless table according to the output result of the table classification model. The type of the target table image can be accurately identified, thereby improving the processing speed of subsequent borderless tables.

[0257] Furthermore, the content in the first border frame is in an editable state.

[0258] According to the table image processing device provided by the present invention, the content in the first border frame is in an editable state, which meets the editing needs of the user and improves the user experience.

[0259] Since the principle of the device described in the embodiment of the present invention is the same as that of the method described in the above embodiment, a more detailed explanation will not be repeated here.

[0260] Fig.16 FIG. 1 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Fig.16 As shown, the present invention provides an electronic device, including: a processor (processor) 1601, a memory (memory) 1602 and a bus 1603;

[0261] The processor 1601 and the memory 1602 communicate with each other via the bus 1603;

[0262] Processor 1601 is used to call program instructions in memory 1602 to execute the methods provided in the above-mentioned method embodiments, for example, including: determining that the target table image is a borderless table; processing the target table image through a table recognition structure to obtain a first border frame; adding the content in the target table image to the first border frame.

[0263] In an embodiment of the present invention, a non-transitory computer-readable storage medium is provided, and the non-transitory computer-readable storage medium stores computer instructions. The computer instructions enable the computer to execute the methods provided in the above-mentioned method embodiments, for example, including: determining that the target table image is a borderless table; processing the target table image through a table recognition structure to obtain a first border frame; and adding the content in the target table image to the first border frame.

[0264] The present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above methods, and the methods include: determining that the target table image is a borderless table; processing the target table image through a table recognition structure to obtain a first border frame; and adding the content in the target table image to the first border frame.

[0265] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0266] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing a table image, characterized in that: include: Determine that the target table image is a borderless table; Processing the target table image by a table recognition structure to obtain a first border frame; Add the content in the target table image to the first border frame, Wherein, the table recognition structure is a line or a sliding window; In the case where the table recognition structure is the line, the table recognition structure includes a first direction line and a second direction line; the processing of the target table image by the table recognition structure to obtain a first border frame includes: based on the content distribution in the target table image, translating the first direction line and the second direction line in the target table image to determine a non-intersecting area in the target table image; determining a target point in the non-intersecting area, connecting the target point to obtain the first border frame, wherein the non-intersecting area is an area not covered by the first direction line or the second direction line; or, In the case where the table recognition structure is the sliding window, the table recognition structure includes a first direction sliding window and a second direction sliding window; the target table image is processed by the table recognition structure to obtain a first border frame, including: based on the content distribution in the target table image, translating the first direction sliding window in the target table image to obtain a first direction frame; translating the second direction sliding window in the target table image to obtain a second direction frame; setting a first direction table line based on the first direction frame, and setting a second direction table line based on the second direction frame to obtain a first border frame, wherein the table recognition structure is moved along the first direction of the target table image to record a first area that does not cover the content; and / or the table recognition structure is moved along the second direction of the target table image to record a second area that does not cover the content; and a table line is generated in at least one of the first area and the second area.

2. The method for processing a table image according to claim 1, characterized in that: The method further comprises: In the case where the table recognition structure is the line, the target table image is processed by the table recognition structure to obtain a first border frame, including: Moving the table recognition structure along a first direction of the target table image to record a first area not covering the content; and / or, moving the table recognition structure along a second direction of the target table image to record a second area not covered by the content; A table line is generated in at least one of the first area and the second area to obtain a first border frame.

3. The method for processing a table image according to claim 1, characterized in that: The step of translating the first direction line and the second direction line in the target table image based on the content distribution in the target table image to determine a non-intersection area in the target table image includes: Binarizing the target table image to obtain a first binary image; wherein the first value in the first binary image is the pixel corresponding to the content distribution area in the target table image, and the second value in the first binary image is the pixel corresponding to the non-content distribution area in the target table image; Processing pixels in the first binary image according to a first direction to obtain lines in a first direction; Processing pixels in the first binary image according to a second direction to obtain lines in a second direction; The first direction lines are crossed with the second direction lines, and the area not covered by the first direction lines or the second direction lines after the first direction lines cross with the second direction lines is used as a non-crossing area in the target table image.

4. The method for processing a table image according to claim 1, characterized in that: Determining the target point in the non-intersection area includes: Determine the contour of the non-intersection area by contour searching; According to the outline of the non-intersection area, a coordinate point of the non-intersection area is used as the target point.

5. The method for processing a table image according to claim 1, characterized in that: The step of determining that the target table image is a borderless table includes: Inputting the target table image into a pre-trained table classification model; According to the output result of the table classification model, determining that the target table image is a borderless table; The table classification model is obtained by training based on sample tables and category labels of the sample tables.

6. The method for processing a table image according to claim 1, characterized in that: Before the target table image is processed by the table recognition structure to obtain the first border frame, the method further includes: Binarizing the target table image to obtain a second binary image; wherein the second value in the second binary image is the pixel corresponding to the text character in the target table image, and the first value in the second binary image is the pixel in the target table image other than the text character; A table identification structure is set for the second binary image.

7. The method for processing a table image according to claim 1, characterized in that: The step of adding the content in the target table image into the first border frame includes: Performing text detection on the target table image, and obtaining a text box in the target table image according to the text detection result; Setting a text box in the first border frame; Fill the content in the text box in the target table image into the text box of the first border frame.

8. A table image processing device, characterized in that: include: A determination module, used for determining that the target table image is a borderless table; A processing module, used for processing the target table image through a table recognition structure to obtain a first border frame; An adding module, used for adding the content in the target table image into the first border frame, wherein the table recognition structure is a line or a sliding window; In the case where the table recognition structure is the line, the table recognition structure includes a first direction line and a second direction line, and the processing module is further used to, based on the content distribution in the target table image, translate the first direction line and the second direction line in the target table image to determine a non-intersecting area in the target table image; determine a target point in the non-intersecting area, and connect the target point to obtain a first border frame, wherein the non-intersecting area is an area not covered by the first direction line or the second direction line; or, In the case where the table recognition structure is the sliding window, the table recognition structure includes a first direction sliding window and a second direction sliding window; the target table image is processed by the table recognition structure to obtain a first border frame, including: based on the content distribution in the target table image, translating the first direction sliding window in the target table image to obtain a first direction frame; translating the second direction sliding window in the target table image to obtain a second direction frame; setting a first direction table line based on the first direction frame, and setting a second direction table line based on the second direction frame to obtain a first border frame, wherein the table recognition structure is moved along the first direction of the target table image to record a first area that does not cover the content; and / or the table recognition structure is moved along the second direction of the target table image to record a second area that does not cover the content; and a table line is generated in at least one of the first area and the second area.

9. An electronic device, characterized in that: include: processor, memory and bus, wherein, The processor and the memory communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the steps of the table image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to execute the steps of the table image processing method according to any one of claims 1 to 7.

Images