A method and related apparatus for recognizing tabular data in an image.

Abstract

This application discloses a method and related apparatus for recognizing tabular data in an image, applicable to the field of image recognition. The method involves: first, acquiring an image to be recognized containing tabular data; then, identifying the coordinates of multiple boundaries in the image, and locating the table in the image based on the coordinates of the multiple boundaries and a preset table location rule; next, determining the structural type of the located table based on a preset type detection model; and finally, recognizing the data text and data coordinates in the image, and obtaining structured tabular data based on the table's structural type, the coordinates of the multiple boundaries in the located table, the data text, and the data coordinates. Therefore, this application locates the table in the image to be recognized through feature recognition and determines the table's structural type through a type detection model. It can accurately and quickly obtain structured tabular data without being limited by the table's display format, making the tabular data in the image easier to use.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method and related apparatus for recognizing tabular data in an image. Background Technology

[0002] Tables are commonly used to organize and structure data, and people widely employ various forms of tables in communication, scientific research, and data analysis. In many cases, tables exist in image format, making it impossible to directly edit the data and hindering its extraction and use.

[0003] Currently, OCR (Optical Character Recognition) technology is commonly used to recognize images containing tabular data, converting the text information in the table into copyable and pasteable text. This allows the text information to be entered into the desired location for data analysis and other activities. However, compared to traditional manual input, this method only eliminates the manual typing step and is still relatively time-consuming and labor-intensive.

[0004] Therefore, how to make the tabular data in the image easier to use has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and related apparatus for recognizing tabular data in images, which makes tabular data in images easier to use.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, this application provides a method for recognizing tabular data in an image, the method comprising:

[0008] Acquire the image to be recognized, including tabular data;

[0009] Identify the coordinates of multiple boundaries in the image to be identified, and locate the table in the image to be identified based on the coordinates of the multiple boundaries and a preset table positioning rule;

[0010] Based on a preset type detection model, the structure type of the located table is determined;

[0011] The data text and data coordinates in the image to be identified are identified, and structured table data is obtained based on the structure type of the table, the coordinates of multiple boundaries in the table, the data text, and the data coordinates.

[0012] Optionally, identifying the coordinates of multiple boundaries in the image to be identified, and locating the table in the image to be identified based on the coordinates of the multiple boundaries and a preset table positioning rule, includes:

[0013] Identify the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified;

[0014] The density of the multiple boundaries in the image to be identified is calculated based on the coordinates of the multiple boundaries.

[0015] If the density is greater than a preset density threshold, then the table in the image to be identified is located based on the preset line segment clustering rules;

[0016] If the density is less than or equal to a preset density threshold, the table in the image to be identified is located based on the two boundaries with the largest interval among the multiple boundaries.

[0017] Optionally, identifying the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified includes:

[0018] Traverse the pixels in the image to be identified, and determine the coordinates of multiple line segments in the image to be identified based on the color and coordinates of the pixels;

[0019] Based on the coordinates of the multiple line segments and the preset boundary enhancement rules, the coordinates of multiple boundaries in the image to be identified are determined.

[0020] Optionally, identifying the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified includes:

[0021] Traverse the pixels in the image to be identified, and determine the coordinates of the blank areas in the image to be identified based on the color and coordinates of the pixels;

[0022] Based on the coordinates of the blank area, the coordinates of multiple boundaries in the image to be identified are determined.

[0023] Optionally, determining the coordinates of multiple boundaries in the image to be identified based on the coordinates of the blank area includes:

[0024] Based on the coordinates of the blank area, the range of the first table in the image to be identified is determined;

[0025] Identify multiple data coordinates within the first table range, and determine the second table range in the image to be identified based on the interval distance between the multiple data coordinates within the first table range;

[0026] Based on the range of the second table, the coordinates of multiple boundaries in the image to be identified are determined.

[0027] Optionally, the step of identifying the data text and data coordinates in the image to be identified, based on the structure type of the table, the coordinates of multiple boundaries located in the table, the data text, and the data coordinates, yields structured table data, including:

[0028] Identify the data text and data coordinates in the image to be identified;

[0029] Based on the table's structure type, the coordinates of multiple boundaries and data coordinates located in the table, the coordinates of the table's category items and sub-data are determined; the data includes category items and sub-data.

[0030] Structured tabular data is obtained based on the intersection of the data text, the category item coordinates, the sub-data coordinates, and the coordinate range of the category item coordinates and the sub-data coordinates in the arrangement direction of the category items.

[0031] Secondly, this application provides a device for recognizing tabular data in an image, the device comprising: an acquisition module, a positioning module, a determination module, and a structuring module;

[0032] The acquisition module is used to acquire the image to be recognized, including tabular data;

[0033] The positioning module is used to identify the coordinates of multiple boundaries in the image to be identified, and to locate the table in the image to be identified based on the coordinates of the multiple boundaries and a preset table positioning rule.

[0034] The determining module is used to determine the structure type of the located table based on a preset type detection model;

[0035] The structuring module is used to identify the data text and data coordinates in the image to be identified, and to obtain structured table data based on the structure type of the table, the coordinates of multiple boundaries in the table, the data text, and the data coordinates.

[0036] Optionally, the positioning module includes: a boundary recognition unit, a calculation unit, and a positioning unit;

[0037] The boundary recognition unit is used to identify the color and coordinates of pixels in the image to be recognized to determine the coordinates of multiple boundaries in the image to be recognized;

[0038] The calculation unit is used to calculate the density of the multiple boundaries in the image to be identified based on the coordinates of the multiple boundaries;

[0039] The positioning unit is used to locate the table in the image to be identified based on a preset line segment clustering rule when the density is greater than a preset density threshold; and to locate the table in the image to be identified based on the two boundaries with the largest interval among the multiple boundaries when the density is less than or equal to the preset density threshold.

[0040] Thirdly, this application provides a computer device, the computer device comprising: a memory and a processor;

[0041] The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computer device performs the method for recognizing tabular data in an image as described in any one of the first aspects.

[0042] Fourthly, this application provides a computer-readable storage medium storing computer instructions, which, when executed on a computer device, enable the computer device to perform the method for recognizing tabular data in an image as described in any one of the first aspects.

[0043] Compared with the prior art, this application has the following beneficial effects:

[0044] This application provides a method for recognizing tabular data in an image. The method involves: first, acquiring an image containing tabular data; then, identifying the coordinates of multiple boundaries in the image, and locating the table in the image based on these coordinates and a preset table localization rule; next, determining the structure type of the located table based on a preset type detection model; and finally, identifying the data text and data coordinates in the image, and obtaining structured tabular data based on the table's structure type, the coordinates of the multiple boundaries in the located table, and the data text and coordinates. Therefore, this application, through feature recognition and table localization in the image, and by determining the structure type of the located table through a type detection model, can accurately and quickly obtain the relationships between data in a table, regardless of its specific display format, thus obtaining structured tabular data. When needed, this structured tabular data can be used for automatic data entry, making the tabular data in the image easier to use. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a method for recognizing tabular data in an image, as provided in this application embodiment;

[0047] Figure 2 This is a schematic diagram of data text recognition provided in an embodiment of this application;

[0048] Figure 3 This application provides a schematic diagram of structured tabular data as an embodiment of the present application.

[0049] Figure 4 A flowchart of a table positioning method provided in an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of a line segment recognition result provided in an embodiment of this application;

[0051] Figure 6 A schematic diagram of a boundary enhancement result provided in an embodiment of this application;

[0052] Figure 7 This is a schematic diagram of a blank area recognition result provided in an embodiment of this application;

[0053] Figure 8 A schematic diagram of a borderless table provided in an embodiment of this application;

[0054] Figure 9 This application provides a schematic diagram of the location for adding auxiliary line segments in an embodiment.

[0055] Figure 10 A schematic diagram of a dense boundary table positioning process provided in an embodiment of this application;

[0056] Figure 11 A schematic diagram of a non-dense boundary table positioning process provided in an embodiment of this application;

[0057] Figure 12 A schematic diagram of a device for recognizing tabular data in an image provided in an embodiment of this application;

[0058] Figure 13 This is a structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0059] As described earlier, OCR (Optical Character Recognition) technology is currently commonly used to recognize images containing tabular data, converting the text information in the table into copyable and pasteable text. This allows the text information to be entered into the desired application location, enabling the use of tabular data in images during data analysis and other activities. However, compared to traditional manual input, this method only eliminates the manual typing step and is still relatively time-consuming and labor-intensive.

[0060] In view of this, this application provides a method for recognizing tabular data in an image. The method involves: first, acquiring an image to be recognized containing tabular data; then, identifying the coordinates of multiple boundaries in the image to be recognized, and locating the table in the image based on the coordinates of the multiple boundaries and a preset table location rule; next, determining the structural type of the located table based on a preset type detection model; and finally, identifying the data text and data coordinates in the image to be recognized, and obtaining structured tabular data based on the table's structural type, the coordinates of the multiple boundaries in the located table, the data text, and the data coordinates. Therefore, this application embodiment, through feature recognition and table location in the image to be recognized, and by determining the structural type of the located table through a type detection model, can accurately and quickly obtain the relationships between data in the table, regardless of the specific display format of the table, thereby obtaining structured tabular data. When the tabular data is needed, the structured tabular data can be used to achieve automatic data entry, making the tabular data in the image easier to use.

[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0062] See Figure 1 The figure is a flowchart of a method for recognizing tabular data in an image according to an embodiment of this application. The method includes:

[0063] S101: Obtain the image to be recognized, including tabular data.

[0064] Tables are commonly used to organize and structure data, and people widely employ various forms of tables in communication, scientific research, and data analysis. In many cases, tables exist in image form, allowing only viewing but not direct editing of the data, making data extraction and use difficult.

[0065] For example, the image to be recognized can be a binarized image. Further, the pixel value of the pixel closer to the dark side in the binarized image can be set to (0,0,0), that is, the pixel can be set to black; and the pixel value of the pixel closer to the bright side can be set to (225,225,225), that is, the pixel can be set to white. In this way, the influence of image color on the subsequent recognition process can be reduced, and the boundary between light and dark can be widened to highlight the table boundary, which is usually a black line segment.

[0066] S102: Identify the coordinates of multiple boundaries in the image to be identified, and locate the table in the image to be identified based on the coordinates of the multiple boundaries and the preset table positioning rules.

[0067] For example, the pixels of the image to be recognized can be traversed and searched. Based on the color and coordinates of the pixels, multiple boundaries and / or multiple blank areas in the image to be recognized can be identified. The table in the image to be recognized can then be located based on the coordinates of the identified multiple boundaries, or based on the coordinates of the multiple boundaries determined by the coordinates of the multiple blank areas.

[0068] S103: Determine the structure type of the located table based on the preset type detection model.

[0069] The structure type of the table can include, but is not limited to, the first type where the category item is in the first row of the table and the second type where the category item is in the first column of the table. As an example, a large number of tables located by the method described in step S102 can be used as a training set, and a type detection model can be trained using a deep learning network. The structure type of the table can then be determined by inputting the located table into the trained type detection model.

[0070] S104: Identify the data text and data coordinates in the image to be identified, and obtain structured table data based on the table structure type, the coordinates of multiple boundaries in the located table, the data text and data coordinates.

[0071] For example, data text and data coordinates can be obtained based on technologies such as OCR (Optical Character Recognition). The data coordinates can be obtained through OCR technology, such as... Figure 2 The coordinates of the text boxes shown are easy to understand. Figure 2 All rectangular boxes enclosing text, numbers, etc., are text boxes as described in the embodiments of this application; table data may include category items and sub-data.

[0072] Optionally, the position of the category items relative to the entire table can be determined first based on the table's structure type. Then, the category item coordinates are determined based on the table boundary coordinates and data coordinates. The category item coordinates can be the coordinate range of the category item text box in the X and Y axes. Next, the correspondence between sub-data and category items can be determined by comparing the intersection of the coordinate ranges of the sub-data text box and the category item text box in the category item arrangement direction. For example, if the coordinate range of the sub-data text box in the category item arrangement direction is [10, 20], and the coordinate range of the category item text box in the category item arrangement direction is [15, 25], then the two coordinate ranges intersect at [15, 20]. Figure 2 The data is categorized into "Chinese Name," "English Name," "Result," "Unit," and "Reference Value," with the categorization direction aligned along the X-axis. Finally, by combining the coordinate range of the sub-data text boxes in the direction perpendicular to the categorization direction with the data text, a structured tabular data is obtained, which is then referenced. Figure 2 The direction perpendicular to the direction in which the categories are arranged is the Y-axis direction.

[0073] Specifically, such as Figure 2 As shown in the figure, this is a schematic diagram of data text recognition provided in an embodiment of this application. The table structure type is the first type of category item in the first row of the table. By using the minimum distance between the Y-axis coordinate of the upper boundary of the table and the Y-axis coordinate range in the data coordinates, the category item coordinates and sub-data coordinates can be distinguished from the data coordinates of the table. Then, based on whether there is an intersection between the coordinate ranges of the sub-data text boxes and the category item text boxes in the X-axis direction, the correspondence between the sub-data and the category items can be obtained. Finally, by combining the coordinate range of the sub-data text boxes in the Y-axis direction and the data text, the following can be obtained: Figure 3 The structured tabular data shown.

[0074] Optionally, the coordinate ranges of the sub-data text boxes along the category item arrangement direction can be shrunk inward by a preset proportion or size before comparing the intersection of the coordinate ranges of the sub-data text boxes and category item text boxes along the category item arrangement direction. Alternatively, the intersection of the coordinate ranges of the sub-data text boxes and category item text boxes along the category item arrangement direction can be compared first. When the coordinate ranges of one sub-data text box and multiple category item text boxes intersect in the category item arrangement direction, the coordinate ranges of the sub-data text box along the category item arrangement direction can be shrunk inward by a preset proportion or size. This reduces the likelihood of the same sub-data text box intersecting with the coordinate ranges of multiple category item text boxes along the category item arrangement direction, resulting in more accurate structured tabular data.

[0075] Still with Figure 2 Taking the data text recognition diagram shown below as an example, in such a case... Figure 2If the coordinate range of the text box for the sub-data "white blood cells" intersects with that of the text box for the category item "Chinese name" on the X-axis, then "white blood cells" is the sub-data corresponding to the category item "Chinese name". If the coordinate range of the text box for the sub-data "red blood cell distribution width CV" intersects with that of both the text boxes for the category items "Chinese name" and "English name", then the coordinate range of the text box for the sub-data "red blood cell distribution width CV" on the X-axis can be shrunk inward by a preset proportion at both ends. For example, if the coordinate range of this text box on the X-axis is 130-180, it can be shrunk inward by a preset proportion of 20% to 135-175. Then, the coordinate range of the category item text box and the shrunk sub-data text box on the X-axis can be compared again.

[0076] Therefore, in this embodiment of the application, tables in the image to be identified are located by feature recognition, and the structure type of the located table is determined by the type detection model. This allows for accurate and rapid extraction of tables in the image to be identified and determination of their structure type, regardless of the specific display format of the table. Furthermore, the table boundary coordinates, data text, and data coordinates in the table can be referenced to obtain structured table data. When table data is needed, the structured table data can be used to achieve automatic data entry, making the table data in the image easier to use.

[0077] See Figure 4 The figure is a flowchart of a table positioning method provided in an embodiment of this application. The method includes:

[0078] S401: Traverse the pixels in the image to be recognized, and determine whether there is a bounded table in the image based on the color and coordinates of the pixels. If yes, proceed to step S402; otherwise, proceed to step S404.

[0079] In this embodiment, the image to be identified has already been binarized. For example, the pixels in the image to be identified can be traversed, connecting pixels with the same first-direction coordinate value and consecutive second-direction coordinate values ​​at (0,0,0), until a pixel with a value of (255,255,255) is found, thus identifying a line segment. Here, the first direction can be the horizontal coordinate direction, and the second direction can be the vertical coordinate direction; or the first direction can be the vertical coordinate direction, and the second direction can be the horizontal coordinate direction.

[0080] Optionally, the presence of a bounded table in the image to be identified can be determined based on preset conditions such as the number of line segments identified in the entire image and / or the positional relationship between multiple line segments. For example, it can be set that if the number of mutually parallel line segments in the image to be identified is less than three, then it is determined that there is no bounded table in the image to be identified.

[0081] S402: Traverse the pixels in the image to be recognized, and determine the coordinates of multiple line segments in the image to be recognized based on the color and coordinates of the pixels.

[0082] As an example, the coordinates of a line segment with a pixel value of (0,0,0) can be determined by the maximum and minimum values ​​of the coordinates of multiple pixels in each direction. For instance, if the ordinate of multiple pixels in a line segment is 10, the minimum x-coordinate is 10, and the maximum x-coordinate is 100, then the ordinate of the line segment can be determined to be 10, and the x-coordinate range is 10-100. This is merely an example; it is understood that in this embodiment, the x-coordinate and ordinate of the line segment can both be a coordinate range or a single point.

[0083] S403: Based on the coordinates of multiple line segments and preset boundary enhancement rules, determine the coordinates of multiple boundaries in the image to be recognized.

[0084] Specifically, boundary enhancement rules are used to determine whether multiple line segments belong to the same boundary. For example, when two line segments with a similarity in the range of their vertical coordinates are identified that is higher than a preset similarity threshold, it can be determined whether the two line segments belong to the same boundary by judging whether the distance in the horizontal coordinate direction is less than a preset distance threshold.

[0085] like Figure 5 As shown in the figure, this is a schematic diagram of a line segment recognition result provided in an embodiment of this application. The image to be recognized may be, for example, an image obtained by scanning a printout. Due to possible abnormalities in the printing or scanning process, a long boundary in the image to be recognized may be broken in the middle. Consequently, during the line segment recognition process, the boundary that should have been a complete line segment is recognized as... Figure 5 The diagram shows multiple line segments with small intervals.

[0086] Specifically, a distance threshold between two line segments can be preset. The system determines whether the distance between two line segments with a similarity higher than the preset similarity threshold in the horizontal direction is less than the preset distance threshold. If so, then... Figure 6 As shown in boundary 01, fill the gap between the two line segments to strengthen them into a single boundary; otherwise, as shown in the diagram. Figure 6 As shown in boundary 02 and boundary 03, the two line segments are defined as two boundaries. The coordinates of the boundaries can be obtained from the coordinates of the line segments.

[0087] For example, the preset distance threshold can be 10. If the similarity of the coordinate ranges of the ordinates of line segment A and line segment B is higher than the preset similarity threshold, where the coordinate range of the abscissa of line segment A is 10-100 and the coordinate range of the abscissa of line segment B is 105-180, then the distance between the two line segments in the abscissa direction is 5, which is less than the preset distance threshold of 10. Filling the blank area between the two line segments, a boundary with an abscissa range of 10-180 is obtained through boundary enhancement.

[0088] Therefore, even if a long boundary is missing a small section in the middle due to an anomaly during the printing process or other reasons and is identified as multiple line segments, it can still be correctly identified as a single boundary.

[0089] S404: Traverse the pixels in the image to be recognized, and determine the coordinates of the blank areas in the image to be recognized based on the color and coordinates of the pixels.

[0090] Specifically, the pixels in the image to be recognized can be traversed, and blank areas in the image can be determined based on the coordinates of multiple consecutive pixels with pixel values ​​of (255, 255, 255). Then, the coordinates of the blank areas in the image can be determined based on the identified blank areas and the coordinates of the corresponding pixels. Optionally, rows or columns in the image to be recognized can be divided first using character recognition technology and / or the aforementioned blank area recognition method, and then the blank areas can be determined.

[0091] Optionally, a minimum size for the blank area can be preset. If the area composed of multiple white pixels is smaller than the preset minimum size, these pixels are ignored to avoid recognizing white pixels in the data text, such as numbers, Chinese characters, or English letters, as blank areas.

[0092] S405: Determine the coordinates of multiple boundaries in the image to be identified based on the coordinates of the blank area.

[0093] Specifically, multiple boundaries can be added to the image to be recognized based on the coordinates of the blank areas, thereby obtaining the coordinates of these multiple boundaries. Figure 7 Taking the marked blank area recognition results diagram as an example, among the four marked blank areas, the first row has 7 blank areas and the second row has 8 blank areas. Since there are a lot of blank areas, it can be considered that there is a borderless table here, and a boundary needs to be added. In the third and fourth rows, there are only blank areas at the beginning and end. It can be considered that there is continuous text here, and there is no borderless table, so no boundary needs to be added.

[0094] It should be noted that, Figure 7The blank area identification result shown is for a bounded table; it is used here as an example for illustrative purposes only. In the embodiments of this application, it is possible to identify only, for example... Figure 8 The blank area identification described in step S404 is performed on the image to be identified that does not have a bounded table.

[0095] Optionally, in some other embodiments provided in this application, steps S402 and S404 may be performed simultaneously or sequentially without performing step S401. When multiple line segments that meet preset conditions, such as preset number and positional relationship, are identified in the image to be identified, step S403 is performed; when multiple regular and discontinuous blank areas are identified in the image to be identified, step S405 is performed.

[0096] For example, such as Figure 8 As shown in the figure, this figure is a schematic diagram of a borderless table provided in an embodiment of this application. After the borderless table is divided into rows, the data text in the borderless table can be considered to be separated by discontinuous blank areas. Therefore, if there is a region in the image to be identified that has a large number of regular and discontinuous blank areas after dividing into rows or columns, such as many consecutive rows having "spaces" in similar positions, it can be considered that this region has a borderless table.

[0097] For example Figure 8 The boundless table shown can be defined by the coordinates of the identified blank areas. Then, within the boundless table, corresponding auxiliary line segments can be added to the table with boundaries by referring to the coordinates of the blank areas, thus obtaining the coordinates of multiple boundaries.

[0098] Optionally, embodiments of this application can also narrow down the scope of the borderless table using text clustering. Specifically, the scope of the borderless table determined based on the coordinates of the blank area can be denoted as the first table scope, and multiple data coordinates within the first table scope can be identified. Based on the interval distance and / or relative position of the multiple data coordinates, data with similar interval distance and / or relative position can be grouped into one category, i.e., text clustering can be performed, thereby determining the second table scope in the image to be identified. Finally, based on the second table scope, the coordinates of multiple boundaries in the image to be identified can be determined.

[0099] For example, the upper and lower boundaries of a borderless table can be determined based on the maximum and minimum values ​​of the y-coordinates within the range of the second table. When there are, for example... Figure 9When there are multiple qualifying second table ranges in ranges 91 and 92, auxiliary line segments can be added to each of the multiple second table ranges; alternatively, the upper and lower boundaries of the borderless table can be determined based on the maximum difference between the maximum and minimum values ​​of the y-coordinates in each second table range, and auxiliary line segments can be added for example... Figure 9 The auxiliary line segments shown are used to determine the coordinates of the two boundaries. This reduces the number of auxiliary line segments that need to be added, which in turn reduces the number of boundaries that need to be determined, thus reducing the amount of computation required to determine the coordinates of multiple boundaries.

[0100] S406: Determine whether the density of multiple boundaries in the image to be identified is greater than a preset density threshold based on the coordinates of multiple boundaries. If yes, proceed to step S407; otherwise, proceed to step S408.

[0101] Typically, dense boundary tables have a large number of boundaries that are concentrated in one location, while non-dense boundary tables have fewer boundaries that are more dispersed in one location. For example, the density of multiple boundaries can be determined based on the number of boundaries in the image to be identified and the distance between them, or it can be determined based on the positional relationship of the boundaries in the image to be identified. These are merely examples and do not limit the specific method used in this application to determine the density of multiple boundaries in the image to be identified. If the density of multiple boundaries in the image to be identified is greater than a preset density threshold, the table contained in the image to be identified is considered a dense boundary table; if the density of multiple boundaries in the image to be identified is less than or equal to the preset density threshold, the table contained in the image to be identified is considered a non-dense boundary table. For example, the preset density threshold can be a number of boundaries greater than 6 and a distance between each boundary less than 300 pixels; it can also be a number of parallel boundaries with equal spacing greater than 4, etc. The density threshold can be flexibly set according to actual needs.

[0102] S407: Locate tables in the image to be identified based on preset line segment clustering rules.

[0103] Specifically, see Figure 10 The figure is a schematic diagram of a dense boundary table localization process provided in an embodiment of this application. In the process of locating a dense boundary table by line segment clustering, the distance between multiple parallel boundaries can be calculated based on the coordinates of multiple boundaries. Boundaries with similar distances are grouped into one category. Then, the boundary with the smallest coordinate value in the direction perpendicular to the boundary arrangement is determined as the upper boundary of the table, and the boundary with the largest coordinate value in the direction perpendicular to the boundary arrangement is determined as the lower boundary of the table, thereby locating the table in the image to be identified.

[0104] S408: Locate the table in the image to be identified based on the two boundaries with the largest interval among multiple boundaries.

[0105] Specifically, see Figure 11 The figure is a schematic diagram of a non-dense boundary table localization process provided in an embodiment of this application. For a non-dense boundary table, the distance between multiple parallel boundaries can be calculated, and the two boundaries with the largest distance can be selected as the upper and lower boundaries of the table, thereby locating the table in the image to be identified.

[0106] Therefore, feature recognition can be used to locate bounded and unbounded tables. The location of the table is not limited by whether the table has a boundary, avoiding the inability to locate unbounded tables due to the inability to identify the boundary. This allows for a more comprehensive identification and location of tables in the image to be recognized.

[0107] In other embodiments provided in this application, identifying data text and data coordinates in an image to be identified, and obtaining structured table data based on the table's structure type, the coordinates of multiple boundaries located in the table, the data text, and the data coordinates, may specifically include the following steps:

[0108] S501: Identify the data text and data coordinates in the image to be identified.

[0109] For example, OCR technology can be used to identify data text and data coordinates in an image to be recognized.

[0110] S502: Based on the table's structure type, the coordinates of multiple boundaries and data coordinates within the located table, determine the coordinates of the table's category items and sub-data.

[0111] Specifically, the data may include category items and sub-data, and the table structure type may include types such as a first type where category items are in the first row of the table and a second type where category items are in the first column of the table. Based on the table structure type, for example, if the table structure type is a first type where category items are in the first row of the table, then the category item coordinates and sub-data coordinates can be distinguished from the table's data coordinates according to the relative positions of each data coordinate, based on the data coordinates and the coordinates of multiple boundaries located in the table.

[0112] Optionally, after identifying the category item coordinates, the data text at that coordinate position, i.e., the category item text, can be found based on the category item coordinates. The category item text is then compared to see if it is included in the pre-configured category item thesaurus. If it is, the category item text passes the verification; otherwise, it fails the verification. The table range can be expanded based on the table's structure type. For example, if the table's structure type is the first type mentioned above, the table range can be expanded by one row to obtain new category item coordinates. If the data text corresponding to the new category item coordinate position is included in the pre-configured category item thesaurus, the new category item text passes the verification. Subsequent steps are then performed based on the new category item coordinates and the new category item text. For example, if the category text located by the coordinates of the distinguished category item is "white blood cells", which is not included in the pre-configured category item thesaurus, and the structure type of the table is Type I, then we can expand up one row to obtain new category item coordinates and locate a new category item text "Chinese name" based on this. This new category item text is included in the pre-configured category item thesaurus. Thus, "Chinese name" and the data text in the same row are used as the category item text of the table, and its coordinate position is used as the category item coordinate of the table. All other data texts in the table are sub-data texts.

[0113] S503: Based on the data text, category item coordinates, sub-data coordinates, and the intersection of the coordinate ranges of category item coordinates and sub-data coordinates in the arrangement direction of the category items, structured tabular data is obtained.

[0114] Taking the first table structure type as an example, based on the intersection of the coordinate ranges of the category item coordinates and the sub-data coordinates along the horizontal axis, a correspondence between the sub-data coordinates and the category item coordinates can be established. Then, based on the sub-data coordinates and the category item coordinates, the sub-data text and the category item text can be located, thus obtaining the association between the sub-data text and the category item text. Finally, based on the coordinate range of the sub-data's vertical axis, multiple rows of sub-data can be divided. Using rows as units, based on the association between the sub-data text and the category item text, the following can be obtained: Figure 3 The structured tabular data shown.

[0115] This allows for a more accurate determination of the coordinates of the table's category items and sub-data, resulting in more accurate structured table data.

[0116] See Figure 12 The figure is a schematic diagram of a device for recognizing tabular data in an image according to an embodiment of this application. The device includes: an acquisition module 1201, a positioning module 1202, a determination module 1203, and a structuring module 1204.

[0117] The acquisition module 1201 is used to acquire the image to be recognized, including tabular data.

[0118] The positioning module 1202 is used to identify the coordinates of multiple boundaries in the image to be identified, and to locate the table in the image to be identified based on the coordinates of the multiple boundaries and the preset table positioning rules.

[0119] The determination module 1203 is used to determine the structure type of the located table based on a preset type detection model.

[0120] The structure module 1204 is used to identify the data text and data coordinates in the image to be identified. Based on the table structure type, the coordinates of multiple boundaries in the located table, the data text and data coordinates, structured table data is obtained.

[0121] Optionally, the positioning module 1202 includes: a boundary recognition unit, a calculation unit, and a positioning unit.

[0122] The boundary recognition unit is used to identify the color and coordinates of pixels in the image to be recognized to determine the coordinates of multiple boundaries in the image.

[0123] Optionally, the boundary recognition unit can be specifically used to: traverse the pixels in the image to be recognized, determine the coordinates of multiple line segments in the image to be recognized based on the color and coordinates of the pixels; determine the coordinates of multiple boundaries in the image to be recognized based on the coordinates of the multiple line segments and a preset boundary enhancement rule; and / or traverse the pixels in the image to be recognized, determine the coordinates of blank areas in the image to be recognized based on the color and coordinates of the pixels; and determine the coordinates of multiple boundaries in the image to be recognized based on the coordinates of the blank areas.

[0124] Specifically, determining the coordinates of multiple boundaries in the image to be recognized based on the coordinates of the blank area may include: determining the first table range in the image to be recognized based on the coordinates of the blank area; recognizing multiple data coordinates within the first table range; determining the second table range in the image to be recognized based on the interval distance between the multiple data coordinates within the first table range; and determining the coordinates of multiple boundaries in the image to be recognized based on the second table range.

[0125] The calculation unit is used to calculate the density of multiple boundaries in the image to be identified based on the coordinates of multiple boundaries.

[0126] The localization unit is used to locate the table in the image to be identified based on a preset line segment clustering rule when the density is greater than a preset density threshold; and to locate the table in the image to be identified based on the two boundaries with the largest interval among multiple boundaries when the density is less than or equal to the preset density threshold.

[0127] Optionally, the structured module 1204 includes: a data recognition unit, a coordinate determination unit, and a structured unit.

[0128] The data recognition unit is used to identify the data text and data coordinates in the image to be recognized.

[0129] The coordinate determination unit is used to determine the coordinates of the category items and sub-data in a table based on the table's structure type, the coordinates of multiple boundaries in the located table, and the data coordinates; the data includes category items and sub-data.

[0130] Structured units are used to obtain structured tabular data based on the intersection of the coordinate ranges of data text, category item coordinates, sub-data coordinates, category item coordinates, and sub-data coordinates in the arrangement direction of the category items.

[0131] See Figure 13 The figure is a structural diagram of a computer device provided in an embodiment of this application. The computer device 01 includes a memory 08 and a processor 03.

[0132] One or more computer programs are stored in the one or more memories 08, and the one or more computer programs include instructions; when the instructions are executed by the processor 08, the computer device 01 performs the method for recognizing tabular data in the above image.

[0133] For example, the computer device 01 is manifested in the form of a general-purpose computing device. The components of the computer device 01 may include, but are not limited to: one or more processors or processing units 03, system memory 08, and bus 04 connecting different system components (including system memory 08 and processing unit 03).

[0134] Bus 04 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0135] Computer device 01 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 01, including volatile and non-volatile media, removable and non-removable media.

[0136] System memory 08 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 09 and / or cache memory 10. Computer device 01 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 13 Not shown; usually referred to as a "hard drive"). Although Figure 13 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 04 via one or more data media interfaces. Memory 08 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0137] A program / utility 12 having a set (at least one) of program modules 13 may be stored in, for example, memory 08. Such program modules 13 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of the present invention.

[0138] Computer device 01 can also communicate with one or more external devices 02 (e.g., keyboard, pointing device, display 07, etc.), and with one or more devices that enable a user to interact with the computer device 01, and / or with any device that enables the computer device 01 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 06. Furthermore, computer device 01 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 05. Figure 13 As shown, network adapter 05 communicates with other modules of computer device 01 via bus 04. It should be understood that, although... Figure 13 As not shown in the diagram, it can be used in conjunction with computer device 01 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] The processor unit 03 executes various functional applications and data processing by running programs stored in the system memory 08, such as implementing the method for recognizing tabular data in images provided in the embodiments of this application.

[0140] In addition, this application also provides a computer-readable storage medium storing computer instructions, which, when executed on a computer device, enable the computer device to perform a method for recognizing tabular data in the aforementioned image.

[0141] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and storage medium embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0142] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for recognizing tabular data in an image, characterized in that, The method includes: Acquire the image to be recognized, including tabular data; The method involves identifying the coordinates of multiple boundaries in the image to be identified, and locating the table in the image based on the coordinates of the multiple boundaries and a preset table location rule. Specifically, this involves: identifying the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified; calculating the density of the multiple boundaries in the image to be identified based on the coordinates of the multiple boundaries; if the density is greater than a preset density threshold, locating the table in the image to be identified based on a preset line segment clustering rule; if the density is less than or equal to the preset density threshold, locating the table in the image to be identified based on the two boundaries with the largest distance between them; the boundaries are the boundaries of the table; and the line segment clustering rule is used to cluster the boundaries. Based on a preset type detection model, the structure type of the located table is determined; The process involves identifying the data text and data coordinates in the image to be identified, and obtaining structured table data based on the table's structure type, the coordinates of multiple boundaries located in the table, the data text, and the data coordinates. Specifically, this includes: identifying the data text and data coordinates in the image to be identified; determining the table's category item coordinates and sub-data coordinates based on the table's structure type, the coordinates of multiple boundaries located in the table, and the data coordinates; the data includes category items and sub-data; and obtaining structured table data based on the intersection of the coordinate ranges of the data text, the category item coordinates, the sub-data coordinates, and the category item coordinates and sub-data coordinates in the arrangement direction of the category items.

2. The method according to claim 1, characterized in that, The step of identifying the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified includes: Traverse the pixels in the image to be identified, and determine the coordinates of multiple line segments in the image to be identified based on the color and coordinates of the pixels; Based on the coordinates of the multiple line segments and the preset boundary enhancement rules, the coordinates of multiple boundaries in the image to be identified are determined; the boundary enhancement rules are used to determine whether multiple line segments belong to the same boundary.

3. The method according to claim 1, characterized in that, The step of identifying the color and coordinates of pixels in the image to be identified to determine the coordinates of multiple boundaries in the image to be identified includes: Traverse the pixels in the image to be identified, and determine the coordinates of the blank areas in the image to be identified based on the color and coordinates of the pixels; Based on the coordinates of the blank area, the coordinates of multiple boundaries in the image to be identified are determined.

4. The method according to claim 3, characterized in that, Determining the coordinates of multiple boundaries in the image to be identified based on the coordinates of the blank area includes: Based on the coordinates of the blank area, the range of the first table in the image to be identified is determined; Identify multiple data coordinates within the first table range, and determine the second table range in the image to be identified based on the interval distance between the multiple data coordinates within the first table range; Based on the range of the second table, the coordinates of multiple boundaries in the image to be identified are determined.

5. A device for recognizing tabular data in an image, characterized in that, The device includes: an acquisition module, a positioning module, a determination module, and a structuring module; The acquisition module is used to acquire the image to be recognized, including tabular data; The positioning module is used to identify the coordinates of multiple boundaries in the image to be identified, and to locate the table in the image to be identified based on the coordinates of the multiple boundaries and a preset table positioning rule; the boundary is the boundary of the table. The determining module is used to determine the structure type of the located table based on a preset type detection model; The structuring module is used to identify data text and data coordinates in the image to be identified, and to obtain structured table data based on the structure type of the table, the coordinates of multiple boundaries located in the table, the data text, and the data coordinates; specifically: identifying data text and data coordinates in the image to be identified; determining the category item coordinates and sub-data coordinates of the table based on the structure type of the table, the coordinates of multiple boundaries located in the table, and the data coordinates; the data includes category items and sub-data; and obtaining structured table data based on the intersection of the coordinate ranges of the data text, the category item coordinates, the sub-data coordinates, and the category item coordinates and sub-data coordinates in the arrangement direction of the category items. The positioning module includes: a boundary recognition unit, a calculation unit, and a positioning unit; The boundary recognition unit is used to identify the color and coordinates of pixels in the image to be recognized to determine the coordinates of multiple boundaries in the image to be recognized; The calculation unit is used to calculate the density of the multiple boundaries in the image to be identified based on the coordinates of the multiple boundaries; The positioning unit is used to locate the table in the image to be identified based on a preset line segment clustering rule when the density is greater than a preset density threshold; and to locate the table in the image to be identified based on the two boundaries with the largest interval among the multiple boundaries when the density is less than or equal to the preset density threshold; the line segment clustering rule is used to cluster the boundaries.

6. A computer device, characterized in that, The computer device includes: a memory and a processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computer device performs a method for recognizing tabular data in an image as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when executed on a computer device, enable the computer device to perform a method for recognizing tabular data in an image as described in any one of claims 1-4.

Images