Table detection identification method, device and equipment and storage medium

Abstract

The application discloses a table detection and recognition method and device, equipment and a storage medium, to solve the problem that the related art table recognition method is not robust to noise. The application adopts a straight line detection algorithm model to perform straight line detection on a target image, obtains straight line detection results corresponding to different straight line classifications, classifies the straight lines into different straight line combinations according to different position relationships of the straight lines in the straight line detection results, determines feature vectors corresponding to the different straight line combinations, inputs the feature vectors into corresponding table classifiers for classification processing, obtains target straight line combinations, finally, determines the intersection points of the straight lines in the target straight line combinations of different straight line classifications, and obtains the coordinates of the table field region in the target image according to the intersection points of the straight lines. The application classifies the relationship between the straight lines to obtain correct straight line combinations and determine correct tables, thereby enhancing the robustness of table recognition and effectively improving the recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a table detection and recognition method, device, equipment and storage medium. BACKGROUND

[0002] At present, in some text recognition scenarios that need to perform table recognition, whether the content of the table can be correctly recognized greatly affects the subsequent text recognition. Accurately recognizing the positions of each item in the table can improve the accuracy of text recognition according to prior knowledge and structured correction. There are usually two ideas for table recognition. One is to detect the intersection points of straight lines in the table, and to form the table from the intersection points. The other is to detect straight lines in each direction of the table, and to combine the straight lines to form the table.

[0003] In related technologies, most table recognition methods detect and recognize each straight line in the table, recognize the straight lines and form the table by using the characteristics of the straight lines, do not well utilize the mutual relationship between the straight lines in the table to find the correct combination of the straight lines to form the table, and do not well have robustness to noise. SUMMARY

[0004] The purpose of the present application is to provide a table detection and recognition method, device, equipment and storage medium to solve the problem of poor robustness to noise of the table recognition method in related technologies.

[0005] In a first aspect, the present application provides a table detection and recognition method, which comprises:

[0006] performing straight line detection on a target image by using a straight line detection algorithm model to obtain a straight line detection result;

[0007] divide the straight line detection result into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result;

[0008] divide the straight lines into at least one straight line combination corresponding to different straight line classifications according to different position relationships of the straight lines in the straight line detection result corresponding to the different straight line classifications;

[0009] determine the distance between any two straight lines in the straight line combination corresponding to the different straight line classifications, and obtain a feature vector of the straight line combination corresponding to the different straight line classifications according to the distance between any two straight lines in the straight line combination;

[0010] input the feature vectors of the straight line combinations of each straight line classification as input vectors into a table classifier corresponding to the straight line classification to perform classification processing, and obtain a target straight line combination corresponding to the different straight line classifications;

[0011] determining intersection points of the lines in the target line combination of the different line categories, and obtaining coordinates of the table field region in the target image according to the intersection points of the lines.

[0012] In a possible implementation, the lines in the line detection result are classified into line detection results corresponding to different line categories according to the coordinates of the lines in the line detection result, including:

[0013] horizontal line detection results of a horizontal line category are obtained according to the coordinates of the horizontal lines in the line detection result; and

[0014] vertical line detection results of a vertical line category are obtained according to the coordinates of the vertical lines in the line detection result.

[0015] In a possible implementation, the lines are classified into at least one line combination corresponding to different line categories according to different position relationships of the lines in the line detection result, including:

[0016] horizontal lines are classified into at least one horizontal line combination corresponding to a horizontal line category according to different position relationships of the horizontal lines in the horizontal line detection result;

[0017] vertical lines are classified into at least one vertical line combination corresponding to a vertical line category according to different position relationships of the vertical lines in the vertical line detection result.

[0018] In a possible implementation, distances between any two lines in a line combination corresponding to different line categories are determined, and a feature vector of the line combination corresponding to a line category is obtained according to the distances between any two lines in the line combination, including:

[0019] distances between any two horizontal lines in a line combination corresponding to a horizontal line category are determined, and a feature vector of the line combination corresponding to the horizontal line category is obtained according to the distances between any two horizontal lines in the horizontal line combination;

[0020] distances between any two vertical lines in a line combination corresponding to a vertical line category are determined, and a feature vector of the line combination corresponding to the vertical line category is obtained according to the distances between any two vertical lines in the vertical line combination.

[0021] In a possible implementation, the feature vectors of the line combinations of the different line categories are taken as input vectors, and a table classifier corresponding to a line category is input for classification processing to obtain target line combinations of the different line categories corresponding to the table feature vectors, including:

[0022] inputting the feature vector of the direct combination of the horizontal straight line classification as an input vector into a table classifier corresponding to the horizontal straight line to perform classification processing, to obtain a target horizontal straight line combination of the horizontal straight line classification corresponding to the table feature vector;

[0023] inputting the feature vector of the direct combination of the vertical straight line classification as an input vector into a table classifier corresponding to the vertical straight line to perform classification processing, to obtain a target vertical straight line combination of the vertical straight line classification corresponding to the table feature vector.

[0024] In a possible implementation, the intersection of the straight lines in the target straight line combination of different straight line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the straight lines, including:

[0025] determining the intersection of the straight lines in the target horizontal straight line combination of the horizontal straight line classification and the straight lines in the target vertical straight line combination of the vertical straight line classification;

[0026] obtaining the coordinates of the table field region in the target image according to the intersection.

[0027] In a possible implementation, the method further includes:

[0028] Before performing straight line detection on the target image by using the straight line detection algorithm model, performing tilt correction processing on the input image by using an image preprocessing model to obtain the target image, and performing the step of performing straight line detection on the target image by using the straight line detection algorithm model.

[0029] In a second aspect, the present application provides a table detection and recognition device, the device includes:

[0030] a straight line detection module configured to perform straight line detection on a target image by using a straight line detection algorithm model to obtain a straight line detection result;

[0031] a straight line classification module configured to divide the straight line detection result into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result;

[0032] a straight line grouping module configured to divide the straight lines into at least one straight line combination corresponding to different straight line classifications according to different positional relationships of the straight lines in the straight line detection result;

[0033] a feature vector acquisition module configured to determine the distance between any two straight lines in a straight line combination corresponding to different straight line classifications, and obtain a feature vector of the straight line combination corresponding to the straight line classification according to the distance between any two straight lines in the straight line combination;

[0034] The feature vector classification module is configured to input the directly combined feature vectors of the straight lines classified as input vectors into a table classifier for classification processing to obtain target straight line combinations corresponding to different straight line classifications;

[0035] The table content determination module is configured to determine intersection points of straight lines in the target straight line combinations of different straight line classifications, and obtain coordinates of a table field region in the target image according to the intersection points of the straight lines.

[0036] In a possible implementation, the straight lines in the straight line detection results are classified into straight line detection results corresponding to different straight line classifications according to coordinates of the straight lines in the straight line detection results, and the straight line classification module is configured to:

[0037] horizontal straight line detection results of a horizontal straight line classification are obtained according to coordinates of the horizontal straight lines in the straight line detection results; and

[0038] vertical straight line detection results of a vertical straight line classification are obtained according to coordinates of the vertical straight lines in the straight line detection results.

[0039] In a possible implementation, the straight lines are classified into at least one straight line combination corresponding to different straight line classifications according to different position relationships of the straight lines in the straight line detection results, and the straight line grouping module is configured to:

[0040] horizontal straight lines are classified into at least one horizontal straight line combination corresponding to a horizontal straight line classification according to different position relationships of the horizontal straight lines in the horizontal straight line detection results;

[0041] vertical straight lines are classified into at least one vertical straight line combination corresponding to a vertical straight line classification according to different position relationships of the vertical straight lines in the vertical straight line detection results.

[0042] In a possible implementation, distances between any two straight lines in a straight line combination corresponding to different straight line classifications are determined, and a feature vector of the straight line combination corresponding to the straight line classification is obtained according to the distances between any two straight lines in the straight line combination, and the feature vector acquisition module is configured to:

[0043] distances between any two horizontal straight lines in a horizontal straight line combination corresponding to a horizontal straight line classification are determined, and a feature vector of the straight line combination corresponding to the horizontal straight line classification is obtained according to the distances between any two horizontal straight lines in the horizontal straight line combination;

[0044] distances between any two vertical straight lines in a vertical straight line combination corresponding to a vertical straight line classification are determined, and a feature vector of the straight line combination corresponding to the vertical straight line classification is obtained according to the distances between any two vertical straight lines in the vertical straight line combination.

[0045] In a possible implementation, a feature vector of a direct combination of the different horizontal line classifications is taken as an input vector, a table classifier corresponding to the horizontal line classification is inputted for classification processing, and a target horizontal line combination of the horizontal line classification corresponding to the table feature vector is obtained.

[0046] A feature vector of a direct combination of the different horizontal line classifications is taken as an input vector, a table classifier corresponding to the horizontal line classification is inputted for classification processing, and a target horizontal line combination of the horizontal line classification corresponding to the table feature vector is obtained.

[0047] A feature vector of a direct combination of the different horizontal line classifications is taken as an input vector, a table classifier corresponding to the horizontal line classification is inputted for classification processing, and a target horizontal line combination of the horizontal line classification corresponding to the table feature vector is obtained.

[0048] In a possible implementation, a point of intersection of a line in the target line combination of the different line classifications is determined, and a coordinate of a table field region in the target image is obtained according to the point of intersection, and the table content determination module is configured to:

[0049] A point of intersection of a line in the target horizontal line combination of the horizontal line classifications and a line in the target vertical line combination of the vertical line classifications is determined.

[0050] A coordinate of a table field region in the target image is obtained according to the point of intersection.

[0051] In a possible implementation, the apparatus further includes:

[0052] An image preprocessing module is configured to, before performing line detection on the target image by using the line detection algorithm model, perform tilt correction processing on the input image by using an image preprocessing model, obtain the target image, and perform the step of performing line detection on the target image by using the line detection algorithm model.

[0053] In a third aspect, the present application provides an electronic device, including:

[0054] A display is configured to display an image.

[0055] A memory is configured to store executable instructions of a processor.

[0056] The processor is configured to execute the executable instructions to implement the table detection and recognition method according to any one of the first aspect.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the table detection and recognition method according to any one of the first aspect.

[0058] In a fifth aspect, the present application provides a computer program product, comprising a computer program:

[0059] The computer program is executed by a processor to implement the table detection and recognition method according to any one of the first aspect.

[0060] The embodiments of the present application at least have the following beneficial effects:

[0061] In the embodiments of the present application, the straight line detection algorithm model is used to detect the target image to obtain a straight line detection result, then the straight line detection result is divided into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result, and the straight lines are divided into at least one straight line combination corresponding to different straight line classifications according to the different position relationships of the straight lines in the straight line detection results corresponding to different straight line classifications, then the distance between any two straight lines in the straight line combination corresponding to different straight line classifications is determined, the feature vector of the straight line combination corresponding to the straight line classification is obtained according to the distance between any two straight lines in the straight line combination, the feature vector of the straight line combination of each straight line classification is taken as an input vector, and the corresponding table classifier is classified to obtain the target straight line combination corresponding to different straight line classifications. Finally, the intersection of the straight lines in the target straight line combination corresponding to different straight line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the straight lines. The present application is aimed at the text recognition scene in which the table needs to be recognized first, the correct straight line combination is obtained by classifying the relationship between the straight lines, so that the correct table is determined, the robustness of the table recognition is enhanced, each field region in the table is better positioned, more accurate position information is provided for the next text recognition, and the recognition accuracy can be effectively improved.

[0062] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings introduced below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0064] Figure 1 An application scenario diagram provided for an embodiment of the present application is shown in FIG. 1.

[0065] Figure 2 A whole flow diagram of a table detection and recognition method provided for an embodiment of the present application is shown in FIG. 2.

[0066] Figure 3 A diagram of a straight line detection and recognition result provided for an embodiment of the present application is shown in FIG. 3.

[0067] Figure 4 A diagram of a horizontal straight line detection and recognition result and a vertical straight line detection and recognition result provided for an embodiment of the present application is shown in FIG. 4.

[0068] Figure 5 A flow diagram of step 206 provided for an embodiment of the present application is shown in FIG. 5.

[0069] Figure 6 A comparison diagram of a target image and a table detection and recognition effect image provided for an embodiment of the present application is shown in FIG. 6.

[0070] Figure 7 A structural diagram of a table detection and recognition device provided for an embodiment of the present application is shown in FIG. 7.

[0071] Figure 8 A structural diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 8. DETAILED DESCRIPTION

[0072] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below clearly and completely with reference to the drawings in the embodiments of the present application. The described embodiments are some embodiments of the present application, but not all embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0073] In addition, in the description of the embodiments of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; “and / or” in the text only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, “multiple” means two or more than two.

[0074] The terms "first", "second", "third", etc. are used only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second", "third", etc. can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0075] In the related art, the table recognition method detects and recognizes each straight line in the table, and uses the characteristics of the straight line itself to recognize the straight line and form the table, without well utilizing the mutual relationship between the straight lines in the table to find the correct combination of straight lines forming the table, and without good robustness to noise.

[0076] Therefore, the present application provides a table detection and recognition method, device, equipment and storage medium to solve the problem of poor robustness to noise of the table recognition method in the related art.

[0077] The inventive concept of the present application can be summarized as follows: by adopting a straight line detection algorithm model to detect straight lines in a target image, obtaining a straight line detection result, then according to the coordinates of the straight lines in the straight line detection result, dividing the straight line detection result into straight line detection results corresponding to different straight line classifications, and according to the different position relationships of the straight lines in the straight line detection results corresponding to different straight line classifications, dividing the straight lines into at least one straight line combination corresponding to different straight line classifications, then determining the distance between any two straight lines in the straight line combination corresponding to different straight line classifications, obtaining a feature vector of the straight line combination corresponding to the straight line classification according to the distance between any two straight lines in the straight line combination, taking the feature vectors of the straight line combinations of each straight line classification as input vectors, inputting the table classifier corresponding to the straight line classification for classification processing, obtaining the target straight line combination of the corresponding different straight line classification, finally, determining the intersection points of the straight lines in the target straight line combination of different straight line classifications, and obtaining the coordinates of the table field region in the target image according to the intersection points of the straight lines. The present application is aimed at the text recognition scene which needs to recognize the table first, and the correct straight line combination is obtained by classifying the relationship between the straight lines, so as to determine the correct table, thereby enhancing the robustness of table recognition, better positioning each field region in the table, providing more accurate position information for the next step of text recognition, and effectively improving the recognition accuracy.

[0078] After introducing the main inventive idea of the embodiments of the present application, the application scenarios to which the technical solutions of the embodiments of the present application can be applied will be briefly introduced below. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application, but not to limit. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0079] Referring to Figure 1As shown in the figure, it is a schematic diagram of an application scenario of a table detection and recognition method provided by an embodiment of the present application.

[0080] The figure includes: a network 10, a server 20, a storage 30, and an electronic device. The server 20 connects the electronic device through the network. In a text recognition scenario of recognizing a table in advance, for example, some fixed format text recognition scenarios containing a table such as invoice recognition, it is necessary to first locate the positions of various fields in the invoice, and then perform OCR (optical character recognition) text recognition on each field. By using the method provided by the embodiment of the present application, image data can be obtained by using the electronic device, and table recognition and text recognition can be performed on the image formed by the image data.

[0081] In the description of the present application, only a single server or electronic device is described in detail, but those skilled in the art should understand that the electronic device, network 10, server 20 and storage 30 shown are intended to represent the operation of the electronic device, server and storage involved in the technical solution of the present application. The single server and storage are described in detail at least for the convenience of explanation, and not to imply a limitation on the number, type or location of the electronic device and server. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed therefrom, the underlying concept of the example embodiments of the present application will not change. In addition, although the bidirectional arrow from the storage 30 to the server 20 is shown in the figure for the convenience of explanation, those skilled in the art can understand that the transmission and reception of the above-mentioned data also needs to be realized through the network 10. Figure 1

[0082] It should be noted that the storage in the embodiments of the present application may, for example, be a cache system, a hard disk storage, a memory storage, etc. In addition, the article detection and recognition method proposed in the present application is not only applicable to the application scenario shown, but can also be used in other possible application scenarios, and the embodiments of the present application do not limit the same. Figure 1 The application scenario shown, the application scenario shown, can also be used in other possible application scenarios, and the embodiments of the present application do not limit the same.

[0083] In order to facilitate understanding of the table detection and recognition method provided by the embodiments of the present application, the following will be further described in conjunction with the drawings.

[0084] In one possible implementation, the present application provides a table detection and recognition method, the overall flow chart of which is shown in Figure 2 as follows:

[0085] In step 201, a straight line detection algorithm model is used to detect a target image to obtain a straight line detection result. The straight line detection algorithm includes but is not limited to Hough transform, neural network and other algorithms or neural networks capable of realizing straight line detection.

[0086] ​It should be noted that in a possible implementation, before the target image is subjected to the straight line detection algorithm model for straight line detection, the embodiment of the present application will adopt an image preprocessing model to perform tilt correction processing on the input image to obtain a target image, and then execute the step 201 of adopting the straight line detection algorithm model to perform straight line detection on the target image. By correcting the inclined line segments in the image, the error caused by the inclined line segments in the straight line recognition is avoided, and the straight line detection accuracy and efficiency of the straight line detection algorithm model are improved.

[0087] In step 202, according to the coordinates of the straight lines in the straight line detection result, the straight line detection result is divided into the straight line detection results corresponding to different straight line classifications.

[0088] The straight line detection performed by the present application on the target image can simultaneously obtain horizontal and vertical straight line detection results. As shown in FIG. 1, the light-colored straight lines in the rectangular box are the horizontal and vertical straight line detection results. In order to more accurately recognize the table, the embodiment of the present application divides the straight line detection result into the straight line detection results corresponding to different straight line classifications, that is, the horizontal straight line detection result of the horizontal straight line classification and the vertical straight line detection result of the vertical straight line classification. As shown in FIG. 2, the image located on the upper side is the horizontal straight line detection result, and the image located on the lower side is the vertical straight line detection result of the vertical straight line classification. Figure 3 Figure 4

[0089] In a possible implementation, according to the coordinates of the straight lines in the straight line detection result, the straight line detection result is divided into the straight line detection results corresponding to different straight line classifications, including the following two cases:

[0090] 1. According to the coordinates of the horizontal straight lines in the straight line detection result, the horizontal straight line detection result of the horizontal straight line classification is obtained.

[0091] 2. According to the coordinates of the vertical straight lines in the straight line detection result, the vertical straight line detection result of the vertical straight line classification is obtained.

[0092] For example, the Y coordinates (the horizontal axis coordinate is the X coordinate, and the vertical axis coordinate is the Y coordinate) of the start point and the end point of the horizontal straight line in the straight line detection result are the same, so the present application extracts the straight line with the same Y coordinates of the start point and the end point from the straight line detection result to obtain the horizontal straight line detection result of the horizontal straight line classification. Similarly, the straight line with the same X coordinates of the start point and the end point is extracted from the straight line detection result to obtain the vertical straight line detection result of the vertical straight line classification.

[0093] In step 203, according to the different position relationships of the straight lines in the straight line detection results corresponding to different straight line classifications, the straight lines are divided into at least one straight line combination corresponding to different straight line classifications.

[0094] ​​In a possible implementation, in step 203, the straight lines are divided into at least one straight line combination corresponding to different straight line classifications according to different position relationships of the straight lines in the straight line detection result, including the following two cases:

[0095] 1. The horizontal straight lines are divided into at least one horizontal straight line combination corresponding to the horizontal straight line classification according to different position relationships of the horizontal straight lines in the horizontal straight line detection result.

[0096] 2. The vertical straight lines are divided into at least one vertical straight line combination corresponding to the vertical straight line classification according to different position relationships of the vertical straight lines in the vertical straight line detection result.

[0097] For example, the number of horizontal straight lines in the horizontal straight line detection result is 20, and the horizontal straight lines are divided into at least one horizontal straight line combination corresponding to the horizontal straight line classification according to different position relationships of the 20 horizontal straight lines, that is, the 20 horizontal straight lines are arranged and combined to obtain a plurality of horizontal straight line combinations. Similarly, the number of vertical straight lines in the vertical straight line detection result is 15, and the 15 vertical straight lines are arranged and combined to obtain a plurality of vertical straight line combinations.

[0098] In a possible implementation, the straight lines are divided into at least one straight line combination corresponding to different straight line classifications, which can be grouped according to a plurality of rules. For example, for the horizontal straight line classification, the number of horizontal straight lines in the horizontal straight line detection result is 20, and the grouping rule can be set according to the number of horizontal straight lines in the table, for example, the number of horizontal straight lines in the table is N, N is less than 20, and the application embodiment can set that each specified number of horizontal straight lines is divided into a group, and the specified number is set to a value equal to N; the grouping rule can also be set according to the height of the table, and the distance between any two horizontal straight lines in a horizontal straight line combination is not greater than a specified distance, for example, the height of the table is H, and the application embodiment can set that the distance between any two horizontal straight lines in a horizontal straight line combination is not greater than a specified distance, and the specified distance is set to a value slightly greater than H. For the vertical straight line classification, the principle of setting the grouping rule is the same, and the application will not be described here.

[0099] In step 204, the distance between any two straight lines in the straight line combination corresponding to different straight line classifications is determined, and the feature vector of the straight line combination corresponding to the straight line classification is obtained according to the distance between any two straight lines in the straight line combination.

[0100] In a possible implementation, in step 204, the distance between any two straight lines in the straight line combination corresponding to different straight line classifications is determined, and the feature vector of the straight line combination corresponding to the straight line classification is obtained according to the distance between any two straight lines in the straight line combination, including the following two cases:

[0101] 1. Determine the distance between any two horizontal straight lines in the straight line combination corresponding to the horizontal straight line classification, and obtain the feature vector of the straight line combination corresponding to the horizontal straight line classification according to the distance between any two horizontal straight lines in the horizontal straight line combination.

[0102] 2. Determine the distance between any two vertical straight lines in the straight line combination corresponding to the vertical straight line classification, and obtain the feature vector of the straight line combination corresponding to the vertical straight line classification according to the distance between any two vertical straight lines in the vertical straight line combination.

[0103] It should be noted that if the number of straight line combinations corresponding to different straight line classifications is N, the distance between any two straight lines in any group of straight line combinations will be determined and normalized to obtain the feature vector of this group of straight line combinations. Finally, N feature vectors corresponding to N groups of straight line combinations are obtained.

[0104] In step 205, the feature vectors of the straight line combinations of each straight line classification are input as input vectors into the table classifier corresponding to the straight line classification for classification processing to obtain the target straight line combination corresponding to different straight line classifications.

[0105] Since the straight line classification is horizontal straight line classification and vertical straight line classification, in order to more accurately obtain the target straight line combination, the present application will use the table classifier corresponding to the straight line classification for classification processing, including the first table classifier corresponding to the horizontal straight line and the second table classifier corresponding to the vertical straight line.

[0106] In one possible implementation, in step 205, the feature vectors of the direct combinations of different straight line classifications are input as input vectors into the table classifier corresponding to the straight line classification for classification processing to obtain the target straight line combination corresponding to different straight line classifications of the table feature vector, including the following two cases:

[0107] 1. The feature vectors of the direct combinations of the horizontal straight line classification are input as input vectors into the first table classifier corresponding to the horizontal straight line for classification processing to obtain the target horizontal straight line combination corresponding to the first table feature vector of the horizontal straight line classification.

[0108] 2. The feature vectors of the direct combinations of the vertical straight line classification are input as input vectors into the second table classifier corresponding to the vertical straight line for classification processing to obtain the target vertical straight line combination corresponding to the second table feature vector of the vertical straight line classification.

[0109] For example, the number of direct combinations of horizontal straight line classification is 20, and the number of corresponding feature vectors is also 20. The 20 feature vectors are taken as input vectors to input the first table classifier corresponding to the horizontal straight line to perform classification processing, and finally the target horizontal straight line combination of the horizontal straight line classification corresponding to the first table feature vector is obtained. The first table feature vector is the feature vector that achieves good classification effect for the first table classifier, that is, the correct table horizontal direction straight line combination. The feature vectors of the direct combinations of the vertical straight line classification are taken as input vectors to realize the acquisition of the target vertical straight line combination. The principle of the process is the same, and details are not described herein.

[0110] In step 206, the intersection of the lines in the target line combination of different line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the lines.

[0111] In one possible implementation, in step 206, the intersection of the lines in the target line combination of different line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the lines. The flowchart is as shown in Figure 5 , which includes:

[0112] In step 501, the intersection of the lines in the target horizontal straight line combination of the horizontal straight line classification and the lines in the target vertical straight line combination of the vertical straight line classification is determined.

[0113] In step 502, the coordinates of the table field region in the target image are obtained according to the intersection.

[0114] It should be noted that after obtaining the target horizontal straight line combination and the target vertical straight line combination, all lines in the two directions are combined to obtain the lines in the table (including horizontal straight lines and vertical straight lines). The present application will first determine the intersection coordinates of the horizontal straight lines and the vertical straight lines in the table, and then obtain the coordinates of the table field region in the target image according to the intersection. As shown in Figure 6 , the left is the target image, in which there is a table. The identification effect of the table detection and identification method provided by the present application is as shown in Figure 6 , the present application obtains all intersection information of the table, and the table detection result is more accurate and more robust to noise.

[0115] In summary, by adopting the straight line detection algorithm model to perform straight line detection on the target image, a straight line detection result is obtained, then according to the coordinates of the straight lines in the straight line detection result, the straight line detection result is divided into straight line detection results corresponding to different straight line classifications, and according to different position relationships of the straight lines in the straight line detection results corresponding to different straight line classifications, the straight lines are divided into at least one straight line combination corresponding to different straight line classifications, then the distance between any two straight lines in the straight line combination corresponding to different straight line classifications is determined, the feature vector of the straight line combination corresponding to the straight line classification is obtained according to the distance between any two straight lines in the straight line combination, the feature vectors of the straight line combinations of each straight line classification are taken as input vectors, and the table classifier corresponding to the straight line classification is input for classification processing to obtain the target straight line combination of the corresponding different straight line classification, finally, the intersection of the straight lines in the target straight line combination of different straight line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the straight lines. The present application is aimed at the text recognition scene in which a table needs to be recognized first, the correct straight line combination is obtained by classifying the relationship between the straight lines, so that the correct table is determined, the robustness of table recognition is enhanced, each field region in the table is better positioned, more accurate position information is provided for the next step of text recognition, and the recognition accuracy can be effectively improved.

[0116] Based on the same inventive concept, the present application also provides a table detection and recognition device 700, as shown in Figure 7 The device 700 comprises:

[0117] A straight line detection module 701 is configured to adopt a straight line detection algorithm model to perform straight line detection on a target image to obtain a straight line detection result.

[0118] A straight line classification module 702 is configured to divide the straight line detection result into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result.

[0119] A straight line grouping module 703 is configured to divide the straight lines into at least one straight line combination corresponding to different straight line classifications according to different position relationships of the straight lines in the straight line detection result.

[0120] A feature vector acquisition module 704 is configured to determine the distance between any two straight lines in the straight line combination corresponding to different straight line classifications, and obtain the feature vector of the straight line combination corresponding to the straight line classification according to the distance between any two straight lines in the straight line combination.

[0121] A feature vector classification module 705 is configured to take the feature vectors of the straight line combinations of each straight line classification as input vectors, input the table classifier for classification processing, and obtain the target straight line combination of the corresponding different straight line classification.

[0122] The table content determination module 706 is configured to determine the intersection of lines in the target line combination of the different line categories, and obtain the coordinates of the table field region in the target image according to the intersection of the lines.

[0123] In a possible implementation, the line classification module is configured to perform the following steps:

[0124] obtain the horizontal line detection result of the horizontal line category according to the coordinates of the horizontal line in the line detection result; and

[0125] obtain the vertical line detection result of the vertical line category according to the coordinates of the vertical line in the line detection result.

[0126] In a possible implementation, the line grouping module is configured to perform the following steps:

[0127] group the horizontal lines in the horizontal line detection result into at least one horizontal line combination corresponding to the horizontal line category according to the different position relationships of the horizontal lines in the horizontal line detection result;

[0128] group the vertical lines in the vertical line detection result into at least one vertical line combination corresponding to the vertical line category according to the different position relationships of the vertical lines in the vertical line detection result.

[0129] In a possible implementation, the feature vector acquisition module is configured to perform the following steps:

[0130] determine the distance between any two horizontal lines in the line combination corresponding to the horizontal line category, and obtain the feature vector of the line combination corresponding to the horizontal line category according to the distance between any two horizontal lines in the horizontal line combination;

[0131] determine the distance between any two vertical lines in the line combination corresponding to the vertical line category, and obtain the feature vector of the line combination corresponding to the vertical line category according to the distance between any two vertical lines in the vertical line combination.

[0132] In a possible implementation, the feature vector classification module is configured to perform the following steps:

[0133] The feature vector of the direct combination of the horizontal straight line classification is taken as an input vector, and a table classifier corresponding to the horizontal straight line is input for classification processing to obtain a target horizontal straight line combination of the horizontal straight line classification corresponding to the table feature vector.

[0134] The feature vector of the direct combination of the vertical straight line classification is taken as an input vector, and a table classifier corresponding to the vertical straight line is input for classification processing to obtain a target vertical straight line combination of the vertical straight line classification corresponding to the table feature vector.

[0135] In a possible implementation, a determination is performed on an intersection of straight lines in the target straight line combination of different straight line classifications, and coordinates of a table field region in the target image are obtained according to the intersection of the straight lines, and the table content determination module is configured to:

[0136] An intersection of a straight line in the target horizontal straight line combination of the horizontal straight line classification and a straight line in the target vertical straight line combination of the vertical straight line classification is determined.

[0137] Coordinates of a table field region in the target image are obtained according to the intersection.

[0138] In a possible implementation, the apparatus further includes:

[0139] An image preprocessing module is configured to, before performing straight line detection on the target image by using the straight line detection algorithm model, perform tilt correction processing on the input image by using an image preprocessing model, obtain the target image, and perform the step of performing straight line detection on the target image by using the straight line detection algorithm model.

[0140] The electronic device 130 according to this embodiment of the present application will be described below with reference to Figure 8 FIG. 1. Figure 8 The electronic device 130 shown is merely an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0141] As shown in Figure 8 The electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 can include, but are not limited to, the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components, including the memory 132 and the processor 131.

[0142] The bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor or local bus using any of a variety of bus structures.

[0143] The memory 132 can include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and can further include read only memory (ROM) 1323.

[0144] The memory 132 can also include a program / utility 1325 having a set of programs / modules 1324, including an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, which can include implementation of a network environment, for example.

[0145] The electronic device 130 can also communicate with one or more external devices 134 such as a keyboard or a pointing device, by way of I / O interface 135. Furthermore, the electronic device 130 can communicate with one or more devices that enable user interaction with the electronic device 130, and / or one or more devices that enable communication of the electronic device 130 with one or more other electronic devices. This communication can be via an I / O interface 135. The electronic device 130 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet) by way of a network adapter 136. As illustrated, the network adapter 136 communicates with the other modules of the electronic device 130 by way of the bus 133. It should be appreciated that other hardware and / or software modules can be used in conjunction with the electronic device 130, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0146] In an example embodiment, the present application also provides a computer readable storage medium including instructions, such as the memory 132 including instructions, which can be executed by the processor 131 of the electronic device 130 to complete the above-described table detection and recognition method. Optionally, the computer readable storage medium can be a non-transitory computer readable storage medium, such as a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0147] In an example embodiment, a computer program product is also provided, including a computer program which, when executed by the processor 131, implements the table detection and recognition method as provided by the present application.

[0148] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0149] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing machine, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0150] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0152] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A form detection and recognition method, characterized by, The method comprises: performing straight line detection on the target image using a straight line detection algorithm model to obtain a straight line detection result; dividing the straight line detection result into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result; the straight line classifications include a horizontal straight line classification and a vertical straight line classification, and the straight line detection results include horizontal straight line detection results and vertical straight line detection results; grouping the straight lines in the straight line detection results corresponding to the different straight line classifications into at least one straight line combination corresponding to each straight line classification according to different position relationships of the straight lines in the straight line detection results corresponding to the different straight line classifications and a preset grouping rule; the preset grouping rule includes grouping each specified number of straight lines into one straight line combination, and the straight line combinations include horizontal straight line combinations and vertical straight line combinations; determining the distance between any two straight lines in the straight line combinations corresponding to the different straight line classifications, and obtaining a feature vector of the straight line combination corresponding to each straight line classification according to the distance between any two straight lines in the straight line combination; inputting the feature vectors of the straight line combinations of each straight line classification as input vectors into a table classifier corresponding to each straight line classification to perform classification processing, and obtaining target straight line combinations of different straight line classifications corresponding to the table feature vectors; determining the intersection points of the straight lines in the target straight line combinations of the different straight line classifications, and obtaining the coordinates of the table field region in the target image according to the intersection points of the straight lines.

2. The method of claim 1, wherein, dividing the straight line detection result into straight line detection results corresponding to different straight line classifications according to the coordinates of the straight lines in the straight line detection result, comprises: obtaining horizontal straight line detection results of the horizontal straight line classification according to the coordinates of the horizontal straight lines in the straight line detection result; and obtaining vertical straight line detection results of the vertical straight line classification according to the coordinates of the vertical straight lines in the straight line detection result.

3. The method of claim 2, wherein, grouping the straight lines in the straight line detection results corresponding to the different straight line classifications into at least one straight line combination corresponding to each straight line classification according to different position relationships of the straight lines in the straight line detection results corresponding to the different straight line classifications, comprises: grouping the horizontal straight lines in the horizontal straight line detection results into at least one horizontal straight line combination corresponding to the horizontal straight line classification according to different position relationships of the horizontal straight lines in the horizontal straight line detection results; the preset grouping rule includes grouping each specified number of horizontal straight lines into one horizontal straight line combination or grouping any two horizontal straight lines into one horizontal straight line combination if the distance between the any two horizontal straight lines is not greater than a specified distance; grouping the vertical straight lines in the vertical straight line detection results into at least one vertical straight line combination corresponding to the vertical straight line classification according to different position relationships of the vertical straight lines in the vertical straight line detection results; the preset grouping rule includes grouping each specified number of vertical straight lines into one vertical straight line combination or grouping any two vertical straight lines into one vertical straight line combination if the distance between the any two vertical straight lines is not greater than a specified distance.

4. The method of claim 3, wherein, determining the distance between any two straight lines in the straight line combinations corresponding to the different straight line classifications, and obtaining a feature vector of the straight line combination corresponding to each straight line classification according to the distance between any two straight lines in the straight line combination, comprises: determining the distance between any two horizontal straight lines in the straight line combinations corresponding to the horizontal straight line classification, and obtaining a feature vector of the straight line combination corresponding to the horizontal straight line classification according to the distance between any two horizontal straight lines in the horizontal straight line combination; The distance between any two vertical lines in the line combination corresponding to the vertical line classification is determined, and a feature vector of the line combination corresponding to the vertical line classification is obtained according to the distance between any two vertical lines in the vertical line combination.

5. The method of claim 4, wherein, The feature vector of the line combination of the different line classifications is taken as an input vector, and a table classifier corresponding to the line classification is input for classification processing to obtain a target line combination of the different line classifications corresponding to the table feature vector, including: The feature vector of the line combination of the horizontal line classification is taken as an input vector, and a first table classifier corresponding to the horizontal line is input for classification processing to obtain a target horizontal line combination of the horizontal line classification corresponding to the first table feature vector; The feature vector of the line combination of the vertical line classification is taken as an input vector, and a second table classifier corresponding to the vertical line is input for classification processing to obtain a target vertical line combination of the vertical line classification corresponding to the second table feature vector.

6. The method of claim 5, wherein, The intersection of lines in the target line combination of the different line classifications is determined, and the coordinates of the table field region in the target image are obtained according to the intersection of the lines, including: The intersection of lines in the target horizontal line combination of the horizontal line classification and the lines in the target vertical line combination of the vertical line classification is determined; The coordinates of the table field region in the target image are obtained according to the intersection.

7. The method of claim 1, wherein, The method further includes: Before performing line detection on the target image by using the line detection algorithm model, performing tilt correction processing on the input image by using an image preprocessing model to obtain the target image and perform the step of performing line detection on the target image by using the line detection algorithm model.

8. A form detection and recognition apparatus, characterized by, The device includes: A line detection module configured to perform line detection on the target image by using the line detection algorithm model to obtain a line detection result; A line classification module configured to divide the line detection result into line detection results corresponding to different line classifications according to the coordinates of the lines in the line detection result; the line classification includes a horizontal line classification and a vertical line classification, and the line detection result includes a horizontal line detection result and a vertical line detection result; A line grouping module configured to divide the lines into at least one line combination corresponding to different line classifications according to different positional relationships of the lines in the line detection result according to a preset grouping rule; the preset grouping rule includes dividing each specified number of lines into a line combination or dividing any two lines with a distance not greater than a specified distance into a line combination, and the line combination includes a horizontal line combination and a vertical line combination; A feature vector acquisition module configured to determine the distance between any two lines in the line combination corresponding to the different line classifications, and obtain a feature vector of the line combination corresponding to the line classification according to the distance between any two lines in the line combination; A feature vector classification module configured to take the feature vectors of the line combinations of the different line classifications as input vectors, and input a table classifier for classification processing to obtain a target line combination of the different line classifications corresponding to the table feature vector. The table content determination module is configured to determine intersection points of lines in the target line combination of the different line categories, and obtain coordinates of a table field region in the target image according to the intersection points of the lines.

9. An electronic device, comprising: The method comprises the following steps: a display for displaying an image; a memory for storing executable instructions of a processor; the processor is configured to execute the executable instructions to implement the table detection and recognition method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer readable storage medium are executed by the processor of the electronic device, the electronic device can execute the table detection and recognition method according to any one of claims 1-7.

Images