Artificial intelligence-based text recognition method, device, computer equipment and medium

Abstract

The present invention is applicable to the field of artificial intelligence, and in particular relates to a text recognition method, device, computer equipment and medium based on artificial intelligence. The present invention inputs a table image into a trained classification model, determines at least one row title area, and performs character recognition in each row title area to obtain at least two column title boxes. Then, the table image is divided into regions according to the positions of each column title box in each row title area, and each title row line is determined. The table image is divided into regions to obtain at least one table partition, and each title row line in the table partition is translated in a preset direction. The character boxes identified in the table partition are matched in the same line, and at least one group of character boxes in the same line is determined, and finally the recognized content of the table image is determined. The other characters in the table image are matched in the same line by the column titles of the known line, and each group of character boxes in the same line is determined to complete the content recognition of the table image, thereby improving the text recognition accuracy of the table image.

Description

Technical Field

[0001] The present invention is applicable to the field of artificial intelligence, and in particular relates to a text recognition method, device, computer equipment and medium based on artificial intelligence. Background Art

[0002] Image and text recognition, also known as optical character recognition (OCR), refers to the process in which electronic devices examine characters printed on paper, determine their shapes by detecting dark and light patterns, and then use character recognition methods to translate the shapes into computer text.

[0003] During the image and text recognition process, since the optical character recognition method does not perform tilt correction on the image, it has a good character recognition effect for table images without tilt or distortion. However, when the table image has distortion problems such as distortion, perspective, and different upper and lower slopes, the text recognition accuracy in the table image is low.

[0004] Therefore, in the field of image recognition, how to improve the accuracy of text recognition in distorted table images has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a text recognition method, apparatus, computer device, and medium based on artificial intelligence to solve the problem of low accuracy of text recognition in distorted table images by existing character recognition methods.

[0006] In a first aspect, an embodiment of the present invention provides a text recognition method based on artificial intelligence, the text recognition method comprising:

[0007] Obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row header area of the table image;

[0008] Performing character recognition in each of the row header areas to obtain at least two column header boxes in the row header area;

[0009] Determine each title row fold line according to the position of each column title frame in each row title area, and divide the table image into regions according to each title row fold line to obtain at least one table partition;

[0010] For any table partition, by translating each of the title row fold lines in the table partition in a preset direction, the character boxes identified in the table partition are matched in the same row to determine at least one group of character boxes in the same row;

[0011] The identification content of the table image is determined based on the column title box and the in-row character box in each of the row title areas.

[0012] In a second aspect, an embodiment of the present invention provides an artificial intelligence-based text recognition device, the text recognition device comprising:

[0013] A region classification module is used to obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row header region of the table image;

[0014] a title recognition module, configured to perform character recognition in each row title area to obtain at least two column title boxes in the row title area;

[0015] an area division module, configured to determine each title row fold line according to the position of each column title frame within each row title area, and divide the table image into areas according to each title row fold line to obtain at least one table partition;

[0016] A peer matching module is configured to perform peer matching on the character boxes identified in any table partition by translating each of the title row fold lines in the table partition in a preset direction, thereby determining at least one group of peer character boxes.

[0017] The content recognition module is used to determine the recognition content of the table image based on the column title box and the row character box in each row title area.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the text recognition method as described in the first aspect when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text recognition method as described in the first aspect is implemented.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: by inputting a table image into a trained classification model, determining at least one row title area of the table image, and performing character recognition in each row title area, obtaining at least two column title boxes in the row title area, then determining each title row fold line according to the position of each column title box in each row title area, dividing the table image into regions according to each title row fold line, obtaining at least one table partition, and for any table partition, performing peer matching on the identified character boxes in the table partition by translating each title row fold line in the table partition in a preset direction, determining at least one group of peer character boxes, and finally determining the recognized content of the table image based on the column title boxes and peer character boxes in each row title area. By performing peer matching on other characters in the table image through the known peer column titles, each group of peer character boxes is determined to complete the content recognition of the table image, thereby improving the text recognition accuracy of the table image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a schematic diagram of an application environment of an artificial intelligence-based text recognition method provided in Example 1 of the present invention;

[0023] Figure 2 This is a flowchart of an artificial intelligence-based text recognition method provided in Example 1 of the present invention;

[0024] Figure 3 This is a schematic structural diagram of an artificial intelligence-based text recognition device provided in the second embodiment of the present invention;

[0025] Figure 4 This is a structural diagram of a computer device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0026] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0027] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0028] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0029] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0033] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0034] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0035] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0036] The first embodiment of the present invention provides an artificial intelligence-based text recognition method that can be applied in Figure 1 In an application environment, a client communicates with a server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0037] See also Figure 2 , is a flowchart of a text recognition method based on artificial intelligence provided by the first embodiment of the present invention. The above text recognition method can be applied to Figure 1 In the client, the text recognition method may include the following steps:

[0038] Step S201: obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row title area of the table image.

[0039] Among them, the table image to be identified is a table image with distortion problems such as distortion, perspective, and different upper and lower slopes. The table image includes title content and non-title content, and due to the differences in fonts, number of words and other factors between the title content and the non-title content, the title content is generally easier to identify than the non-title content. Therefore, this embodiment first identifies the row title area in the table image according to the classification model. Specifically, the table image to be identified is input into the trained classification model for image segmentation to obtain a region segmentation map, and the region segmentation map is used as a mask to be fused with the corresponding table image to determine at least one row title area of the table image, so that the characters in the non-title area can be identified based on the characters in the identified row title area to improve the recognition accuracy of the distorted table image.

[0040] Optionally, the classification model includes an encoder and a decoder, the training sample is a table image, and the training label is a pixel annotation value in the training sample, wherein the training sample includes an actual row header area and an actual non-row header area, the pixel annotation value in the actual row header area is 1, and the pixel annotation value in the actual non-row header area is 0;

[0041] The training process of the classification model includes:

[0042] Input the table image into the encoder for feature extraction to obtain the table area features;

[0043] Input the table area features into the decoder for upsampling to obtain a region segmentation map, where the region segmentation map includes the row title area and the non-row title area. The pixel value of the row title area is 1, and the pixel value of the non-row title area is 0;

[0044] The loss function is calculated based on the pixel values in the region segmentation map and the pixel annotation values in the training samples. The parameters of the encoder and decoder are reversely corrected according to the gradient descent method until the loss function converges to obtain a trained classification model.

[0045] Image segmentation is the process of dividing an image into specific regions with unique properties and identifying objects of interest. In this example, a classification model is used to segment a table image and extract the row header regions within it. To process table images in batches during data processing, a pre-trained classification model is used to segment the table image and output a segmentation map to identify the row header regions within the table image.

[0046] Among them, the classification model includes an encoder and a decoder. The encoder is used to extract features of the input table image to obtain the regional features of the table image, and the decoder is used to decode the regional features and output the regional segmentation map.

[0047] In this embodiment, the classification model uses table images as training samples and pixel annotation values in the training samples as training labels, wherein the training samples include actual row title areas and actual non-row title areas. When training the classification model, the pixel annotation values of the actual row title areas are artificially set to 1, and the pixel annotation values of the actual non-row title areas are artificially set to 0, which is used to evaluate the region segmentation map output by the classification model, so that the classification model can be trained according to the evaluation results to obtain a trained classification model.

[0048] During the training process of the classification model, the table image is input into the encoder for feature extraction to obtain the table area features, and then the table area features are input into the decoder for upsampling to obtain the region segmentation map, where the region segmentation map includes the row title area and the non-row title area. The pixel value of the row title area is 1, and the pixel value of the non-row title area is 0.

[0049] Then, the loss function is calculated based on the pixel values in the region segmentation map and the pixel annotation values in the training samples. When the loss function is small, it means that the accuracy of the image segmentation model is high. When the loss function is large, it means that the accuracy of the image segmentation model is low. It is necessary to reversely correct the parameters of the encoder and decoder according to the gradient descent method until the loss function converges to obtain a trained classification model to improve the accuracy of the classification model.

[0050] For example, the label value of the i-th pixel in the table image as a training sample is recorded as B i , the pixel value of the i-th pixel in the region segmentation map output by the classification model is recorded as A i , where i = 1, 2, ..., N, N is the number of pixels in the table image and the region segmentation map, and N is a positive integer.

[0051] The loss function Loss is calculated based on the pixel values in the region segmentation map and the corresponding pixel annotation values in the training sample:

[0052]

[0053] Where A i is the pixel value of the i-th pixel in the region segmentation map, B i is the label value of the i-th pixel in the table image corresponding to the training sample, and N is the number of pixels in the table image and the region segmentation map.

[0054] The above steps include obtaining a table image to be identified, inputting the table image into a trained classification model, determining at least one row title area of the table image, performing image segmentation on the table image through the classification model, and obtaining the row title area in the table image, so that the characters in the non-title area can be identified based on the characters in the row title area, thereby improving the recognition accuracy of the distorted table image.

[0055] Step S202: Perform character recognition in each row header area to obtain at least two column header boxes in the row header area.

[0056] Among them, character recognition refers to the process of checking the characters printed on the target image through electronic equipment, determining the character shape by detecting the pattern of light and dark, and then translating the shape into computer text using character recognition methods. For example, OCR (optical character recognition) technology recognizes optical characters on the target image through image processing and pattern recognition technology.

[0057] In this embodiment, the optical characters in each row header area are recognized by OCR technology to obtain character boxes in each row header area. The character content in the character box is the column title in the table image, and at least two column title boxes can be obtained in each row header area.

[0058] The above-mentioned steps of performing character recognition in each row title area to obtain at least two column title boxes in the row title area, determining the column title box according to the characters, and serving as the basis for subsequent character recognition in the non-title area, so as to translate the characters in the table image into computer text, thereby improving the efficiency and accuracy of text recognition in the table image.

[0059] Step S203 , determining each title row fold line according to the position of each column title box in each row title area, and dividing the table image into regions according to each title row fold line to obtain at least one table partition.

[0060] Among them, the title row broken line is determined according to the position of each column title box in each row title area, and can be used to represent the peer trend of the column title box in the corresponding row title area, so as to be used to match the characters in the non-row title area.

[0061] Table partitions are obtained by dividing the table image into areas based on the various title row fold lines. Each table partition includes two title row fold lines that divide it. The two title row fold lines are distributed on the upper and lower sides of the corresponding table partition. The two title row fold lines can be combined to perform in-line matching of characters in the table partition to improve the accuracy of in-line matching of characters in the non-row title area, thereby improving the text recognition accuracy of the table image.

[0062] Optionally, each header row fold line is determined according to the position of each column header box within each row header area, including:

[0063] For any row header area, sort the column header boxes according to the center point coordinates of the column header boxes in the row header area;

[0064] According to the sorting results, the center points of the column title boxes are connected in sequence to obtain the corresponding title row polylines.

[0065] Among them, the column title box is a rectangular box, and the center point of the rectangular box is the center point of the column title box. In this embodiment, the center point coordinates of the column title box are used to represent the position of the column title box, wherein the center point coordinates refer to the coordinates of the center point in the table image.

[0066] Specifically, the column headers of the same row are arranged in sequence from left to right in the table. Therefore, in this embodiment, for any row header area, the center point coordinates of each column header box in the row header area are first determined, and the column header boxes are sorted in order from small to large according to the center point row coordinate values. Then, according to the sorting results, the center points of each column header box are connected in sequence from left to right to obtain the corresponding header row broken line.

[0067] In one embodiment, the column title frames may be sorted in descending order of the center point row coordinate values, and then the center points of the column title frames may be connected from right to left according to the sorting results to obtain the corresponding title row broken lines.

[0068] In one embodiment, the column title box is a rectangular box with four vertices. The coordinates of any vertex can be used to represent the position of the column title box, and the column title boxes are sorted in the order of the vertex coordinate values. According to the sorting results, the center points of the column title boxes are connected in sequence to obtain the corresponding title row broken line.

[0069] Optionally, the table image is divided into regions according to each title row fold line to obtain at least one table partition, including:

[0070] For any title row fold line, determine the leftmost center point segment and the rightmost center point segment that make up the title row fold line, extend the leftmost center point segment to intersect with the left edge of the table image, and extend the rightmost center point segment to intersect with the right edge of the table image to obtain the extended title row fold line;

[0071] Divide the area between two adjacent extended title row fold lines into a table partition, and obtain at least one table partition in the table image.

[0072] Among them, the title row broken line is obtained by connecting the center points of each column title box in sequence. The connecting line segment between the center points of two adjacent column title boxes can be used as the center point segment of the title row broken line, and the leftmost center point segment and the rightmost center point segment are determined according to the center point coordinates corresponding to each center point segment.

[0073] In this embodiment, since each column title box is some distance away from the edge of the table image, the table area cannot be completely divided according to the title row fold line alone. Therefore, first, the leftmost center point line segment is extended to intersect with the left edge of the table image, and the rightmost center point line segment is extended to intersect with the right edge of the table image to obtain the extended title row fold line. The extended title row fold line can then completely complete the area division of the table image.

[0074] Therefore, the area between two adjacent extended title row fold lines is determined to be a table partition, and the table image is divided into areas to obtain at least one table partition in the table image, wherein each table partition includes the corresponding two extended title row fold lines and the area between the two extended title row fold lines.

[0075] The above-mentioned step of determining each title row fold line according to the position of each column title box in each row title area, dividing the table image into areas according to each title row fold line, and obtaining at least one table partition can match other characters in the table partition in the same line according to the title row fold lines known to belong to the same line in the table partition, thereby improving the accuracy of the same line matching of characters in non-row title areas, thereby improving the text recognition accuracy of the table image.

[0076] Step S204 , for any table partition, by translating each header row line in the table partition in a preset direction, the identified character boxes in the table partition are matched in the same row to determine at least one group of character boxes in the same row.

[0077] Because the header row fold line is derived from the column header boxes that are known to belong to the same row, it can be used to represent the same-row trend at the corresponding position. Therefore, for two header row fold lines belonging to the same table partition and the area between them, the same-row trend of the two header row fold lines can be combined to perform the same-row matching on the characters in the middle area, thereby improving the accuracy of the same-row matching of the characters in the middle area of each table partition.

[0078] Specifically, for any table partition, first identify the character boxes within the table partition, and then perform peer matching on each character box by translating each title row line within the table partition in a preset direction within the table partition to determine the character boxes belonging to the same row, so as to obtain at least one group of peer character boxes.

[0079] Optionally, for any table partition, character boxes identified in the table partition are matched in the same row by translating each header row line in the table partition in a preset direction to determine at least one group of character boxes in the same row, including:

[0080] For any table partition, perform character recognition on the table partition to obtain the character frames within the table partition, and determine the character frames to be matched based on the center point coordinates of each character frame;

[0081] Determine the upper and lower vertices of each character box according to the vertex coordinates of the four vertices of each character box;

[0082] Determine the two header row fold lines belonging to the same table partition as an upper header row fold line and a lower header row fold line, translate the header row fold lines according to a first preset direction until the upper header row fold line passes through the upper vertex of the character box to be matched, and translate the lower header row fold line according to a second preset direction until the lower header row fold line passes through the lower vertex of the character box to be matched;

[0083] Calculate the first distance between each character box and the translated upper title line fold line, and the second distance between each character box and the translated lower title line fold line, and determine at least one group of character boxes in the same row based on the first distance and the second distance.

[0084] For any table partition, firstly, character recognition is performed on the table partition using OCR technology to obtain character frames within the table partition, and the center point coordinates of each character frame are determined to represent the position of each character frame.

[0085] Among them, the character box to be matched is used to perform peer matching on other character boxes in the same table partition, and determine the character box belonging to the same row as the character box to be matched, so as to improve the accuracy of text recognition in the table image. In this embodiment, the position of each character box is determined according to the coordinates of the center point of each character box, the character box to be matched.

[0086] In this embodiment, the character box is a rectangular box with four vertices. The upper vertex and lower vertex of each character box are determined based on the vertex coordinates of the four vertices of each character box. That is, for any character box, the vertical coordinate values of its four vertex coordinates are compared, and the vertex with the smallest vertical coordinate value is the upper vertex, and the vertex with the largest vertical coordinate value is the lower vertex.

[0087] When performing in-line matching of the character box to be matched based on the two title row fold lines in the table partition, first determine the upper title row fold line and the lower title row fold line based on the upper and lower positions of the two title row fold lines in the table image, and translate the title row fold line according to the first preset direction until the upper title row fold line passes the upper vertex of the character box to be matched, and translate the lower title row fold line according to the second preset direction until the lower title row fold line passes the lower vertex of the character box to be matched. Among them, the first preset direction is the vertical downward direction, and the second preset direction is the vertical upward direction, so that the upper title row fold line and the lower title row fold line are translated toward the middle character box to be matched to pass the upper vertex and the lower vertex of the character box to be matched. Then, the area between the translated upper title row fold line and the translated lower title row fold line can be determined, which is the table area belonging to the same row as the corresponding character box to be matched.

[0088] Then, for other character frames other than the character frame to be matched in the table partition, when the distance between the character frame and the table area is close, it can be determined that the character frame and the corresponding character frame to be matched belong to the same row. Since the upper and lower area edges of the table area are the upper title row fold line after translation and the lower title row fold line after translation, in this embodiment, the first distance between each character frame and the upper title row fold line after translation, and the second distance between each character frame and the lower title row fold line after translation are calculated, and the first distance and second distance of each character frame are used to determine the distance between each character frame and the corresponding table area, thereby determining the character frame whose distance meets the preset conditions and the corresponding character frame to be matched belong to the same row, thereby obtaining a group of character frames in the same row, so as to improve the text recognition accuracy of the table image by improving the accuracy of the same row matching of the character frames in each table partition.

[0089] Optionally, after determining at least one group of character frames in the same row, the method further includes:

[0090] For any table partition, filter out character frames in the same row and determine the retained character frames;

[0091] For the retained character frames, the steps of repeatedly determining the character frames to be matched based on the center point coordinates of each character frame, translating the title row fold line according to the first preset direction until the upper title row fold line passes the upper vertex of the character frame to be matched, translating the lower title row fold line according to the second preset direction until the lower title row fold line passes the lower vertex of the character frame to be matched, calculating the first distance between each character frame and the translated upper title row fold line, and the second distance between each character frame and the translated lower title row fold line, and determining at least one group of character frames in the same row based on the first distance and the second distance. After determining a character frame to be matched and completing the determination of a group of character frames in the same row, for other character frames that have not been determined as character frames in the same row, it is necessary to re-determine a character frame to be matched and match the corresponding character frames in the same row, and so on, until all character frames have completed the same row matching, and at least one group of character frames in the same row corresponding to the table partition is obtained.

[0092] Specifically, for any table partition, each group of in-line character frames determined after in-line matching will be screened out, and the character frames to be matched will be determined from the retained character frames based on the center point coordinates, so that the two title row fold lines corresponding to the table partition are translated in the preset direction, and in-line matching is performed on the character frames retained in the table partition, and the character frames to be matched and the character frames that meet the preset in-line conditions are determined to be a group of in-line character frames, so as to improve the matching accuracy of each group of in-line character frames.

[0093] Optionally, calculating a first distance between each character box and the translated upper title line fold line, and a second distance between each character box and the translated lower title line fold line, and determining at least one group of character boxes in the same row based on the first distance and the second distance includes:

[0094] Calculating a first distance between each character box and the translated upper title line fold line, and a second distance between each character box and the translated lower title line fold line;

[0095] Calculate the sum of the first distance and the second distance;

[0096] The sum of the distances is compared with a preset peer condition, and the character frame to be matched and the character frame satisfying the preset peer condition are determined to be a group of peer character frames.

[0097] Among them, when the distance between the character box and the table area is close, it can be determined that the character box and the corresponding character box to be matched belong to the same row. Therefore, in this embodiment, the first distance between the upper vertex of each character box and the translated upper title row fold line, and the second distance between the lower vertex of the character box and the translated lower title row fold line are calculated respectively, and the sum of the distances is obtained to characterize the distance between the character box and the table area.

[0098] The distance sum is then compared with a preset peer condition. In this embodiment, the preset peer condition is less than a preset threshold. That is, when the distance sum corresponding to a character frame is less than the preset threshold, the character frame satisfies the preset peer condition, thereby determining that the character frame to be matched and the character frames that satisfy the preset peer condition are a group of peer character frames. The preset threshold can be set based on the average distance between the translated upper title line fold line and the translated lower title line fold line. For example, if the calculated average distance between the translated upper title line fold line and the translated lower title line fold line is L, the preset threshold is set to 0.2L.

[0099] The above-mentioned method of translating each title row line in the table partition in a preset direction for any table partition performs peer matching on the character boxes identified in the table partition, and determines at least one group of peer character boxes. Based on each title row line, peer matching is performed on other character boxes in the corresponding table partition, and each group of peer character boxes in the table partition is determined, which greatly improves the text recognition accuracy of table images.

[0100] Step S205 , determining the recognition content of the table image based on the column title frame and the row character frame in each row title area.

[0101] Among them, the column title box in each row title area is the character box of the column title belonging to the same row, and the same row character box is the character box of the content characters belonging to the same row. By further combining the positions of each group of same row character boxes, the recognition content of the table image can be determined to complete the text recognition of the table image.

[0102] The above steps of determining the recognition content of the table image based on the column title box and the same row character box in each row title area combine the column title box and the same row character box with the corresponding positions, determine the recognition content of the table image, and improve the text recognition accuracy of the table image.

[0103] The embodiment of the present invention inputs a table image into a trained classification model, determines at least one row title area of the table image, and performs character recognition in each row title area to obtain at least two column title boxes in the row title area. Then, each title row fold line is determined according to the position of each column title box in each row title area, and the table image is divided into regions according to each title row fold line to obtain at least one table partition. For any table partition, each title row fold line in the table partition is translated in a preset direction, and the character boxes identified in the table partition are matched in the same line to determine at least one group of character boxes in the same line. Finally, the recognition content of the table image is determined based on the column title boxes and character boxes in the same line in each row title area. The other characters in the table image are matched in the same line by the column titles of the known same line, and each group of character boxes in the same line is determined to complete the content recognition of the table image, thereby improving the text recognition accuracy of the table image.

[0104] Corresponding to the text recognition method of the above embodiment, Figure 3 A structural block diagram of an artificial intelligence-based text recognition device provided in a second embodiment of the present invention is given. For ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0105] See also Figure 3 , the text recognition device comprises:

[0106] The region classification module 31 is used to obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row header region of the table image;

[0107] A title recognition module 32 is used to perform character recognition in each row title area to obtain at least two column title boxes in the row title area;

[0108] A region division module 33 is configured to determine each title row fold line according to the position of each column title frame within each row title region, and to divide the table image into regions according to each title row fold line to obtain at least one table partition;

[0109] A peer matching module 34 is configured to perform peer matching on the character boxes identified in any table partition by translating each header row line in the table partition in a preset direction to determine at least one group of peer character boxes.

[0110] The content recognition module 35 is used to determine the recognition content of the table image based on the column title box and the row character box in each row title area.

[0111] Optionally, the region classification module 31 includes:

[0112] The classification model determination submodule is used to determine that the classification model includes an encoder and a decoder. The training sample is a table image, and the training label is the pixel annotation value in the training sample. The training sample includes the actual row header area and the actual non-row header area. The pixel annotation value of the actual row header area is 1, and the pixel annotation value of the actual non-row header area is 0.

[0113] The feature extraction submodule is used to input the table image into the encoder for feature extraction to obtain the table area features;

[0114] The feature decoding submodule is used to input the table area features into the decoder for upsampling to obtain a region segmentation map, where the region segmentation map includes a row header area and a non-row header area. The pixel value of the row header area is 1, and the pixel value of the non-row header area is 0.

[0115] The parameter correction submodule is used to calculate the loss function based on the pixel values in the region segmentation map and the pixel annotation values in the training samples, and reversely correct the parameters of the encoder and decoder according to the gradient descent method until the loss function converges to obtain a trained classification model.

[0116] Optionally, the area division module 33 includes:

[0117] The sorting submodule is used to sort each column title frame in any row title area according to the center point coordinates of each column title frame in the row title area;

[0118] The broken line connection submodule is used to connect the center points of each column title box in sequence according to the sorting results to obtain the corresponding title row broken line.

[0119] Optionally, the area division module 33 includes:

[0120] The polyline extension submodule is used to determine the leftmost center point segment and the rightmost center point segment of any header row polyline, extend the leftmost center point segment to intersect with the left edge of the table image, and extend the rightmost center point segment to intersect with the right edge of the table image to obtain the extended header row polyline;

[0121] The area division submodule is used to divide the area between two adjacent extended title row fold lines into a table partition, thereby obtaining at least one table partition in the table image.

[0122] Optionally, the peer matching module 34 includes:

[0123] The first character frame selection submodule is used to perform character recognition on any table partition to obtain character frames within the table partition and determine the character frames to be matched based on the center point coordinates of each character frame;

[0124] A vertex determination submodule is used to determine the upper vertex and the lower vertex of each character box according to the vertex coordinates of the four vertices of each character box;

[0125] A polyline translation submodule is used to determine the two header row polylines belonging to the same table partition as an upper header row polyline and a lower header row polyline, translate the header row polylines according to a first preset direction until the upper header row polyline passes through the upper vertex of the character box to be matched, and translate the lower header row polyline according to a second preset direction until the lower header row polyline passes through the lower vertex of the character box to be matched;

[0126] The first in-line matching submodule is used to calculate the first distance between each character box and the translated upper title row fold line, and the second distance between each character box and the translated lower title row fold line, and determine at least one group of in-line character boxes based on the first distance and the second distance.

[0127] Optionally, the peer matching module 34 further includes:

[0128] The second character frame selection submodule is used to screen out character frames in the same row in any table partition to determine the retained character frames;

[0129] The second in-line matching submodule is used to repeatedly determine the character frames to be matched according to the center point coordinates of each character frame for the retained character frames, translate the title row fold line according to the first preset direction until the upper title row fold line passes through the upper vertex of the character frame to be matched, translate the lower title row fold line according to the second preset direction until the lower title row fold line passes through the lower vertex of the character frame to be matched, calculate the first distance between each character frame and the translated upper title row fold line, and the second distance between each character frame and the translated lower title row fold line, and determine at least one group of in-line character frames according to the first distance and the second distance.

[0130] Optionally, the first peer matching submodule includes:

[0131] a distance calculation unit, configured to calculate a first distance between each character box and the translated upper title line fold line, and a second distance between each character box and the translated lower title line fold line;

[0132] Calculate the sum of the first distance and the second distance;

[0133] The same-in-line matching unit is used to compare the distance sum with a preset same-in-line condition, and determine that the character frame to be matched and the character frame that meets the preset same-in-line condition are a group of same-in-line character frames.

[0134] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0135] Figure 4 This is a schematic diagram of the structure of a computer device provided in Example 3 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, the steps of any of the above-mentioned text recognition method embodiments are implemented.

[0136] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 4The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0137] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor, or any conventional processor.

[0138] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of the computer device, and in other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders (BootLoader), data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0139] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0140] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0141] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0143] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0144] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A text recognition method based on artificial intelligence, characterized in that: The text recognition method comprises: Obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row header area of the table image; Performing character recognition in each of the row header areas to obtain at least two column header boxes in the row header area; Determine each title row fold line according to the position of each column title frame in each row title area, and divide the table image into regions according to each title row fold line to obtain at least one table partition; For any table partition, by translating each of the title row fold lines in the table partition in a preset direction, the character boxes identified in the table partition are matched in the same row to determine at least one group of character boxes in the same row; The identification content of the table image is determined based on the column title box and the in-row character box in each of the row title areas.

2. The text recognition method according to claim 1, characterized in that The classification model includes an encoder and a decoder, the training sample is the table image, and the training label is the pixel annotation value in the training sample, wherein the training sample includes an actual row header area and an actual non-row header area, the pixel annotation value of the actual row header area is 1, and the pixel annotation value of the actual non-row header area is 0; The training process of the classification model includes: Inputting the table image into the encoder for feature extraction to obtain table area features; Inputting the table region features into the decoder for upsampling to obtain a region segmentation map, wherein the region segmentation map includes a row header region and a non-row header region, the pixel value of the row header region is 1, and the pixel value of the non-row header region is 0; The loss function is calculated based on the pixel values in the region segmentation map and the pixel annotation values in the training samples, and the parameters of the encoder and the decoder are reversely corrected according to the gradient descent method until the loss function converges to obtain a trained classification model.

3. The text recognition method according to claim 1, characterized in that The step of determining each title row fold line according to the position of each column title frame in each row title area includes: For any row header area, sorting the column header frames according to the center point coordinates of the column header frames in the row header area; According to the sorting result, the center points of the column title boxes are connected in sequence to obtain the corresponding title row broken lines.

4. The text recognition method according to claim 1, characterized in that The step of dividing the table image into regions according to each of the title row fold lines to obtain at least one table partition includes: For any of the title row fold lines, determine the leftmost center point line segment and the rightmost center point line segment that constitute the title row fold line, extend the leftmost center point line segment to intersect with the left edge of the table image, and extend the rightmost center point line segment to intersect with the right edge of the table image, to obtain the extended title row fold line; The area between two adjacent extended title row fold lines is divided into a table partition to obtain at least one table partition in the table image.

5. The text recognition method according to claim 1, characterized in that: For any table partition, performing in-line matching on the identified character boxes in the table partition by translating each of the title row fold lines in the table partition in a preset direction to determine at least one group of in-line character boxes includes: For any table partition, character recognition is performed on the table partition to obtain character frames within the table partition, and character frames to be matched are determined based on the center point coordinates of each character frame; Determining the upper vertex and the lower vertex of each character box according to the vertex coordinates of the four vertices of each character box; Determine two title row fold lines belonging to the same table partition as an upper title row fold line and a lower title row fold line, translate the title row fold lines according to a first preset direction until the upper title row fold line passes through the upper vertex of the to-be-matched character box, and translate the lower title row fold line according to a second preset direction until the lower title row fold line passes through the lower vertex of the to-be-matched character box; Calculate the first distance between each character box and the translated upper title row fold line, and the second distance between each character box and the translated lower title row fold line, and determine at least one group of character boxes in the same row based on the first distance and the second distance.

6. The text recognition method according to claim 5, characterized in that: After determining the at least one group of in-line character frames, the method further includes: For any table partition, filter out the character frames in the same row in each group to determine the character frames to be retained; For the retained character boxes, repeatedly determine the character boxes to be matched based on the center point coordinates of each character box, translate the title row fold line according to the first preset direction until the upper title row fold line passes through the upper vertex of the character box to be matched, translate the lower title row fold line according to the second preset direction until the lower title row fold line passes through the lower vertex of the character box to be matched, calculate the first distance between each character box and the translated upper title row fold line, and the second distance between each character box and the translated lower title row fold line, and determine at least one group of character boxes in the same row based on the first distance and the second distance.

7. The text recognition method according to claim 5, characterized in that: The calculating of a first distance between each character box and the translated upper title line fold line, and a second distance between each character box and the translated lower title line fold line, and determining at least one group of character boxes in the same row according to the first distance and the second distance, comprises: Calculating a first distance between each character box and the translated upper title line fold line, and a second distance between each character box and the translated lower title line fold line; calculating a sum of the first distance and the second distance; The sum of the distances is compared with a preset peer condition, and the character frame to be matched and the character frame satisfying the preset peer condition are determined to be a group of peer character frames.

8. A text recognition device based on artificial intelligence, characterized in that: The text recognition device comprises: A region classification module is used to obtain a table image to be identified, input the table image into a trained classification model, and determine at least one row header region of the table image; a title recognition module, configured to perform character recognition in each row title area to obtain at least two column title boxes in the row title area; an area division module, configured to determine each title row fold line according to the position of each column title frame within each row title area, and divide the table image into areas according to each title row fold line to obtain at least one table partition; A peer matching module is configured to perform peer matching on the character boxes identified in any table partition by translating each of the title row fold lines in the table partition in a preset direction, thereby determining at least one group of peer character boxes. The content recognition module is used to determine the recognition content of the table image based on the column title box and the row character box in each row title area.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the text recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text recognition method according to any one of claims 1 to 7 is implemented.

Images