Text recognition method and device, electronic equipment and storage medium

Abstract

The invention provides a text recognition method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and image processing. According to the specific implementation scheme, the method comprises the steps of determining an outer contour point set of a target text region according to a segmentation result of the bent target text region in a to-be-processed image; for each target point in a plurality of target points determined from the outer contour point set, determining an angular point of the target text area from the plurality of target points according to a position relationship between the target point and a central point of the target text area and a position relationship between adjacent target points; correcting a target text region sub-image defined by a boundary line of the target text region to obtain a corrected target text region sub-image, the boundary line being determined according to the angular point; and identifying a target text from the target text region sub-image.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of deep learning and image processing technology. More specifically, the present disclosure provides a text recognition method, device, electronic device, storage medium and computer program product. Background Art

[0002] Seal text recognition refers to locating the seal area and identifying and extracting the text information in the seal from the document or image. Seals, as a symbol and evidence with legal significance for enterprises, institutions, social groups, government departments and even the country, play an important role in modern social life.

[0003] As one of the downstream tasks of OCR (Optical Character Recognition), there are many practical application scenarios, such as contract information extraction, contract seal review, invoice review, etc. However, it is time-consuming and laborious to extract or collect key information from these document images using manpower. How to automatically adapt to the shapes, text layouts, curved texts and other features of different seals and complete seal text recognition is a problem with both value and challenges. Summary of the invention

[0004] The present disclosure provides a text recognition method, device, electronic device, storage medium and computer program product.

[0005] According to a first aspect, a text recognition method is provided, the method comprising: determining an outer contour point set of a target text area according to a segmentation result of a curved target text area in an image to be processed; for each target point among a plurality of target points determined from the outer contour point set, determining a corner point of the target text area from the plurality of target points according to a positional relationship between the target point and a center point of the target text area and a positional relationship between adjacent target points; correcting a target text area sub-image defined by a boundary line of the target text area to obtain a corrected target text area sub-image, wherein the boundary line is determined according to the corner point; and recognizing a target text from the target text area sub-image.

[0006] According to a second aspect, a text recognition device is provided, which includes: an outer contour point set determination module, which is used to determine the outer contour point set of the target text area according to the segmentation result of the curved target text area in the image to be processed; a corner point determination module, which is used to determine the corner point of the target text area from multiple target points for each target point in the outer contour point set according to the positional relationship between the target point and the center point of the target text area and the positional relationship between adjacent target points; a correction module, which is used to correct a target text area sub-image defined by a boundary line of the target text area to obtain a corrected target text area sub-image, wherein the boundary line is determined according to the corner point; and a text recognition module, which is used to recognize the target text from the target text area sub-image.

[0007] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0008] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0009] According to a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and wherein the computer program implements the method provided according to the present disclosure when executed by a processor.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of an exemplary system architecture to which a text recognition method and apparatus can be applied according to an embodiment of the present disclosure;

[0013] Figure 2 is a flowchart of a text recognition method according to an embodiment of the present disclosure;

[0014] Figure 3 is a system schematic diagram of a text recognition method according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram of a method for correcting curved text according to an embodiment of the present disclosure;

[0016] Figure 5 is a block diagram of a text recognition device according to an embodiment of the present disclosure; and

[0017] Figure 6 The present invention is a block diagram of an electronic device for implementing a text recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] There are few special studies on seal text recognition in the OCR field, and more research is focused on the technology of natural scene text recognition. When the method of natural scene text recognition is transferred to the seal text field, it is not well adapted to the accurate recognition of various forms of seals (such as circles).

[0020] Current seal text recognition methods mainly include two categories: end-to-end text recognition and two-stage text recognition.

[0021] End-to-end text recognition methods, for example, share the detection network structure and the recognition network structure, and obtain the text boundary through curve fitting. However, it is difficult to use a unified method to solve the multi-form seals, and it is also difficult to accurately fit the outer boundary. In addition, since the amount of training data required for detection and recognition is quite different, it is often difficult to obtain good text recognition results by using unified data for end-to-end training, and the recognition accuracy is poor.

[0022] The two-stage text recognition method is a method of serial detection, correction and recognition. The current algorithm design for correcting curved text on seals has poor robustness, the authenticity of the corrected image is poor, and the corrected image cannot be directly recognized using a high-precision text recognition model. A secondary training is required to obtain a dedicated text recognition model.

[0023] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0024] In the technical solution of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0025] Figure 1 is a schematic diagram of an exemplary system architecture to which a text recognition method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0026] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0027] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices, including but not limited to smart phones, tablet computers, laptops, etc.

[0028] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results to the terminal device.

[0029] The text recognition method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the text recognition device provided in the embodiment of the present disclosure can generally be set in the server 105. The text recognition method provided in the embodiment of the present disclosure can also be executed by a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the text recognition device provided in the embodiment of the present disclosure can also be set in a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0030] Figure 2 is a flowchart of a text recognition method according to an embodiment of the present disclosure.

[0031] like Figure 2 As shown, the text recognition method 200 includes operations S210 to S240.

[0032] In operation S210 , an outer contour point set of the target text region is determined according to the segmentation result of the target text region of a curved shape in the image to be processed.

[0033] The image to be processed may be an image containing a seal, and the image to be processed may include multiple text regions. The text regions may include rectangular text regions and curved text regions. The curved text regions may be used as target text regions.

[0034] The text segmentation model can be used to perform text segmentation on the image to be processed to obtain a segmentation result for each text region. The segmentation result can be a binary segmentation result, for example, the pixel value in the region with text is 1, and the pixel value in the region without text is 0.

[0035] For a curved target text region, the outer contour point set of the target text region can be determined according to the binary segmentation result of the target text region. For example, the outer contour points are determined at the boundary with a value of 1.

[0036] In operation S220, for each of the plurality of target points determined from the outer contour point set, a corner point of the target text region is determined from the plurality of target points according to a positional relationship between the target point and a center point of the target text region and a positional relationship between adjacent target points.

[0037] For example, the points in the outer contour point set can be clustered to obtain multiple target points. Among the multiple target points, the target points that meet the conditions can be determined as the corner points of the target text area.

[0038] For example, the distance from a corner point to the center point of the target text area is the largest compared to the distance from a non-corner point to the center point of the target text area. Therefore, whether the target point is a corner point can be determined based on the distance from the target point to the center point.

[0039] In addition, multiple target points are connected to obtain multiple edges. The angle between two adjacent edges of a corner point tends to 90°, and the angle between two adjacent edges of a non-corner point is 180°. Therefore, whether the target point is a corner point can be determined based on the angle between the adjacent edges of the target point.

[0040] The above two conditions can be combined to determine whether the target point is a corner point. For example, the target point with the largest distance to the center point and the angle between two adjacent sides approaching 90° is determined as a corner point. For example, there can be four corner points.

[0041] In operation S230, the target text region sub-image defined by the boundary line of the target text region is corrected to obtain a corrected target text region sub-image, wherein the boundary line is determined according to the corner points.

[0042] After determining each corner point of the target text area, the boundary line of the target text area can be determined based on the corner points. Since the corner points determined based on the distance from the target point to the center point and the positional relationship of adjacent target points are more accurate, the boundary line of the target text area determined based on the corner points is also more accurate.

[0043] After the boundary line is determined, the target text area sub-image surrounded by the boundary line can be determined. The target text area sub-image is corrected to obtain a corrected target text area sub-image. Compared to the correction directly based on the outer contour point or the boundary line fitted by the outer contour point, the embodiment of the present disclosure determines the corner point and the boundary line according to the distance from the target point to the center point and the angle between the adjacent sides of the target point. Since the obtained boundary line is more accurate, the target text area sub-image surrounded by the boundary line is corrected to improve the accuracy and authenticity of the corrected target text area sub-image.

[0044] In operation S240, target text is recognized from the target text region sub-image.

[0045] The corrected target text area sub-image may no longer be a curved text, but a horizontal text. Therefore, the corrected target text area sub-image may be directly input into a text recognition model to obtain the target text. The text recognition model may be a conventional OCR recognition model, without the need for secondary training of a recognition model adapted to curved text.

[0046] The embodiment of the present disclosure determines multiple target points based on the outer contour point set of the target text area of ​​the curved shape, and determines the corner points and boundary lines of the target text area from the multiple target point sets based on the distance from the target point to the center point and the positional relationship of the adjacent target points, which can improve the accuracy of determining the corner points and boundary lines. Correcting the sub-image of the target text area defined by the boundary line can improve the authenticity of the corrected sub-image of the target text area, and then performing text recognition on the corrected sub-image of the target text area can improve the accuracy of text recognition.

[0047] Figure 3 is a system schematic diagram of a text recognition method according to an embodiment of the present disclosure.

[0048] like Figure 3 As shown, this embodiment includes a target detection module 310 , a text segmentation module 320 , a shape determination module 330 , a curved text correction module 340 and a text recognition module 350 .

[0049] The document image 301 may be a contract document image, an official document image, a legal document image, a certificate document image, an invoice document image, etc. The target detection module 310 is used to determine the target object from the document image 301 ; and to intercept the target object from the document image 301 to obtain the image to be processed 311 .

[0050] The target detection module 310 may include a layout analysis positioning model, which can identify the layout structure of the document image 301 and identify various elements such as text, pictures, tables, seals, etc. from the document image 301. The identified seal is used as the target object, and the seal is cut out from the document image 301 to obtain the image to be processed 311.

[0051] The layout analysis and positioning model can be trained using a seal document composite image. Since the number of documents with seals is limited, the embodiment of the present disclosure provides a seal document synthesis tool to generate a seal document composite image. Using the seal document composite image to train the layout analysis and positioning model makes the layout analysis and positioning model not limited by the small amount of training data, and reduces the high cost problem caused by the need for manual annotation of the seal area.

[0052] The above-mentioned seal document synthesis tool can first obtain an existing seal image, then count the colors in the seal image, and determine the color with the highest proportion as the background color. Then, the seal image is cut out based on the background color to obtain the seal part that does not contain the background. Next, the seal part that does not contain the background is synthesized with the document to be synthesized to obtain a seal synthesis document. The seal form can be expanded by random position areas, scaling, blurring and other operations. For example, it can be scaled, blurred, adjusted to grayscale, etc. to obtain seal synthesis documents in various forms. Using a seal document synthesis image containing various forms to train a layout analysis and positioning model can enable the layout analysis and positioning model to recognize seals of various forms from document images.

[0053] The above-mentioned image to be processed 311 can be identified, located and segmented from the document image 301 using the layout analysis and positioning model.

[0054] After obtaining the image to be processed 311 , the text segmentation model may be used to detect, locate and segment the text regions in the image to be processed 311 , and obtain text region segmentation results 321 for each of the multiple text regions.

[0055] According to an embodiment of the present disclosure, multiple text areas in an image to be processed are segmented to obtain a segmentation result and a minimum bounding rectangle of each text area; for each text area, the shape of the text area is determined based on the segmentation result and the minimum bounding rectangle of the text area; and a text area with a curved shape is determined as a target text area.

[0056] The segmentation result of each text region may be a binary segmentation result. The binary segmentation result means that the pixel value of the text region is 1, and the pixel value of the non-text region is 0. The minimum bounding rectangle may be the minimum bounding rectangle of the text region whose pixel value is 1.

[0057] Next, the shape determination module 330 can be used to determine the shape of the segmented text area. The shape of the text area can include a rectangular shape and a curved shape. For each text area, the shape determination module 330 can determine the shape of the text area based on the binary segmentation result of the text area and the minimum circumscribed rectangle.

[0058] For example, the overlap rate between the segmentation result of a rectangular text area and the minimum bounding rectangle is high, while the overlap rate between the segmentation result of a curved text area and the minimum bounding rectangle is low. Therefore, an overlap rate threshold (such as 0.7) can be set to determine text areas with an overlap rate higher than the threshold as rectangular text areas, and text areas with an overlap rate not higher than the threshold as curved text areas.

[0059] For a rectangular text area, a sub-image of the rectangular text area can be directly cut out, and the cut sub-image of the rectangular text area is sent to the text recognition model 350 to obtain a text recognition result of the rectangular text area, that is, a target text 351 .

[0060] For curved text areas, the curved text correction module 340 can be used to correct the curved text area sub-image to obtain a corrected text area sub-image. The text in the corrected text area sub-image is no longer curved but horizontal. Therefore, the corrected text area sub-image can be sent to the text recognition model 350 to obtain the text recognition result of the curved text area, that is, the target text 351.

[0061] The embodiments of the present disclosure extract an image to be processed containing a seal from a document image, segment the image to be processed into multiple text areas, determine the shape of each text area, directly perform text recognition on rectangular text areas, and perform text recognition after correction on curved text areas. This can effectively extract texts of various forms in the seal and improve the accuracy of seal text recognition.

[0062] Combine the following Figure 4 The curved text correction method provided by the embodiment of the present disclosure is described.

[0063] Figure 4 is a schematic diagram of a method for correcting curved text according to an embodiment of the present disclosure.

[0064] like Figure 4As shown, the seal image 410 includes a curved text area and a rectangular text area. Each text area can be segmented by the text segmentation model to obtain a binary segmentation result of each text area. For the curved text area 411, the outer contour point set can be determined according to the binary segmentation result of the area, such as the points around the curved text area 411 in the seal image 410 as the outer contour points. The points in the outer contour point set are relatively dense, and may have the characteristics of being non-smooth and having deviation points.

[0065] According to an embodiment of the present disclosure, points in the outer contour point set are clustered according to the distances between each outer contour point in the outer contour point set to obtain a clustered outer contour point set; and deviating points are removed from the clustered outer contour point set to obtain multiple target points.

[0066] By clustering the points in the above outer contour point concentration, the outer contour points with close distances can be clustered together, and then the clustered points are smoothed and de-noised by removing deviation values, so as to obtain a plurality of relatively smooth target points. For example, the points around the curved text area 421 in the seal image 420 are target points.

[0067] According to an embodiment of the present disclosure, each of a plurality of target points is connected to obtain a plurality of edges; for each target point, an evaluation value of the target point is determined based on the angle between two adjacent edges of the target point and the distance between the target point and the center point of the target text area; and a corner point of the target text area is determined based on the evaluation value.

[0068] As shown in the stamp image 431, each target point can be connected in one direction to obtain multiple directed edges (or vectors). For each target point, the angle between two adjacent vectors of the target point can be calculated. As shown in the stamp image 432, the distance from each target point to the center point can be calculated, and the center point can be the center point of the minimum circumscribed rectangle of the curved text area. For each target point, the evaluation value of the target point can be determined according to the angle between two adjacent vectors of the target point and the distance from the target point to the center point.

[0069] According to an embodiment of the present disclosure, an angle evaluation value of a target point is determined based on the fact that an angle between two adjacent sides of a corner point tends to a first angle and an angle between two adjacent sides of a non-corner point tends to a second angle; a distance evaluation value of the target point is determined based on the fact that a distance from a corner point to a center point is greater than a distance from a non-corner point to a center point; and an evaluation value of the target point is determined based on the angle evaluation value and the distance evaluation value.

[0070] For example, the angle between two adjacent vectors of a corner point tends to be 90 degrees (the first angle), while the angle between two adjacent vectors of a non-corner point tends to be 180 degrees (the second angle). Therefore, this can be used as a basis to determine the angle evaluation value of the target point. In addition, the distance from a corner point to a center point is greater than the distance from a non-corner point to a center point, so this can be used as a basis to determine the distance evaluation value of the target point. Finally, the angle evaluation value and the distance evaluation value can be combined to determine the total evaluation value of the target point.

[0071] For example, for each target point, the angle between the two adjacent vectors of the target point can be inverted. The larger the inverted angle is, the larger the angle evaluation value of the target point is. In addition, the greater the distance between the target point and the center point is, the greater the distance evaluation value is. Finally, the angle evaluation value and the distance evaluation value are weighted to obtain a comprehensive evaluation value.

[0072] The comprehensive evaluation values ​​of multiple target points can form an evaluation value matrix, and the four points with the largest evaluation values ​​can be determined from the evaluation value matrix as the four corner points of the curved text area. The upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point can be determined according to the coordinates of the four corner points.

[0073] Next, the upper boundary line of the curved text area can be determined according to the upper left corner point and the upper right corner point, and the lower boundary line of the curved text area can be determined according to the lower left corner point and the lower right corner point.

[0074] For example, the upper boundary line can be obtained by connecting the upper left corner point, the upper right corner point and the target point between the upper left corner point and the upper right corner point with the upper left corner point and the upper right corner point as control points. The lower boundary line can be obtained by connecting the lower left corner point, the lower right corner point and the target point between the lower left corner point and the lower right corner point with the lower left corner point and the lower right corner point as control points. As shown in the stamp image 440, a smoother boundary line with higher accuracy can be obtained for the curved text area.

[0075] Compared with the method of directly fitting the boundary line according to the outer contour points, the embodiment of the present disclosure determines more accurate corner points, and determines the upper and lower boundary lines with the upper left corner point (lower left corner point) and the upper right corner point (lower right corner point) as control points, making the boundary line more precise. Therefore, it can improve the accuracy of determining the text reading order of the seal and the accuracy of correcting the bent text of the seal.

[0076] Next, the curved text area sub-image may be corrected according to the boundary line of the curved text area.

[0077] According to an embodiment of the present disclosure, multiple sampling points of the upper boundary line and multiple sampling points of the lower boundary line are determined respectively; and based on the multiple sampling points of the upper boundary line and the multiple sampling points of the lower boundary line, a target text area sub-image defined by the boundary line is divided into multiple image blocks; and the multiple corrected image blocks are spliced ​​to obtain a corrected target text area sub-image.

[0078] For example, the upper boundary line and the lower boundary line may be densely sampled to obtain multiple sampling points on the upper boundary line and multiple sampling points on the lower boundary line. For example, the points on the upper boundary and the lower boundary of the curved text area in the seal image 450 are sampling points. In addition, instead of sampling based on the upper and lower boundary lines, dense sampling may be performed on the target points on the upper boundary line and the target points on the lower boundary to obtain multiple sampling points on the upper boundary line and multiple sampling points on the lower boundary line.

[0079] The multiple sampling points on the upper boundary line and the multiple sampling points on the lower boundary line can be grouped, and the points in each group are interpolated so that the number of points on the upper boundary line and the number of points on the lower boundary in each group are consistent. Then, two points on the upper boundary line and two points on the lower boundary can be selected in sequence to form a quadrilateral, and multiple quadrilaterals can be formed, each of which is an image block. Each image block can be corrected separately, and then the multiple corrected image blocks are spliced ​​to obtain the corrected target text area sub-image 460. It should be noted that each image block in the corrected target text area sub-image 460 contains a character only as an example, and the actual segmentation may not be performed according to the characters.

[0080] The text in the corrected target text region sub-image 460 is no longer curved but horizontal, so text recognition can be performed directly to obtain the target text 470 .

[0081] The embodiment of the present disclosure determines sampling points on the upper and lower boundary lines, divides the curved text area into multiple image blocks based on the sampling points, corrects and splices each image block to obtain a corrected target text area sub-image, which can improve the authenticity of the corrected target text area sub-image, thereby improving the text recognition effect of the corrected target text area sub-image and improving the text recognition accuracy.

[0082] According to an embodiment of the present disclosure, the present disclosure also provides a text recognition device.

[0083] Figure 5 is a block diagram of a text recognition device according to an embodiment of the present disclosure.

[0084] like Figure 5As shown, the text recognition device 500 includes an outer contour point set determination module 510 , a corner point determination module 520 , a correction module 530 and a text recognition module 540 .

[0085] The outer contour point set determination module 510 is used to determine the outer contour point set of the target text region according to the segmentation result of the curved target text region in the image to be processed.

[0086] The corner point determination module 520 is used to determine the corner point of the target text area from the multiple target points for each target point determined from the outer contour point set according to the positional relationship between the target point and the center point of the target text area and the positional relationship between adjacent target points.

[0087] The correction module 530 is used to correct the target text region sub-image defined by the boundary line of the target text region to obtain a corrected target text region sub-image, wherein the boundary line is determined according to the corner points.

[0088] The text recognition module 540 is used to recognize the target text from the target text region sub-image.

[0089] The corner point determination module 520 includes a connection unit, an evaluation unit and a corner point determination unit.

[0090] The connection unit is used to connect each target point among the multiple target points to obtain multiple edges.

[0091] The evaluation unit is used to determine, for each target point, an evaluation value of the target point according to the angle between two adjacent sides of the target point and the distance between the target point and the center point of the target text area.

[0092] The corner point determination unit is used to determine the corner points of the target text area according to the evaluation value.

[0093] The evaluation unit includes an angle and distance determination subunit, an angle evaluation subunit, a distance evaluation subunit and an evaluation value determination subunit.

[0094] The angle and distance determination subunit is used to determine the angle between two adjacent sides of the target point and the distance between the target point and the center point of the target text area.

[0095] The angle evaluation subunit is used to determine the angle evaluation value of the target point based on the fact that the angle between two adjacent sides of the corner point tends to a first angle and the angle between two adjacent sides of the non-corner point tends to a second angle.

[0096] The distance evaluation subunit is used to determine the distance evaluation value of the target point based on the fact that the distance from the corner point to the center point is greater than the distance from the non-corner point to the center point.

[0097] The evaluation value determination subunit is used to determine the evaluation value of the target point according to the angle evaluation value and the distance evaluation value.

[0098] The correction module 530 includes a segmentation unit and a correction unit.

[0099] The segmentation unit is used to segment the target text area sub-image defined by the boundary line to obtain a plurality of image blocks.

[0100] The correction unit is used to splice multiple corrected image blocks to obtain a corrected target text area sub-image.

[0101] The boundary line includes an upper boundary line and a lower boundary line. The segmentation unit includes a sampling point determination subunit and a segmentation subunit.

[0102] The sampling point determination subunit is used to determine a plurality of sampling points of the upper boundary line and a plurality of sampling points of the lower boundary line respectively.

[0103] The segmentation subunit is used to segment the target text area sub-image defined by the boundary line into a plurality of image blocks according to a plurality of sampling points of the upper boundary line and a plurality of sampling points of the lower boundary line.

[0104] The text recognition device 500 further includes a clustering module and a removal module.

[0105] The clustering module is used to cluster the points in the outer contour point set according to the distances between the outer contour points in the outer contour point set to obtain a clustered outer contour point set.

[0106] The removal module is used to remove deviating points from the clustered outer contour point set to obtain multiple target points.

[0107] The text recognition device 500 further includes a segmentation module, a shape determination module and a target text area determination module.

[0108] The segmentation module is used to segment multiple text regions in the processed image and obtain the segmentation result and the minimum bounding rectangle of each text region.

[0109] The shape determination module is used to determine the shape of each text region according to the segmentation result and the minimum bounding rectangle of the text region.

[0110] The target text region determining module is used to determine a text region in a curved shape as a target text region.

[0111] The text recognition device 500 further includes a target object determination module and a to-be-processed image determination module.

[0112] The target object determination module is used to determine the target object from the document image.

[0113] The module for determining the image to be processed is used to capture the target object from the document image to obtain the image to be processed.

[0114] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0115] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0116] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0117] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the text recognition method. For example, in some embodiments, the text recognition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the text recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the text recognition method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A text recognition method, comprising: Determine an outer contour point set of the target text area according to a segmentation result of a curved target text area in the image to be processed; For each of the plurality of target points determined from the outer contour point set, determining a corner point of the target text area from the plurality of target points according to a positional relationship between the target point and a center point of the target text area and a positional relationship between adjacent target points; Correcting a target text region sub-image defined by a boundary line of the target text region to obtain a corrected target text region sub-image, wherein the boundary line is determined based on the corner point; and The target text is recognized from the target text region sub-image.

2. The method according to claim 1, wherein: The step of determining, for each of the plurality of target points determined from the outer contour point set, a corner point of the target text area from the plurality of target points according to a positional relationship between the target point and a center point of the target text area and a positional relationship between adjacent target points comprises: Connect each of the plurality of target points to obtain a plurality of edges; For each target point, determining an evaluation value of the target point according to an angle between two adjacent sides of the target point and a distance between the target point and a center point of the target text area; and The corner points of the target text area are determined according to the evaluation values.

3. The method according to claim 2, wherein: For each target point, determining the evaluation value of the target point according to the angle between two adjacent sides of the target point and the distance between the target point and the center point of the target text area includes: for each target point, Determine the angle between two adjacent sides of the target point and the distance between the target point and the center point of the target text area; Determine the angle evaluation value of the target point based on the fact that the angle between two adjacent sides of the corner point tends to a first angle and the angle between two adjacent sides of the non-corner point tends to a second angle; Determining a distance evaluation value of the target point based on the distance from the corner point to the center point being greater than the distance from the non-corner point to the center point; and An evaluation value of the target point is determined according to the angle evaluation value and the distance evaluation value.

4. The method according to claim 1, wherein: The correcting the target text region sub-image defined by the boundary line of the target text region to obtain the corrected target text region sub-image comprises: Segmenting the target text area sub-image defined by the boundary line to obtain a plurality of image blocks; and The multiple corrected image blocks are spliced ​​together to obtain the corrected target text area sub-image.

5. The method according to claim 4, wherein: The boundary line includes an upper boundary line and a lower boundary line; and the segmenting of the target text area sub-image defined by the boundary line to obtain a plurality of image blocks includes: respectively determining a plurality of sampling points of the upper boundary line and a plurality of sampling points of the lower boundary line; and According to the multiple sampling points of the upper boundary line and the multiple sampling points of the lower boundary line, the target text area sub-image defined by the boundary line is divided into multiple image blocks.

6. The method according to claim 1, further comprising: Clustering the points in the outer contour point set according to the distances between the outer contour points in the outer contour point set to obtain a clustered outer contour point set; as well as The deviating points are removed from the clustered outer contour point set to obtain the multiple target points.

7. The method according to claim 1, further comprising: Segmenting multiple text regions in the image to be processed to obtain a segmentation result and a minimum bounding rectangle of each text region; For each text region, determining the shape of the text region according to the segmentation result and the minimum bounding rectangle of the text region; as well as A text area in a curved shape is determined as the target text area.

8. The method according to claim 1, further comprising: determining a target object from a document image; as well as The target object is intercepted from the document image to obtain the image to be processed.

9. A text recognition device, comprising: An outer contour point set determination module is used to determine an outer contour point set of the target text area according to the segmentation result of the curved target text area in the image to be processed; A corner point determination module, for determining, for each of the plurality of target points determined from the outer contour point set, a corner point of the target text area from the plurality of target points according to a positional relationship between the target point and a center point of the target text area and a positional relationship between adjacent target points; a correction module, configured to correct a target text region sub-image defined by a boundary line of the target text region to obtain a corrected target text region sub-image, wherein the boundary line is determined based on the corner point; and The text recognition module is used to recognize the target text from the target text area sub-image.

10. The device according to claim 9, wherein: The corner point determination module comprises: A connection unit, used to connect each of the plurality of target points to obtain a plurality of edges; an evaluation unit, configured to determine, for each target point, an evaluation value of the target point according to an angle between two adjacent sides of the target point and a distance between the target point and a center point of the target text area; and A corner point determination unit is used to determine the corner points of the target text area according to the evaluation value.

11. The device according to claim 10, wherein: The evaluation unit comprises: An angle and distance determination subunit, used to determine the angle between two adjacent sides of the target point and the distance between the target point and the center point of the target text area; An angle evaluation subunit, used for determining an angle evaluation value of the target point based on the fact that the angle between two adjacent sides of the corner point tends to a first angle and the angle between two adjacent sides of the non-corner point tends to a second angle; A distance evaluation subunit is used to determine a distance evaluation value of the target point based on the fact that the distance from the corner point to the center point is greater than the distance from the non-corner point to the center point; and The evaluation value determination subunit is used to determine the evaluation value of the target point according to the angle evaluation value and the distance evaluation value.

12. The device according to claim 9, wherein: The correction module comprises: a segmentation unit, configured to segment the target text region sub-image defined by the boundary line to obtain a plurality of image blocks; and The correction unit is used to splice a plurality of corrected image blocks to obtain the corrected target text area sub-image.

13. The device according to claim 12, wherein: The boundary line includes an upper boundary line and a lower boundary line; the segmentation unit includes: a sampling point determination subunit, configured to respectively determine a plurality of sampling points of the upper boundary line and a plurality of sampling points of the lower boundary line; and The segmentation subunit is used to segment the target text area sub-image defined by the boundary line into a plurality of image blocks according to the plurality of sampling points of the upper boundary line and the plurality of sampling points of the lower boundary line.

14. The apparatus according to claim 9, further comprising: A clustering module, used for clustering the points in the outer contour point set according to the distances between the outer contour points in the outer contour point set to obtain a clustered outer contour point set; as well as The removal module is used to remove the deviating points from the clustered outer contour point set to obtain the multiple target points.

15. The apparatus according to claim 9, further comprising: A segmentation module, used for segmenting multiple text regions in the image to be processed to obtain a segmentation result and a minimum bounding rectangle of each text region; A shape determination module, used for determining the shape of each text region according to the segmentation result and the minimum bounding rectangle of the text region; as well as The target text region determining module is used to determine the text region in a curved shape as the target text region.

16. The apparatus according to claim 9, further comprising: A target object determination module, used for determining a target object from a document image; as well as The module for determining the image to be processed is used to capture the target object from the document image to obtain the image to be processed.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images