A text recognition method, a text recognition device and a scanning pen

Abstract

The present application discloses a text recognition method, a text recognition device, a scanning pen and a computer-readable storage medium. The method includes: performing text detection on a text image to be recognized to obtain a text box; calculating the distance between the text box and the boundary of the text image to be recognized; if the distance is less than a preset distance threshold, the text content in the text box is recognized by a text recognition algorithm based on connection time classification CTC to obtain a CTC recognition result; based on the CTC recognition result, it is determined whether there is a segmentation requirement; if there is a segmentation requirement, the CTC recognition result is segmented, and a text recognition result is obtained based on the segmented CTC recognition result. Through the present application scheme, the accuracy of text recognition can be guaranteed.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a text recognition method, a text recognition device, a scanning pen and a computer-readable storage medium. Background Art

[0002] The appearance of the scanning pen is similar to a pen. It is a handheld device that allows users to scan and input text by holding the scanning pen. The scanning pen can display, interpret the meaning of words, translate, and voice broadcast the scanned text. When scanning and inputting, the scanning pen usually splices the scanned image frames into a temporary image in real time, and then sends the spliced ​​temporary image to the optical character recognition (OCR) module for character recognition, and displays the recognition results on the screen in real time.

[0003] Since the timing of OCR recognition is random, at a certain moment, the last character at the end of the spliced ​​temporary image may be incomplete, that is, character truncation occurs, which may affect the accuracy of text recognition. Summary of the invention

[0004] The present application provides a text recognition method, a text recognition device, a scanning pen and a computer-readable storage medium, which can ensure the accuracy of text recognition.

[0005] In a first aspect, the present application provides a text recognition method, comprising:

[0006] Perform text detection on the text image to be recognized and obtain the text box;

[0007] Calculating the distance between the text box and the boundary of the text image to be recognized;

[0008] If the distance is less than the preset distance threshold, the text content in the text box is recognized by a text recognition algorithm based on connectionist temporal classification (CTC) to obtain a CTC recognition result;

[0009] Based on the above CTC recognition results, determine whether there is a need for segmentation;

[0010] If there is a segmentation requirement, the CTC recognition result is segmented, and a text recognition result is obtained based on the segmented CTC recognition result.

[0011] In a second aspect, the present application provides a text recognition device, comprising:

[0012] A detection module, used to perform text detection on the text image to be recognized and obtain a text box;

[0013] A calculation module, used for calculating the distance between the text box and the boundary of the text image to be recognized;

[0014] A recognition module, configured to recognize the text content in the text box by using a CTC-based text recognition algorithm to obtain a CTC recognition result if the distance is less than a preset distance threshold;

[0015] A judgment module, used to judge whether there is a segmentation requirement based on the above CTC recognition result;

[0016] The segmentation module is used to segment the CTC recognition result if there is a segmentation requirement, and obtain a text recognition result based on the segmented CTC recognition result.

[0017] In a third aspect, the present application provides a scanning pen, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.

[0019] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it implements the steps of the method of the first aspect.

[0020] Compared with the prior art, the present application has the following beneficial effects: the distance between the text box and the boundary of the text image to be recognized is used to preliminarily determine whether there is a possibility of character truncation. When it is found that the distance is less than the preset distance threshold, it is considered that the possibility of character truncation is relatively high. At this time, the text content in the text box is recognized by a CTC-based text recognition algorithm, and the intermediate result output by the text recognition algorithm, that is, the CTC recognition result, is used as the object of analysis to further analyze whether there is a need for segmentation. When there is a need for segmentation, the CTC recognition result is segmented, and an accurate text recognition result is obtained based on the segmented CTC recognition result. The above process can avoid the influence of truncated characters on the text recognition results, and ensure the accuracy of single text recognition.

[0021] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a schematic diagram of the implementation process of the text recognition method provided in the embodiment of the present application;

[0024] Figure 2 is an example diagram of a text box provided in an embodiment of the present application;

[0025] Figure 3 is an example diagram of the distance between the text box and the boundary of the text image to be recognized provided in an embodiment of the present application;

[0026] Figure 4 is an example diagram of the intermediate segmentation position and the target segmentation position provided in the embodiment of the present application;

[0027] Figure 5 is an example diagram of an application scenario of the text recognition method provided in an embodiment of the present application;

[0028] Figure 6 is a structural block diagram of a text recognition device provided in an embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the structure of the scanning pen provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0031] In order to illustrate the technical solution proposed in this application, a specific embodiment is provided below for illustration.

[0032] The following is an explanation of the text recognition method proposed in the embodiment of the present application. Figure 1 , the implementation process of the text recognition method is described as follows:

[0033] Step 101: perform text detection on the text image to be recognized to obtain a text box.

[0034] In the embodiment of the present application, after the scanning pen obtains the text image to be recognized through the scanning picture captured in real time by the camera, it can detect whether there is text content in the text image to be recognized through a preset text detection algorithm. As an example only, the text detection algorithm can be a connectionist text proposal network (CTPN) algorithm, and of course it can also be other types of algorithms, which are not limited here.

[0035] When the text detection algorithm detects text content, the text detection algorithm may output a corresponding text box, including the coordinates of each vertex of the text box in the text image to be recognized. It can be understood that the text box actually selects the text content contained in the text image to be recognized. As an example only, the text box may specifically be: the minimum enclosing rectangular box of the text content contained in the text image to be recognized.

[0036] For easier understanding, see Figure 2 . Figure 2 An example of a text box obtained after text detection on a text image to be recognized is given.

[0037] Step 102: Calculate the distance between the text box and the boundary of the text image to be recognized.

[0038] In the embodiment of the present application, considering that in real life, the characters in the text are usually arranged horizontally from left to right; when the user uses a scanning pen to scan, the text is usually scanned horizontally from left to right. Therefore, based on common scanning operations, the truncation of the text image to be recognized usually refers to the truncation of the rightmost character of the text by the right boundary of the text image to be recognized, resulting in incomplete display of the rightmost character. Based on this, the boundary of the text image to be recognized proposed in this step usually refers to the right boundary of the text image to be recognized. That is, after the scanning pen obtains the text box through text detection, it can calculate the distance between the text box and the right boundary of the text image to be recognized.

[0039] Step 103: If the distance is less than a preset distance threshold, the text content in the text box is recognized by a CTC-based text recognition algorithm to obtain a CTC recognition result.

[0040] In an embodiment of the present application, when the rightmost character of the text in the text image to be recognized is truncated by the right border, the text box will be closer to the right border. Based on this, a distance threshold is pre-set in the scanning pen. When the distance calculated in step 102 is less than the distance threshold, it is considered that there is a certain possibility that the text is truncated. At this time, the scanning pen will recognize the text content in the text box through a CTC-based text recognition algorithm. Among them, CTC is a technology used to solve the problem that it is difficult to correspond one-to-one between input sequences and output sequences, and is commonly used in speech recognition and OCR.

[0041] For the CTC-based text recognition algorithm, the CTC recognition result is the intermediate output of the text recognition method, not the final output. The CTC recognition result is specifically manifested as a sequence of multiple output values.

[0042] Step 104: Based on the CTC recognition result, determine whether there is a segmentation requirement.

[0043] In the embodiment of the present application, the scanning pen can analyze the text content selected by the text box based on the CTC recognition result to further determine whether the rightmost character of the text is truncated. If the rightmost character of the text is truncated based on the CTC recognition result, there is a need for segmentation; conversely, if the rightmost character of the text is not truncated based on the CTC recognition result, there is no need for segmentation.

[0044] Step 105: If there is a segmentation requirement, the CTC recognition result is segmented, and a text recognition result is obtained based on the segmented CTC recognition result.

[0045] In the embodiment of the present application, if it is known through step 104 that there is a need for segmentation, the scanning pen can first perform segmentation processing on the CTC recognition result, specifically: in the CTC recognition result, the output value that may have recognition errors caused by the truncation of the text is deleted. Since the output value that may have recognition errors has been deleted from the CTC recognition result, the scanning pen can obtain accurate text recognition results based on the segmented CTC recognition result, avoiding the interference of the truncated rightmost character on the text recognition result.

[0046] It can be understood that if the distance calculated by step 102 is not less than the distance threshold, it is considered that there is no possibility of character truncation. At this time, the scanning pen can obtain accurate text recognition results by performing text recognition on the text image selected by the text box normally, and there is no need to analyze the CTC recognition results. Similarly, if it is determined by step 104 that there is no need to segment, there is no need to make any adjustments to the CTC recognition results, and accurate text recognition results can be obtained directly based on the unadjusted CTC recognition results.

[0047] In some embodiments, the user may move the scanning pen in an irregular manner when scanning, which may cause the text lines contained in the text image to be recognized to be in a non-horizontal state; accordingly, the text box obtained in step 101 will also be in a non-horizontal state in the text image to be recognized. Based on this, step 102 may specifically include:

[0048] A1. Determine the maximum horizontal coordinate value of the vertex.

[0049] The maximum horizontal coordinate value of the vertex refers to the maximum value of the horizontal coordinate values ​​of the vertices of the text box. As an example, the text box is usually a rectangle with four vertices, which can be recorded as V1, V2, V3 and V4. Assuming that the coordinates of V1 are (x1, y1), the coordinates of V2 are (x2, y2), the coordinates of V3 are (x3, y3), and the coordinates of V4 are (x4, y4), the scanning pen can compare x1, x2, x3 and x4 to determine the maximum value. This maximum value is the maximum horizontal coordinate value of the vertex.

[0050] It is understandable that, unless otherwise specified, all concepts related to coordinates proposed in the embodiments of the present application refer to: coordinates in the pixel coordinate system of the text image to be recognized.

[0051] A2. Determine the maximum horizontal coordinate value of the image.

[0052] The maximum horizontal coordinate value of the image refers to the maximum value of the horizontal coordinate values ​​of each pixel point of the text image to be recognized. Considering that the text image to be recognized is generally a regular rectangle, and the origin of the pixel coordinate system is the upper left vertex of the text image to be recognized, the maximum horizontal coordinate value of the image is usually the horizontal coordinate value of the right border of the text image to be recognized.

[0053] A3. Determine the difference between the maximum horizontal coordinate value of the image and the maximum horizontal coordinate value of the vertex as the distance between the text box and the boundary of the text image to be recognized.

[0054] The scanning pen can use the difference between the maximum horizontal coordinate value of the image and the maximum horizontal coordinate value of the vertex as the distance between the text box and the boundary of the text image to be recognized. It can be understood that the distance is in pixels.

[0055] For easier understanding, see Figure 3 . Figure 3 Examples of the distances between a horizontal text box and a non-horizontal text box and the border of the text image to be recognized are given. Where d1 is the distance between the horizontal text box and the border of the text image to be recognized. d2 is the distance between the non-horizontal text box and the border of the text image to be recognized.

[0056] In some embodiments, in order to accurately determine whether segmentation is required when the distance between the text box and the boundary of the to-be-recognized text image is less than a distance threshold, step 104 may include:

[0057] B1. Check whether there is a valid output value among the last N output values ​​in the CTC recognition result.

[0058] CTC can be understood as dividing the text image selected by the text box into several segments of equal length, and each segment will have an output value. The output value may be a valid character or it may be empty. As an example only, when the output value is empty, the output value is a given character Φ; of course, other characters can also be defined to indicate that the output value is empty, which is not limited here. It can be understood that when the output value is a valid character, the output value is a valid output value. Let the width of the text image selected by the text box be w, and the number of segments divided be num, then the length of each segment w1=w / num. That is, the CTC recognition result is: a sequence consisting of num output values.

[0059] As mentioned above, truncation usually occurs at the end; that is, the last character in the text box is usually truncated. Based on this, the scanning pen pre-sets a value N and only traverses the last N output values ​​of the CTC recognition result. This can effectively improve the efficiency of truncation detection while ensuring the accuracy of truncation detection. Among them, N is a positive integer, and N<num.

[0060] B2. If there is a valid output value among the last N output values, it is determined that there is a need for segmentation.

[0061] When there is a valid output value among the last N output values, it is considered that the last character is very likely to be truncated, and it is determined that there is a need for segmentation. When the CTC recognition result is subsequently segmented, the last valid output value in the CTC recognition result can be directly deleted, so that the final text recognition result is not affected by the character that may be truncated.

[0062] In some embodiments, the scanning pen is continuously stitching images during the scanning and input process. When performing text recognition, part of the stitched image may have been recognized (that is, the first scanned part has been recognized), and part of the image has not been recognized (that is, the later scanned part has not been recognized). In order to ensure the efficiency of text recognition, the scanning pen will not repeat the recognition of the recognized part. Since the scanning pen does not actually recognize the characters that may be truncated in the text image to be recognized through steps 101-105, in order to ensure that the truncated characters appear completely in the next text image to be recognized, after step 105, the text recognition method also includes:

[0063] C1. Determine the target segmentation position in the text image to be recognized.

[0064] In the embodiment of the present application, the scanning pen can determine a segmentation position in the text image to be recognized, which is recorded as the target segmentation position. The target segmentation position can be determined by analyzing the CTC recognition result, which specifically includes the following process:

[0065] C11. Determine a target valid output value among the valid output values ​​of the last N output values ​​in the CTC recognition result.

[0066] It can be understood that if there is only a single valid output value among the last N output values, then the single valid output value can be determined as the target valid output value; if there are more than two valid output values ​​among the last N output values, then considering that truncation usually occurs in the last character in the text box, the scanning pen will determine the last valid output value sorted among the more than two valid output values ​​as the target valid output value.

[0067] C12. Based on the target effective output value, determine the target segmentation position in the text image to be recognized.

[0068] It can be understood that since the CTC recognition result is a sequence of num output values, the output values ​​can be numbered in order starting from 1; that is, each output value has a unique index number in the CTC recognition result, and the value range of the index number is: 1, 2, 3, ..., num. Since the CTC recognition result is obtained by the scanning pen performing text recognition on the text image selected by the text box, the scanning pen can determine the middle cutting position in the text box based on the target index number, wherein the target index number refers to: the index number of the target valid output value in the CTC recognition result. For ease of explanation, the target index number can be denoted as i. Afterwards, the scanning pen can determine the target cutting position based on the middle cutting position and the starting position of the text box in the text image to be recognized. That is, the middle cutting position in the text box needs to be mapped to the text image to be recognized in order to obtain the target cutting position in the text image to be recognized.

[0069] When the text box is in the horizontal state: let the starting position of the text box be j, then the target segmentation position can be expressed as: i*w1+j or (i-1)*w1+j; that is, the horizontal coordinate of the left border of the text box is j, and the target segmentation position is the line segment represented by the horizontal coordinate i*w1+j, or the line segment represented by the horizontal coordinate (i-1)*w1+j.

[0070] See also Figure 4 , Figure 4 Examples of intermediate segmentation positions and target segmentation positions are given.

[0071] C2. Obtain a new text image to be recognized, and return to execute step 101 and subsequent steps based on the new text image to be recognized.

[0072] In the stitched image obtained by the scanning pen at the next moment, there are both recognized image regions and unrecognized image regions, and the two are specifically divided by the target segmentation position determined in the most recent time. Based on this, the scanning pen can obtain a new text image to be recognized through the stitched image, where the starting position of the new text image to be recognized is determined based on the target segmentation position. That is, the target segmentation position can be mapped to the currently obtained stitched image to obtain a new text image to be recognized. Obviously, through the above steps C1 and C2, it can be ensured that the truncated characters can appear completely in the next text image to be recognized.

[0073] For the convenience of understanding the above steps 101-105 and steps C1-C2, please refer to Figure 5 , Figure 5 A specific scenario example is given:

[0074] The scanning pen starts this scan at time T0 and performs image stitching for the first time at time T1, obtaining the stitched image I1 at time T1. Since during this scan process, before obtaining the stitched image I1, text recognition has not been performed yet, therefore, the stitched image I1 is the first text image to be recognized. The process of executing each step based on the first text image to be recognized is specifically as follows: Through step 102, it is determined that the distance is less than the distance threshold; through step 103, the CTC recognition result of the first text image to be recognized is obtained as "ΦLeiΦΦΦΦΦΦΦFengΦΦΦΦΦΦCunΦΦΦ"; assuming N is 5, then through step 104, it can be known that there is a segmentation requirement; through steps C1 and C2, the target segmentation position P1 in the stitched image I1 is determined. That is, in the stitched image I1, the image region before the target segmentation position P1 has been recognized.

[0075] After that, at time T2, the scanning pen performed image stitching for the second time and obtained the stitched image I2 at time T2. Obviously, the stitched image I2 is based on the stitched image I1 with new images stitched on it. That is to say, the stitched image I1 is a part of the stitched image I2. Since during this scanning process, before obtaining this stitched image I2, text recognition has been performed, and the previous text recognition output the target segmentation position P1, therefore, the scanning pen can find the target segmentation position P1 in this stitched image I2. Obviously, the image before (i.e., to the left of) the target segmentation position P1 has been recognized, and the image after (i.e., to the right of) the target segmentation position P1 has not been recognized. The scanning pen can then segment the stitched image I2 based on the target segmentation position P1 to obtain the text image to be recognized for the second time. Obviously, the starting position of this text image to be recognized for the second time is the target segmentation position P1 output based on the text image to be recognized for the first time (i.e., the previous one). The process of performing each step based on this text image to be recognized for the second time is specifically as follows: Through step 102, it is determined that the distance is less than the distance threshold; through step 103, the CTC recognition result of the text image to be recognized for the second time is obtained as "Φ uncle ΦΦΦΦΦ uncle ΦΦΦΦΦΦΦΦ, ΦΦΦΦΦΦ you ΦΦΦΦΦΦΦ are ΦΦΦ"; through step 104, it is known that there is a segmentation requirement; through steps C1 and C2, the target segmentation position P2 in this text image to be recognized for the second time is determined. That is to say, in the stitched image 2, the image area before the target segmentation position P2 has been recognized.

[0076] Next, at time T3, the scanning pen performed image stitching for the third time and obtained the stitched image I3 at time T3. Obviously, the stitched image I3 is based on the stitched image I2 with new images stitched on it. That is to say, the stitched image I2 is a part of the stitched image I3. Since during this scanning process, before obtaining this stitched image I3, text recognition has been performed, and the previous text recognition output the target segmentation position P2, therefore, the scanning pen can find the target segmentation position P2 in this stitched image I3. Obviously, the image before (i.e., to the left of) the target segmentation position P2 has been recognized, and the image after (i.e., to the right of) the target segmentation position P2 has not been recognized. The scanning pen can then segment the stitched image I3 based on the target segmentation position P2 to obtain the text image to be recognized for the third time. Obviously, the starting position of this text image to be recognized for the third time is the target segmentation position P2 output based on the text image to be recognized for the second time (i.e., the previous one). The process of performing each step based on this text image to be recognized for the third time is specifically as follows: Through step 102, it is determined that the distance is greater than the distance threshold, indicating that the possibility of character truncation is very small. At this time, no segmentation processing is required, and the text recognition result of this text image to be recognized for the third time can be directly output based on the CTC text recognition algorithm.

[0077] As can be seen from the above, through the embodiment of the present application, the distance between the text box and the boundary of the text image to be recognized is used to preliminarily determine whether there is a possibility of character truncation. When it is found that the distance is less than the preset distance threshold, it is considered that the possibility of character truncation is relatively high. At this time, the text content in the text box is recognized by the CTC-based text recognition algorithm, and the intermediate result output by the text recognition algorithm, that is, the CTC recognition result, is used as the object of analysis to further analyze whether there is a segmentation requirement. When there is a segmentation requirement, the CTC recognition result is segmented, and an accurate text recognition result is obtained based on the segmented CTC recognition result. The above process can avoid the influence of truncated characters on the text recognition results, and ensure the accuracy of single text recognition.

[0078] Corresponding to the text recognition method provided above, the present application embodiment also provides a text recognition device. Figure 6 As shown, the text recognition device 600 includes:

[0079] A detection module 601 is used to perform text detection on the text image to be recognized and obtain a text frame;

[0080] A calculation module 602 is used to calculate the distance between the text box and the boundary of the text image to be recognized;

[0081] The recognition module 603 is used to recognize the text content in the text box by using a text recognition algorithm based on connection time classification CTC to obtain a CTC recognition result if the distance is less than a preset distance threshold;

[0082] A determination module 604 is used to determine whether there is a segmentation requirement based on the above CTC recognition result;

[0083] The segmentation module 605 is used to segment the CTC recognition result if there is a segmentation requirement, and obtain a text recognition result based on the segmented CTC recognition result.

[0084] Optionally, the calculation module 602 includes:

[0085] A first coordinate value determining unit is used to determine a maximum horizontal coordinate value of a vertex, where the maximum horizontal coordinate value of a vertex is: a maximum value among the horizontal coordinate values ​​of the vertices of the text box;

[0086] The second coordinate value determining unit is used to determine the maximum horizontal coordinate value of the image, where the maximum horizontal coordinate value of the image is the maximum value among the horizontal coordinate values ​​of the pixels of the text image to be recognized;

[0087] The distance determination unit is used to determine the difference between the maximum horizontal coordinate value of the image and the maximum horizontal coordinate value of the vertex as the distance.

[0088] Optionally, the above-mentioned determination module 604 includes:

[0089] A detection unit, used to detect whether there is a valid output value among the last N output values ​​in the CTC recognition result, wherein N is a preset positive integer and N is less than the total number of output values ​​included in the CTC recognition result;

[0090] The determining unit is used to determine whether there is a segmentation requirement if there is a valid output value among the last N output values.

[0091] Optionally, the text recognition device 600 further includes:

[0092] A position determination module, used to determine the target segmentation position in the above-mentioned text image to be recognized;

[0093] The image acquisition module is used to acquire a new text image to be recognized, and trigger the operation of the detection module 601 based on the new text image to be recognized, wherein the starting position of the new text image to be recognized is determined based on the target segmentation position.

[0094] Optionally, the above-mentioned location determination module includes:

[0095] A target effective output value determination unit, used to determine a target effective output value among the effective output values ​​existing in the last N output values;

[0096] The target segmentation position determination unit is used to determine the target segmentation position in the above-mentioned text image to be recognized based on the above-mentioned target effective output value.

[0097] Optionally, the target effective output value determination unit is specifically used to determine the only valid output value as the target effective output value if there is a unique valid output value among the last N output values; if there are more than two valid output values ​​among the last N output values, determine the last sorted valid output value among the more than two valid output values ​​as the target effective output value.

[0098] Optionally, the target segmentation position determination unit includes:

[0099] The middle segmentation position determination subunit is used to determine the middle segmentation position in the text box based on the target index number, wherein the target index number is: the index number of the target valid output value in the CTC recognition result;

[0100] The target segmentation position determination subunit is used to determine the target segmentation position according to the intermediate segmentation position and the starting position of the text box in the to-be-recognized text image.

[0101] As can be seen from the above, through the embodiment of the present application, the distance between the text box and the boundary of the text image to be recognized is used to preliminarily determine whether there is a possibility of character truncation. When it is found that the distance is less than the preset distance threshold, it is considered that the possibility of character truncation is relatively high. At this time, the text content in the text box is recognized by the CTC-based text recognition algorithm, and the intermediate result output by the text recognition algorithm, that is, the CTC recognition result, is used as the object of analysis to further analyze whether there is a segmentation requirement. When there is a segmentation requirement, the CTC recognition result is segmented, and an accurate text recognition result is obtained based on the segmented CTC recognition result. The above process can avoid the influence of truncated characters on the text recognition results, and ensure the accuracy of single text recognition.

[0102] Corresponding to the text recognition method provided above, the present application embodiment also provides a scanning pen. Figure 7 The scanning pen 7 in the embodiment of the present application includes: a memory 701, one or more processors 702 ( Figure 7 Only one is shown) and a computer program stored in the memory 701 and executable on the processor. The memory 701 is used to store software programs and units, and the processor 702 executes various functional applications and diagnoses by running the software programs and units stored in the memory 701 to obtain the resources corresponding to the above-mentioned preset events. Specifically, the processor 702 implements the following steps when running the above-mentioned computer program stored in the memory 701:

[0103] Perform text detection on the text image to be recognized and obtain the text box;

[0104] Calculating the distance between the text box and the boundary of the text image to be recognized;

[0105] If the distance is less than a preset distance threshold, the text content in the text box is recognized by a text recognition algorithm based on connection time classification CTC to obtain a CTC recognition result;

[0106] Based on the above CTC recognition results, determine whether there is a need for segmentation;

[0107] If there is a segmentation requirement, the CTC recognition result is segmented, and a text recognition result is obtained based on the segmented CTC recognition result.

[0108] Assuming that the above is the first possible implementation, in a second possible implementation provided on the basis of the first possible implementation, the above calculation of the distance between the text box and the boundary of the to-be-recognized text image includes:

[0109] Determine the maximum horizontal coordinate value of the vertex, where the maximum horizontal coordinate value of the vertex is: the maximum value of the horizontal coordinate values ​​of each vertex in the text box;

[0110] Determine the maximum horizontal coordinate value of the image, where the maximum horizontal coordinate value of the image is: the maximum value of the horizontal coordinate values ​​of each pixel point of the text image to be recognized;

[0111] The difference between the maximum horizontal coordinate value of the image and the maximum horizontal coordinate value of the vertex is determined as the distance.

[0112] In a third possible implementation provided on the basis of the first possible implementation, judging whether there is a segmentation requirement based on the CTC recognition result includes:

[0113] Detect whether there is a valid output value among the last N output values ​​in the CTC recognition result, wherein N is a preset positive integer and N is less than the total number of output values ​​included in the CTC recognition result;

[0114] If there is a valid output value among the last N output values, it is determined that there is a need for segmentation.

[0115] In a fourth possible implementation provided on the basis of the third possible implementation, after the CTC recognition result is segmented and a text recognition result is obtained based on the segmented CTC recognition result, the processor 702 further implements the following steps when running the computer program stored in the memory 701:

[0116] Determine the target segmentation position in the above-mentioned text image to be recognized;

[0117] A new text image to be recognized is obtained, and based on the new text image to be recognized, the text detection step to be performed on the text image to be recognized is returned to obtain the text box step and subsequent steps, wherein the starting position of the new text image to be recognized is determined based on the target segmentation position.

[0118] In a fifth possible implementation provided on the basis of the fourth possible implementation, the step of determining the target segmentation position in the to-be-recognized text image includes:

[0119] Determine a target valid output value among the valid output values ​​of the last N output values;

[0120] Based on the target effective output value, a target segmentation position is determined in the to-be-recognized text image.

[0121] In a sixth possible implementation provided on the basis of the fifth possible implementation, determining the target effective output value among the effective output values ​​existing in the last N output values ​​includes:

[0122] If there is a unique valid output value among the last N output values, the unique valid output value is determined as the target valid output value;

[0123] If there are more than two valid output values ​​among the last N output values, the valid output value that is sorted last among the more than two valid output values ​​is determined as the target valid output value.

[0124] In a seventh possible implementation provided on the basis of the fifth possible implementation, the step of determining the target segmentation position in the to-be-recognized text image based on the target effective output value includes:

[0125] Determine the middle segmentation position in the text box based on the target index number, wherein the target index number is: the index number of the target valid output value in the CTC recognition result;

[0126] The target segmentation position is determined according to the intermediate segmentation position and the starting position of the text box in the to-be-recognized text image.

[0127] It should be understood that in the embodiment of the present application, the processor 702 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0128] The memory 701 may include a read-only memory and a random access memory, and provide instructions and data to the processor 702. A part or all of the memory 701 may also include a non-volatile random access memory. For example, the memory 701 may also store information of a device category.

[0129] As can be seen from the above, through the embodiment of the present application, the distance between the text box and the boundary of the text image to be recognized is used to preliminarily determine whether there is a possibility of character truncation. When it is found that the distance is less than the preset distance threshold, it is considered that the possibility of character truncation is relatively high. At this time, the text content in the text box is recognized by the CTC-based text recognition algorithm, and the intermediate result output by the text recognition algorithm, that is, the CTC recognition result, is used as the object of analysis to further analyze whether there is a segmentation requirement. When there is a segmentation requirement, the CTC recognition result is segmented, and an accurate text recognition result is obtained based on the segmented CTC recognition result. The above process can avoid the influence of truncated characters on the text recognition results, and ensure the accuracy of single text recognition.

[0130] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of external device software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0134] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the associated hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The above-mentioned computer-readable storage medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer-readable memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media does not include electrical carrier signals and telecommunication signals.

[0136] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A text recognition method, characterized in that: include: Perform text detection on the text image to be recognized and obtain the text box; Calculating the distance between the text box and the boundary of the text image to be recognized; If the distance is less than a preset distance threshold, the text content in the text box is recognized by a text recognition algorithm based on connection time classification CTC to obtain a CTC recognition result; Based on the CTC recognition result, determine whether there is a need for segmentation; If there is a segmentation requirement, the CTC recognition result is segmented, and a text recognition result is obtained based on the segmented CTC recognition result; The determining whether there is a segmentation requirement based on the CTC recognition result includes: Detecting whether there is a valid output value among the last N output values ​​in the CTC recognition result, wherein N is a preset positive integer and N is less than the total number of output values ​​included in the CTC recognition result; If there is a valid output value among the last N output values, it is determined that there is a need for segmentation.

2. The text recognition method according to claim 1, characterized in that: The calculating the distance between the text box and the boundary of the to-be-recognized text image includes: Determine a maximum horizontal coordinate value of a vertex, where the maximum horizontal coordinate value of a vertex is: a maximum value among the horizontal coordinate values ​​of each vertex of the text box; Determine the maximum horizontal coordinate value of the image, wherein the maximum horizontal coordinate value of the image is: the maximum value of the horizontal coordinate values ​​of each pixel point of the text image to be recognized; The difference between the maximum horizontal coordinate value of the image and the maximum horizontal coordinate value of the vertex is determined as the distance.

3. The text recognition method according to claim 1, characterized in that: After segmenting the CTC recognition result and obtaining the text recognition result based on the segmented CTC recognition result, the text recognition method further includes: Determining a target segmentation position in the to-be-recognized text image; A new text image to be recognized is obtained, and based on the new text image to be recognized, the text detection step of the text image to be recognized is returned to execute, and the step of obtaining a text box and subsequent steps are performed, wherein the starting position of the new text image to be recognized is determined based on the target segmentation position.

4. The text recognition method according to claim 3, characterized in that: The step of determining a target segmentation position in the to-be-recognized text image includes: Determine a target valid output value among the valid output values ​​of the last N output values; Based on the target effective output value, a target segmentation position is determined in the to-be-recognized text image.

5. The text recognition method according to claim 4, characterized in that: The step of determining a target valid output value among the valid output values ​​of the last N output values ​​comprises: If there is a unique valid output value among the last N output values, determining the unique valid output value as the target valid output value; If there are more than two valid output values ​​among the last N output values, the valid output value that is sorted last among the more than two valid output values ​​is determined as the target valid output value.

6. The text recognition method according to claim 4, characterized in that: The step of determining a target segmentation position in the to-be-recognized text image based on the target effective output value comprises: Determine the middle segmentation position in the text box based on the target index number, wherein the target index number is: the index number of the target valid output value in the CTC recognition result; The target segmentation position is determined according to the middle segmentation position and the starting position of the text box in the to-be-recognized text image.

7. A text recognition device, characterized in that: include: A detection module, used to perform text detection on the text image to be recognized and obtain a text box; A calculation module, used for calculating the distance between the text box and the boundary of the text image to be recognized; A recognition module, configured to recognize the text content in the text box by using a text recognition algorithm based on connection time classification (CTC) if the distance is less than a preset distance threshold, and obtain a CTC recognition result; A judgment module, used to judge whether there is a segmentation requirement based on the CTC recognition result; A segmentation module, configured to segment the CTC recognition result if segmentation is required, and obtain a text recognition result based on the segmented CTC recognition result; Wherein, the judgment module includes: A detection unit, configured to detect whether there is a valid output value among the last N output values ​​in the CTC recognition result, wherein N is a preset positive integer and N is less than the total number of output values ​​included in the CTC recognition result; The determining unit is used to determine whether there is a segmentation requirement if there is a valid output value among the last N output values.

8. A scanning pen, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images