Character recognition post-processing method and device, character recognition method and device, medium and equipment

Abstract

The invention discloses a character recognition post-processing method and device, a character recognition method and device, a medium and equipment, and belongs to the field of character recognition. According to the method and the device, the inverse and the normal of the text bar are not judged, but the text bar and the new text bar which is rotated by 180 degrees are respectively input into the character recognition model, so that two groups of recognition results are obtained. And performing low-confidence character elimination and continuous and repeated character combination on the two groups of recognition results. And finally, determining one of the two groups of recognition results as a final character recognition result according to the character length information and the confidence information of the two groups of recognition results after the characters are removed and the continuous and repeated characters are combined. According to the method and the device, the forward direction and the reverse direction of the text bar are not distinguished through a direction classifier, so that the character recognition time is saved, and the character recognition accuracy is improved when the forward direction and the reverse direction of the text bar are difficult to distinguish. In addition, the character recognition accuracy is further improved by eliminating the characters with low confidence coefficient.

Description

Technical Field

[0001] The present application relates to the field of text recognition, and in particular to a text recognition post-processing and text recognition method, device, medium and equipment. Background Art

[0002] Optical Character Recognition (OCR, also known as text recognition) is a technology that uses computers to automatically recognize characters and is an important field of pattern recognition applications. Optical character recognition generally includes the following steps:

[0003] 1. Text detection: Use the text detection algorithm to determine whether there is a text instance, give the specific location of the text strip, and cut out the text strip.

[0004] 2. Use the direction classifier algorithm to distinguish the upright and inverted text strips, and rotate the inverted text strips 180°, while leaving the upright text strips unprocessed.

[0005] 3. Use a text recognition model (such as a CTC text recognition model) to recognize the text information contained in the upright text strip.

[0006] In some special OCR scenarios, such as bank card number OCR, due to the particularity of the bank card number text, it is very difficult to distinguish the front and back of the bank card number text. Figure 1 The bank card number text strip captured is as follows: Figure 2 As shown, Figure 2 (a) is an upright text strip, and (b) is an inverted text strip. Figure 2 The error probability of the bank card number text strip shown is very high. If the inverted text strip is input into the text recognition model, an incorrect result will inevitably be output. Summary of the invention

[0007] To address the deficiencies of the prior art, the present application provides a text recognition post-processing and text recognition method, device, medium and equipment, which improve the speed and accuracy of text recognition.

[0008] This application provides the following technical solutions:

[0009] In a first aspect, the present application provides a text recognition post-processing method, the method comprising:

[0010] Inputting the acquired text strip and the new text strip obtained by rotating the text strip by 180° into a text recognition model respectively, and obtaining a first character sequence output by the text recognition model, a first confidence sequence formed by the confidences of the characters in the first character sequence, a second character sequence, and a second confidence sequence formed by the confidences of the characters in the second character sequence respectively;

[0011] According to the first confidence sequence, characters with low confidence are removed from the first character sequence and characters that are repeated continuously are combined to obtain a third character sequence, and the confidence of each character in the third character sequence is obtained to form a third confidence sequence; according to the second confidence sequence, characters with low confidence are removed from the second character sequence and characters that are repeated continuously are combined to obtain a fourth character sequence, and the confidence of each character in the fourth character sequence is obtained to form a fourth confidence sequence;

[0012] According to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence, it is determined that the third character sequence or the fourth character sequence is a text recognition result.

[0013] Further, according to the first confidence sequence, the characters with low confidence are removed from the first character sequence and the characters that are repeated continuously are combined to obtain a third character sequence, and the confidence of each character of the third character sequence is obtained to form a third confidence sequence; according to the second confidence sequence, the characters with low confidence are removed from the second character sequence and the characters that are repeated continuously are combined to obtain a fourth character sequence, and the confidence of each character of the fourth character sequence is obtained to form a fourth confidence sequence, including:

[0014] Comparing the confidence of each character in the first character sequence / the second character sequence with a set confidence threshold, and removing characters whose confidence is less than the confidence threshold and their confidences from the first character sequence / the second character sequence and the first confidence sequence / the second confidence sequence respectively; obtaining a fifth character sequence, the confidences of each character in the fifth character sequence to form a fifth confidence sequence, a sixth character sequence, and the confidences of each character in the sixth character sequence to form a sixth confidence sequence;

[0015] The continuously repeated characters in the fifth character sequence / the sixth character sequence are merged, and the statistical value of the confidence of all the characters involved in the merging is used as the confidence of the merged character; the third character sequence, the third confidence sequence, the fourth character sequence and the fourth confidence sequence are obtained.

[0016] Furthermore, the statistical value is the maximum value of the confidence of all characters involved in the merging.

[0017] Further, determining that the third character sequence or the fourth character sequence is a text recognition result according to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence includes:

[0018] If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is greater than a set first difference threshold, the one with the larger number of characters in the third character sequence and the fourth character sequence is used as the character recognition result;

[0019] If the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters of the third character sequence is not equal to that of the fourth character sequence, then determine whether the difference between the average confidence value of all characters of the third character sequence and the fourth character sequence with a smaller number of characters and the average confidence value of all characters of the third character sequence and the fourth character sequence with a larger number of characters is greater than a set second difference threshold; if so, take the third character sequence and the fourth character sequence with a smaller number of characters as the text recognition result; otherwise, take the third character sequence and the fourth character sequence with a larger number of characters as the text recognition result;

[0020] If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters in the third character sequence is equal to that in the fourth character sequence, the one with a higher average confidence value of all characters in the third character sequence and the fourth character sequence is taken as the text recognition result.

[0021] Furthermore, the method further comprises:

[0022] The one of the text strip and the new text strip corresponding to the character recognition result is used as the upright text strip.

[0023] In a second aspect, the present application provides a text recognition post-processing device, the device comprising:

[0024] A data acquisition module, used to input the acquired text strip and the new text strip obtained by rotating the text strip by 180° into a text recognition model, respectively, to obtain a first character sequence output by the text recognition model, a first confidence sequence formed by the confidences of the characters in the first character sequence, a second character sequence, and a second confidence sequence formed by the confidences of the characters in the second character sequence;

[0025] a removal and repetition merging module, configured to remove characters with low confidence and merge characters that are repeated continuously from the first character sequence according to the first confidence sequence to obtain a third character sequence, and obtain the confidence of each character in the third character sequence to form a third confidence sequence; remove characters with low confidence and merge characters that are repeated continuously from the second character sequence according to the second confidence sequence to obtain a fourth character sequence, and obtain the confidence of each character in the fourth character sequence to form a fourth confidence sequence;

[0026] The text recognition result determination module is used to determine whether the third character sequence or the fourth character sequence is a text recognition result according to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence.

[0027] Furthermore, the elimination and duplication merging module includes:

[0028] a removing unit, used for comparing the confidence of each character of the first character sequence / the second character sequence with a set confidence threshold, removing characters with confidences less than the confidence threshold and their confidences from the first character sequence / the second character sequence and the first confidence sequence / the second confidence sequence respectively; obtaining a fifth character sequence, the confidences of each character of the fifth character sequence to form a fifth confidence sequence, a sixth character sequence, and the confidences of each character of the sixth character sequence to form a sixth confidence sequence;

[0029] A repeated merging unit is used to merge the characters that are repeated continuously in the fifth character sequence / the sixth character sequence, and use the statistical value of the confidence of all the characters involved in the merging as the confidence of the merged character; to obtain the third character sequence, the third confidence sequence, the fourth character sequence and the fourth confidence sequence.

[0030] Furthermore, the statistical value is the maximum value of the confidence of all characters involved in the merging.

[0031] Furthermore, the text recognition result determination module includes:

[0032] a length judgment unit, configured to take the one with more characters between the third character sequence and the fourth character sequence as a character recognition result if the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is greater than a set first difference threshold;

[0033] a length and confidence judgment unit, for judging whether the difference between the confidence average of all characters of the third character sequence and the fourth character sequence with a smaller number of characters and the confidence average of all characters of the third character sequence and the fourth character sequence with a larger number of characters is greater than a set second difference threshold if the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters of the third character sequence and the fourth character sequence are not equal, and if so, taking the third character sequence and the fourth character sequence with a smaller number of characters as a text recognition result, otherwise taking the third character sequence and the fourth character sequence with a larger number of characters as a text recognition result;

[0034] A confidence judgment unit is used to take the higher average confidence value of all characters in the third character sequence and the fourth character sequence as the text recognition result if the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters in the third character sequence is equal to that in the fourth character sequence.

[0035] Furthermore, the device also includes:

[0036] The upright text strip determination module is used to take the one of the text strip and the new text strip corresponding to the character recognition result as the upright text strip.

[0037] In a third aspect, the present application provides a computer-readable storage medium for text recognition post-processing, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the text recognition post-processing method described in the first aspect.

[0038] In a fourth aspect, the present application provides a device for post-processing of text recognition, characterized in that it includes at least one processor and a memory storing computer-executable instructions, and when the processor executes the instructions, it implements the steps of the text recognition post-processing method described in the first aspect.

[0039] In a fifth aspect, the present application provides a text recognition method, the method comprising:

[0040] Perform text detection on the image to be recognized to determine whether there is text in the image to be recognized, and if it is determined that there is text, intercept the position where the text exists to obtain a text strip;

[0041] The text recognition result is obtained by performing post-processing through the text recognition post-processing method.

[0042] In a sixth aspect, the present application provides a text recognition device, the device comprising:

[0043] A text detection module is used to perform text detection on the image to be recognized, determine whether there is text in the image to be recognized, and when it is determined that there is text, intercept the position where the text exists to obtain a text strip;

[0044] The text recognition post-processing module is used to perform post-processing through the text recognition post-processing method to obtain the text recognition result.

[0045] In a seventh aspect, the present application provides a computer-readable storage medium for text recognition, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the text recognition method described in the fifth aspect.

[0046] In an eighth aspect, the present application provides a device for text recognition, characterized in that it includes at least one processor and a memory storing computer-executable instructions, and when the processor executes the instructions, it implements the steps of the text recognition method described in the fifth aspect.

[0047] This application has the following beneficial effects:

[0048] In the present application, the inverted or upright position of the text strip is not judged, but the text strip and the new text strip after being rotated 180° are respectively input into the text recognition model to obtain two groups of recognition results. The two groups of recognition results are subjected to low-confidence character elimination and continuous repeated characters merging. Finally, one of the two groups of recognition results is determined as the final text recognition result based on the character length information and confidence information of the two groups of recognition results after character elimination and continuous repeated characters merging. The present application does not distinguish the upright or inverted position of the text strip through a direction classifier, which not only saves the time of text recognition, but also improves the accuracy of text recognition when it is difficult to distinguish the upright or inverted position of the text strip. In addition, the present application further improves the accuracy of text recognition by eliminating characters with low confidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is an example picture of a bank card;

[0050] Figure 2 for Figure 1 Schematic diagrams showing the bank card number text strip of the bank card in upright and inverted positions;

[0051] Figure 3 A flowchart of the text recognition post-processing method of the present application;

[0052] Figure 4 A schematic diagram of a text recognition post-processing device of the present application;

[0053] Figure 5 A flowchart of the text recognition method of the present application;

[0054] Figure 6 A schematic diagram of a text recognition device of the present application. DETAILED DESCRIPTION

[0055] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0056] The present application embodiment provides a text recognition post-processing method for a text recognition post-processing process, such as Figure 3 As shown, the method includes:

[0057] S100: Input the acquired text strip and the new text strip obtained by rotating the text strip by 180° into the text recognition model respectively, and obtain a first character sequence output by the text recognition model, a first confidence sequence formed by the confidence of each character in the first character sequence, a second character sequence, and a second confidence sequence formed by the confidence of each character in the second character sequence respectively.

[0058] In the present application, a text strip refers to a portion of an image area containing text that is intercepted from an image to be identified. When intercepting, text detection is first performed on the image to be identified to determine whether there is text in the image to be identified. When it is determined that there is text, the area where the text exists is intercepted to obtain a text strip. Generally speaking, the intercepted image area containing text is in the shape of a strip, so it is called a text strip. It should be noted that the present application does not limit the text strip to a strip shape, and any shape (such as block) of the intercepted image area containing text is collectively referred to as a text strip.

[0059] In some special OCR scenarios, such as when it is difficult to distinguish the front and back of a bank card number text strip, this application does not use the direction classifier algorithm to distinguish the front and back of the text strip, but instead inputs the text strip and the new text strip rotated 180° into the text recognition model separately to obtain two sets of recognition results.

[0060] One of the text strip and the new text strip after rotating 180° must be inverted and the other must be upright. The two sets of results obtained by the text recognition model (i.e., the first character sequence + the first confidence sequence, the second character sequence + the second confidence sequence) are the text recognition results of the correct placement direction, and the other set is the text recognition results of the wrong placement direction. However, it is not clear which of the text strip and the new text strip after rotating 180° is inverted and which is upright, that is, it is uncertain which text recognition is correct. The present application obtains the correct text recognition result through subsequent operations.

[0061] The present application does not limit the specific implementation of the text recognition model. For example, the text recognition model can select the CTC text recognition model. The text strip and the new text strip rotated 180° are respectively input into the CTC text recognition model to obtain the corresponding first character sequence charList_0 and second character sequence charList_1. The confidence of each character in the first character sequence charList_0 constitutes the first confidence sequence confidenceList_0, and the confidence of each character in the second character sequence charList_1 constitutes the second confidence sequence confidenceList_1.

[0062] An example of the first character sequence charList_0 and the first confidence sequence confidenceList_0 is as follows:

[0063] charList_0:["5","5","5","5","","","7","7","7"]

[0064] confidenceList_0:[0.5,0.8,0.9,0.7,0.5,0.8,0.96,0.85,0.56]

[0065] Among them, "" means no character.

[0066] S200: According to the first confidence sequence, the characters with low confidence are eliminated from the first character sequence and the characters that are repeated continuously are combined to obtain a third character sequence, and the confidence of each character of the third character sequence is obtained to form a third confidence sequence; according to the second confidence sequence, the characters with low confidence are eliminated from the second character sequence and the characters that are repeated continuously are combined to obtain a fourth character sequence, and the confidence of each character of the fourth character sequence is obtained to form a fourth confidence sequence.

[0067] The output results of the text recognition model show that some characters in the first character sequence charList_0 and the second character sequence charList_1 have low confidence, indicating that the possibility of the position being the character is low, so the application removes the characters with low confidence.

[0068] The character sequence and confidence sequence output by the existing text recognition model are directly used for the subsequent continuous repeated character merging operation without eliminating characters with low confidence, which makes it difficult to ensure the accuracy of the subsequent text recognition results. The present application eliminates characters with low confidence by the confidence of each character, thereby improving the accuracy of the text recognition results.

[0069] In addition, the text recognition model usually generates consecutively repeated characters when recognizing each character, so there will be a large number of consecutively repeated characters in the output first character sequence charList_0 and the second character sequence charList_1. After eliminating characters with low confidence, the present application also needs to merge the consecutively repeated characters.

[0070] S300: Determine, based on the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence, whether the third character sequence or the fourth character sequence is a text recognition result.

[0071] From the above, one of the text strip and the new text strip after rotating 180° must be correct and the other must be wrong. For the upright text strip, since the text is upright, it is easy to recognize. For the inverted text strip, since the text is inverted, some of the inverted text cannot be recognized. Therefore, generally speaking, the character length of the text recognition of the upright text strip is greater than that of the inverted text strip.

[0072] Moreover, for an upright text strip, since the text is upright, the confidence of the recognized characters is high. For an inverted text strip, since the text is inverted, the confidence of the recognized characters is low.

[0073] Therefore, by using the length information of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence (ie, confidence information), it is possible to determine which of the third character sequence and the fourth character sequence comes from the upright text strip, and use its characters as the final text recognition result.

[0074] In the present application, the inverted or upright position of the text strip is not judged, but the text strip and the new text strip after being rotated 180° are respectively input into the text recognition model to obtain two groups of recognition results. The two groups of recognition results are subjected to low-confidence character elimination and continuous repeated characters merging. Finally, one of the two groups of recognition results is determined as the final text recognition result based on the character length information and confidence information of the two groups of recognition results after character elimination and continuous repeated characters merging. The present application does not distinguish the upright or inverted position of the text strip through a direction classifier, which not only saves the time of text recognition, but also improves the accuracy of text recognition when it is difficult to distinguish the upright or inverted position of the text strip. In addition, the present application further improves the accuracy of text recognition by eliminating characters with low confidence.

[0075] This application does not limit the method of removing characters. In one example, the removal method is as follows:

[0076] S210: Compare the confidence of each character in the first character sequence / second character sequence with the set confidence threshold, and remove the characters with confidence less than the confidence threshold and their confidence from the first character sequence / second character sequence and the first confidence sequence / second confidence sequence respectively; obtain the fifth character sequence, the confidence of each character in the fifth character sequence forms a fifth confidence sequence, the sixth character sequence, and the confidence of each character in the sixth character sequence forms a sixth confidence sequence.

[0077] For example, when the confidence threshold is set to 0.6, the fifth character sequence charList_0_0 and the fifth confidence sequence confidenceList_0_0 obtained by removing the first character sequence charList_0 and the first confidence sequence confidenceList_0 are as follows.

[0078] charList_0_0:["5","5","5","","7","7"]

[0079] confidenceList_0_0:[0.8,0.9,0.7,0.8,0.96,0.85]

[0080] Similarly, the second character sequence charList_1 and the second confidence sequence confidenceList_1 are eliminated to obtain the sixth character sequence charList_1_0 and the sixth confidence sequence confidenceList_1_0.

[0081] The present application eliminates characters with low confidence by setting a threshold for the confidence of each character, thereby improving the accuracy of text recognition.

[0082] In the prior art, when merging consecutive repeated characters, the average confidence of the character sequence is output as the confidence of the character sequence. Assuming that there are only three characters A, B, and C in the dictionary (the characters in the dictionary of the text recognition model are the characters it can recognize), the character sequence char_list output by a text recognition model and its corresponding confidence sequence confidence_list are as follows:

[0083] char_list:["A","A","A","","","A","A","","","B","","C","C"]

[0084] confidence_list:[0.5,0.8,0.9,0.7,0.5,0.8,0.9,0.7,0.5,0.8,0.96,0.85,0.56]

[0085] Then the existing text recognition model post-processes and merges the repeated "A", "", and "C" to obtain the final string "AABC", and the overall confidence of the string is the average value of confidence_list, 0.728.

[0086] The method of merging consecutive repeated characters in the present application is different from that in the prior art. In one example, the merging method is as follows:

[0087] S220: merge the consecutively repeated characters in the fifth character sequence / sixth character sequence, and use the statistical value of the confidence of all the characters involved in the merger as the confidence of the merged character; obtain a third character sequence, a third confidence sequence, a fourth character sequence and a fourth confidence sequence.

[0088] For example, in the fifth character sequence charList_0_0, the first three consecutive characters "5" are repeated characters and need to be merged. After merging, only one character "5" is retained, and the confidence of the merged character "5" is the statistical value of the confidence of the three characters "5" involved in the merger. Preferably, the statistical value is the maximum value.

[0089] The third character sequence charList_0_1 and the third confidence sequence confidenceList_0_1 obtained after merging consecutive repeated characters from the fifth character sequence charList_0_0 and the fifth confidence sequence confidenceList_0_0 are as follows.

[0090] charList_0_1:["5","","7"]

[0091] confidenceList_0_1:[0.9,0.8,0.96]

[0092] Similarly, after the sixth character sequence charList_1_0 and the sixth confidence sequence confidenceList_1_0 are combined with consecutive repeated characters, the fourth character sequence charList_1_1 and the fourth confidence sequence confidenceList_1_1 are obtained.

[0093] The present application does not directly merge consecutive repeated characters, but merges consecutive repeated characters after removing characters with low confidence. When merging consecutive repeated characters, the average confidence is not used as the overall confidence of the entire character sequence, but a confidence is set for each merged character, and the confidence is the maximum confidence of the characters involved in the merger. The present application improves the accuracy of the confidence of each character, provides a basis for the subsequent final text recognition results, and improves the accuracy of text recognition.

[0094] As an improvement of the embodiment of the present invention, the aforementioned S300 includes:

[0095] S310: If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is greater than a set first difference threshold, the one with more characters in the third character sequence and the fourth character sequence is used as the text recognition result.

[0096] In this application, the length of the character sequence is used as the first judgment criterion. First, the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence needs to be calculated, and then compared with the set first difference threshold. If it is greater than the first difference threshold, the one with the larger number of characters is used as the text recognition result.

[0097] For example, the first difference threshold may be 3, 4, etc. Assuming that the length of the third character sequence is 3 characters longer than that of the fourth character sequence, the characters of the third character sequence are used as the final text for character recognition.

[0098] S320: If the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is not greater than the set first difference threshold and the number of characters of the third character sequence is not equal to that of the fourth character sequence, determine whether the difference between the average confidence value of all characters of the third character sequence with fewer characters and the average confidence value of all characters of the third character sequence with more characters is greater than the set second difference threshold. If so, take the third character sequence with fewer characters as the text recognition result; otherwise, take the third character sequence with more characters as the text recognition result.

[0099] In the present application, the confidence of the character sequence is used as the second judgment criterion. When it is impossible to determine which character sequence is used as the final text recognition result only by the length of the character sequence, the number of characters and the confidence can be comprehensively considered for judgment.

[0100] When the absolute value of the difference in the number of characters is not greater than the set first difference threshold, first determine whether the number of characters in the third character sequence is equal to that in the fourth character sequence. If they are not equal, calculate the difference between the average confidence value of all characters in the one with fewer characters and the average confidence value of all characters in the one with more characters, and compare it with the set second difference threshold. If it is greater than the second difference threshold, the one with fewer characters is used as the text recognition result. If it is less than or equal to the second difference threshold, the one with more characters is used as the text recognition result.

[0101] S330: If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is not greater than the set first difference threshold and the number of characters in the third character sequence is equal to that in the fourth character sequence, the one with the higher average confidence value of all characters in the third character sequence and the fourth character sequence is taken as the text recognition result.

[0102] When the number of characters in the third character sequence is equal to that in the fourth character sequence, the judgment can only be made based on the confidence level, and the one with the higher average confidence level is taken as the text recognition result.

[0103] The present application can also determine the upright text strips. Accordingly, to achieve this purpose, the method of the present application also includes:

[0104] S400: taking the one of the text strip and the new text strip corresponding to the character recognition result as the upright text strip.

[0105] For example, assuming that the third character sequence is the result of text recognition, the text strip corresponding to the third character sequence is the upright text strip. Conversely, assuming that the fourth character sequence is the result of text recognition, the new text strip corresponding to the fourth character sequence after being rotated 180° is the upright text strip.

[0106] The present application uses the character sequence length as the first judgment standard and the confidence as the second judgment standard to distinguish between the forward and reverse directions of a text strip, and the accuracy is higher than the accuracy of the direction classifier in distinguishing between the forward and reverse directions.

[0107] The present application also provides a text recognition post-processing device, such as Figure 4 As shown, the device comprises:

[0108] The data acquisition module 100 is used to input the acquired text strip and the new text strip obtained by rotating the text strip by 180° into the text recognition model respectively, and obtain the first character sequence output by the text recognition model, the first confidence sequence formed by the confidence of each character in the first character sequence, the second character sequence and the second confidence sequence formed by the confidence of each character in the second character sequence respectively.

[0109] The elimination and duplication merging module 200 is used to eliminate low-confidence characters and merge consecutively repeated characters from a first character sequence according to a first confidence sequence to obtain a third character sequence, and obtain the confidence of each character in the third character sequence to form a third confidence sequence; eliminate low-confidence characters and merge consecutively repeated characters from a second character sequence according to a second confidence sequence to obtain a fourth character sequence, and obtain the confidence of each character in the fourth character sequence to form a fourth confidence sequence.

[0110] The text recognition result determination module 300 is used to determine whether the third character sequence or the fourth character sequence is a text recognition result according to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence.

[0111] In the present application, the inverted or upright position of the text strip is not judged, but the text strip and the new text strip after being rotated 180° are respectively input into the text recognition model to obtain two groups of recognition results. The two groups of recognition results are subjected to low-confidence character elimination and continuous repeated characters merging. Finally, one of the two groups of recognition results is determined as the final text recognition result based on the character length information and confidence information of the two groups of recognition results after character elimination and continuous repeated characters merging. The present application does not distinguish the upright or inverted position of the text strip through a direction classifier, which not only saves the time of text recognition, but also improves the accuracy of text recognition when it is difficult to distinguish the upright or inverted position of the text strip. In addition, the present application further improves the accuracy of text recognition by eliminating characters with low confidence.

[0112] This application does not limit the specific implementation of the elimination and duplicate merging modules. In one example, it includes:

[0113] The elimination unit is used to compare the confidence of each character in the first character sequence / second character sequence with a set confidence threshold, and eliminate the characters with confidence less than the confidence threshold and their confidence from the first character sequence / second character sequence and the first confidence sequence / second confidence sequence respectively; obtain the fifth character sequence, the confidence of each character in the fifth character sequence forms a fifth confidence sequence, the sixth character sequence, and the confidence of each character in the sixth character sequence forms a sixth confidence sequence.

[0114] The repeat merging unit is used to merge the characters repeated continuously in the fifth character sequence / sixth character sequence, and use the statistical value of the confidence of all the characters involved in the merging as the confidence of the merged character; to obtain the third character sequence, the third confidence sequence, the fourth character sequence and the fourth confidence sequence.

[0115] Preferably, the statistical value is the maximum value of the confidence of all characters involved in the merging.

[0116] As an improvement of the embodiment of the present application, the text recognition result determination module includes:

[0117] The length judgment unit is used to take the one with more characters in the third character sequence and the fourth character sequence as the text recognition result if the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is greater than a set first difference threshold.

[0118] A length and confidence judgment unit is used to judge whether the difference between the confidence average of all characters of the third character sequence and the fourth character sequence with fewer characters and the confidence average of all characters of the third character sequence and the fourth character sequence with more characters is greater than a set second difference threshold if the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters of the third character sequence and the fourth character sequence are not equal is greater than a set second difference threshold; if so, the third character sequence and the fourth character sequence with fewer characters are used as the text recognition result; otherwise, the third character sequence and the fourth character sequence with more characters are used as the text recognition result.

[0119] A confidence judgment unit is used to take the higher average confidence value of all characters in the third character sequence and the fourth character sequence as the text recognition result if the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters in the third character sequence is equal to that in the fourth character sequence.

[0120] The device of the present invention also includes:

[0121] The module for determining the text strip to be placed upright is used to take the one of the text strip and the new text strip corresponding to the character recognition result as the text strip to be placed upright.

[0122] The device provided in the above embodiment, its implementation principle and the technical effect produced correspond to the embodiment of the above method one by one. For the sake of brief description, the part not mentioned in the embodiment of the device can refer to the corresponding content in the embodiment of the above method. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the modules and units described in the device can refer to the corresponding process in the embodiment of the above method, and will not be repeated here.

[0123] The present application also provides a text recognition method. Figure 5 As shown, the method includes:

[0124] S10: Perform text detection on the image to be recognized to determine whether there is text in the image to be recognized. When it is determined that there is text, the position where the text exists is intercepted to obtain a text strip.

[0125] The present application does not limit the method of text detection, and various text detection algorithms in the prior art can be used for implementation. The specific process of text detection is prior art and will not be described in detail in the present application. The text strip obtained in this step has uncertain forward and backward directions.

[0126] S20: Post-processing is performed using the aforementioned text recognition post-processing method to obtain a text recognition result.

[0127] This step refers to the above-mentioned embodiment, and will not be described in detail in this embodiment.

[0128] In the present application, whether the text strip is inverted or upright is not determined. Instead, the text strip and the new text strip rotated 180° are respectively input into the text recognition model to obtain two groups of recognition results. Then, the two groups of recognition results are subjected to low-confidence character elimination and continuous repeated characters merging through the aforementioned post-processing method. Finally, one of the two groups of recognition results is determined as the final text recognition result based on the character length information and confidence information of the two groups of recognition results after character elimination and continuous repeated characters merging. The present application does not distinguish between upright and inverted text strips through a direction classifier, which not only saves time for text recognition, but also improves the accuracy of text recognition when it is difficult to distinguish between upright and inverted text strips. In addition, the present application further improves the accuracy of text recognition by eliminating characters with low confidence.

[0129] The present application also provides a text recognition device. Figure 6 As shown, the device comprises:

[0130] The text detection module 10 is used to perform text detection on the image to be recognized, and determine whether there is text in the image to be recognized. When it is determined that there is text, the position where the text exists is intercepted to obtain a text strip.

[0131] The text recognition post-processing module 20 is used to perform post-processing using the text recognition post-processing method to obtain text recognition results.

[0132] The device provided in the above embodiment, its implementation principle and the technical effect produced correspond to the embodiment of the above method one by one. For the sake of brief description, the part not mentioned in the embodiment of the device can refer to the corresponding content in the embodiment of the above method. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the modules and units described in the device can refer to the corresponding process in the embodiment of the above method, and will not be repeated here.

[0133] The text recognition post-processing method / text recognition method described in the above embodiments provided by the present application can implement business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the scheme described in the method embodiment of this specification. Therefore, the embodiment of the present application also provides a computer-readable storage medium for text recognition post-processing / text recognition, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the text recognition post-processing method / text recognition method including the above embodiments are implemented.

[0134] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0135] The storage medium described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0136] The embodiment of the present application also provides a device for post-processing text recognition / text recognition, which may be a separate computer, or may include an actual operating device using one or more of the methods or one or more of the embodiments of the present specification. The device for post-processing text recognition / text recognition may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, it implements any one or more of the steps of the post-processing method / text recognition method described above.

[0137] The device described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0138] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text recognition post-processing method, characterized in that: The method comprises: Inputting the acquired text strip and the new text strip obtained by rotating the text strip by 180° into a text recognition model respectively, and obtaining a first character sequence output by the text recognition model, a first confidence sequence formed by the confidences of the characters in the first character sequence, a second character sequence, and a second confidence sequence formed by the confidences of the characters in the second character sequence respectively; According to the first confidence sequence, characters with low confidence are removed from the first character sequence and characters that are repeated continuously are combined to obtain a third character sequence, and the confidence of each character in the third character sequence is obtained to form a third confidence sequence; according to the second confidence sequence, characters with low confidence are removed from the second character sequence and characters that are repeated continuously are combined to obtain a fourth character sequence, and the confidence of each character in the fourth character sequence is obtained to form a fourth confidence sequence; According to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence, it is determined that the third character sequence or the fourth character sequence is a text recognition result.

2. The text recognition post-processing method according to claim 1, characterized in that: The method of removing characters with low confidence from the first character sequence and merging characters that are repeated continuously to obtain a third character sequence according to the first confidence sequence, and obtaining the confidence of each character of the third character sequence to form a third confidence sequence; removing characters with low confidence from the second character sequence and merging characters that are repeated continuously to obtain a fourth character sequence according to the second confidence sequence, and obtaining the confidence of each character of the fourth character sequence to form a fourth confidence sequence includes: Comparing the confidence of each character in the first character sequence / the second character sequence with a set confidence threshold, and removing characters whose confidence is less than the confidence threshold and their confidences from the first character sequence / the second character sequence and the first confidence sequence / the second confidence sequence respectively; obtaining a fifth character sequence, the confidences of each character in the fifth character sequence to form a fifth confidence sequence, a sixth character sequence, and the confidences of each character in the sixth character sequence to form a sixth confidence sequence; The continuously repeated characters in the fifth character sequence / the sixth character sequence are merged, and the statistical value of the confidence of all the characters involved in the merging is used as the confidence of the merged character; the third character sequence, the third confidence sequence, the fourth character sequence and the fourth confidence sequence are obtained.

3. The text recognition post-processing method according to claim 2, characterized in that: The statistical value is the maximum value of the confidence of all characters involved in the merging.

4. The text recognition post-processing method according to any one of claims 1 to 3, characterized in that: The determining, based on the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence, that the third character sequence or the fourth character sequence is a text recognition result includes: If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is greater than a set first difference threshold, the one with the larger number of characters in the third character sequence and the fourth character sequence is used as the character recognition result; If the absolute value of the difference between the number of characters of the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters of the third character sequence is not equal to that of the fourth character sequence, then determine whether the difference between the average confidence value of all characters of the third character sequence and the fourth character sequence with a smaller number of characters and the average confidence value of all characters of the third character sequence and the fourth character sequence with a larger number of characters is greater than a set second difference threshold; if so, take the third character sequence and the fourth character sequence with a smaller number of characters as the text recognition result; otherwise, take the third character sequence and the fourth character sequence with a larger number of characters as the text recognition result; If the absolute value of the difference between the number of characters in the third character sequence and the fourth character sequence is not greater than a set first difference threshold and the number of characters in the third character sequence is equal to that in the fourth character sequence, the one with a higher average confidence value of all characters in the third character sequence and the fourth character sequence is taken as the text recognition result.

5. The text recognition post-processing method according to claim 4, characterized in that: The method further comprises: The one of the text strip and the new text strip corresponding to the character recognition result is used as the upright text strip.

6. A text recognition post-processing device, characterized in that: The device comprises: A data acquisition module, used to input the acquired text strip and the new text strip obtained by rotating the text strip by 180° into a text recognition model, respectively, to obtain a first character sequence output by the text recognition model, a first confidence sequence formed by the confidences of the characters in the first character sequence, a second character sequence, and a second confidence sequence formed by the confidences of the characters in the second character sequence; a removal and repetition merging module, configured to remove characters with low confidence and merge characters that are repeated continuously from the first character sequence according to the first confidence sequence to obtain a third character sequence, and obtain the confidence of each character in the third character sequence to form a third confidence sequence; remove characters with low confidence and merge characters that are repeated continuously from the second character sequence according to the second confidence sequence to obtain a fourth character sequence, and obtain the confidence of each character in the fourth character sequence to form a fourth confidence sequence; The text recognition result determination module is used to determine whether the third character sequence or the fourth character sequence is a text recognition result according to the lengths of the third character sequence and the fourth character sequence and the values ​​of the third confidence sequence and the fourth confidence sequence.

7. A computer-readable storage medium for post-processing of text recognition, characterized in that: It comprises a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the text recognition post-processing method according to any one of claims 1-5 are implemented.

8. A device for post-processing of text recognition, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the text recognition post-processing method described in any one of claims 1 to 5 are implemented.

9. A method for character recognition, characterized in that: The method comprises: Perform text detection on the image to be recognized to determine whether there is text in the image to be recognized, and if it is determined that there is text, intercept the position where the text exists to obtain a text strip; Post-processing is performed by the text recognition post-processing method described in any one of claims 1 to 5 to obtain a text recognition result.

10. A text recognition device, characterized in that: The device comprises: A text detection module is used to perform text detection on the image to be recognized, determine whether there is text in the image to be recognized, and when it is determined that there is text, intercept the position where the text exists to obtain a text strip; The text recognition post-processing module is used to perform post-processing by the text recognition post-processing method described in any one of claims 1-5 to obtain a text recognition result.

Images