A text recognition method and related apparatus
By combining the thought chain decoder and text recognition decoder of the text recognition model, decoding is performed word by word and character by character, correcting text errors in low-quality images and improving the accuracy and robustness of text recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2026-03-25
- Publication Date
- 2026-06-09
AI Technical Summary
Existing text recognition methods perform poorly in low-quality images, especially under conditions such as illegible handwriting, blurry images, and low lighting.
The text recognition model uses a thought chain decoder to decode word by word, correcting errors in the target real text. It also uses a text recognition decoder to decode character by character, combined with visual features for error correction, to generate more semantically accurate recognition text.
It improves the accuracy of text recognition, especially under conditions such as illegible handwriting, blurry images, and low light, and can more accurately restore the characters and structures in the image.
Smart Images
Figure CN122176728A_ABST
Abstract
Claims
1. A text recognition method, characterized in that, include: Acquire the image to be recognized; The visual features of the image to be recognized are extracted using an image encoder of a text recognition model, which also includes a thought chain decoder and a text recognition decoder. The first recognized text is obtained by using the thought chain decoder to perform word-by-word decoding based on the visual features; Using the first recognized text as a prompt, the text recognition decoder is used to decode character by character in combination with the visual features to obtain the second recognized text; The second recognized text is used as the final recognized text corresponding to the image to be recognized.
2. The text recognition method according to claim 1, characterized in that, The step of using the thought chain decoder to perform word-by-word decoding based on the visual features to obtain the first recognized text includes: The thought chain decoder is used to perform word-by-word decoding based on the visual features to obtain the first recognized text and the recognition result label. The recognition result label indicates whether the first recognized text is consistent with the target real text in the image to be recognized. The step of using the text recognition decoder with the first recognized text as a prompt, combining the visual features to perform character-by-character decoding to obtain the second recognized text, and using the second recognized text as the final recognized text corresponding to the image to be recognized, includes: If the recognition result label indicates that the first recognized text is inconsistent with the target real text, then the text recognition decoder uses the first recognized text as a prompt and combines the visual features to perform character-by-character decoding to obtain the second recognized text, which is the final recognized text corresponding to the image to be recognized.
3. The text recognition method according to claim 2, characterized in that, Also includes: If the recognition result label indicates that the first recognized text is consistent with the target real text, then the first recognized text is taken as the final recognized text; or, When the recognition result label indicates that the first recognized text is consistent with the target real text, and the decoding probability corresponding to the first recognized text is obtained, the text recognition decoder uses the first recognized text as a prompt and combines the visual features to perform character-by-character decoding to obtain the second recognized text and the decoding probability corresponding to the second recognized text. The recognized text with the highest decoding probability among the first recognized text and the second recognized text is taken as the final recognized text.
4. The text recognition method according to claim 1, characterized in that, The process by which the thought chain decoder performs word-by-word decoding based on the visual features to obtain the first recognized text includes: Obtain the sub-word sequence that has been decoded before the current time, and map the sub-word sequence to the first word embedding feature; Decoding is performed based on the visual features and the first word embedding features to obtain the sub-words decoded at the current time. The subwords decoded at the current moment are appended to the end of the subword sequence, and the mapping of the subword sequence to the first word embedding feature is returned until the decoding is complete; The sub-word sequence obtained after decoding is used as the first recognized text.
5. The text recognition method according to claim 1, characterized in that, The process by which the text recognition decoder uses the first recognized text as a prompt and combines the visual features to perform character-by-character decoding to obtain the second recognized text includes: Map the first identified text to the second word embedding features; Obtain the character sequence that has been decoded before the current time, and map the character sequence into third-word embedding features; Decoding is performed based on the visual features, the second word embedding features, and the third word embedding features to obtain the character decoded at the current moment; The character decoded at the current moment is appended to the end of the character sequence, and the process of mapping the character sequence to the third word embedding feature is returned until the decoding is complete. The character sequence obtained after decoding is used as the second recognized text.
6. The text recognition method according to claim 1, characterized in that, The training process of the text recognition model includes: Obtain image samples and their corresponding first and second recognition text labels; The image sample is input into the text recognition model to obtain the first decoding result output by the thought chain decoder and the second decoding result output by the text recognition decoder. A first loss is generated based on the first decoding result and the first recognized text label, and a second loss is generated based on the second decoding result and the second recognized text label; A total loss is generated based on the first loss and the second loss, and the text recognition model is trained using the total loss.
7. The text recognition method according to claim 6, characterized in that, The step of generating a total loss based on the first loss and the second loss includes: The number of image samples whose first recognized text label matches the real text in the image sample is determined as a first number, and the number of image samples whose first recognized text label does not match the real text in the image sample is determined as a second number; Based on the first quantity and the second quantity, determine the weighting coefficients corresponding to the first loss and the second loss, respectively; The total loss is obtained by weighting and summing the first loss and the second loss using the weighting coefficients.
8. The text recognition method according to claim 7, characterized in that, The step of determining the weighting coefficients corresponding to the first loss and the second loss based on the first quantity and the second quantity includes: Determine the proportion of the second quantity in the total quantity as a first proportion, wherein the total quantity is the sum of the first quantity and the second quantity; The larger value between the first ratio and the preset weight threshold is selected as the weight coefficient of the first loss; Determine the proportion of the first quantity in the total quantity, as the second proportion; The larger value between the second ratio and the weight threshold is selected as the weight coefficient of the second loss.
9. The text recognition method according to claim 6, characterized in that, The process of generating the first recognized text label includes: Based on the second recognized text tag, a content correction prompt instruction is generated. The content correction prompt instruction is used to prompt the large language model to perform content correction on the second recognized text tag to obtain semantically correct and fluent text. The error correction prompt instruction is input into the large language model to obtain the first recognized text label output by the model.
10. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the text recognition method as described in any one of claims 1 to 9.
11. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the text recognition method as described in any one of claims 1 to 9.
12. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the text recognition method as described in any one of claims 1 to 9.