AI-based text location and content recognition method

Abstract

The present invention discloses an AI-based text position and content recognition method, which relates to the technical field of text recognition. The method extracts sequence features from multi-scale fusion features through a bidirectional LSTM model, aggregates them in combination with context information, introduces an attention mechanism, and outputs context features. Character prediction is performed on the context features through the connected temporal classification CTC algorithm, and the prediction probability is output. The prediction result is decoded in combination with the language model LM to obtain a decoded text sequence. Furthermore, the decoding result is corrected by an adaptive correction model, and the final recognition result is output. The results of each step are summarized, and the recognition results are intelligently decided and feedback adjusted through the comprehensive evaluation coefficient Sfinal. This series of optimization measures enables the system to dynamically adapt to different input conditions, which not only improves the recognition accuracy, but also improves the adaptability and robustness of the system.

Description

Technical Field

[0001] The present invention relates to the field of text recognition technology, and in particular to an AI-based text position and content recognition method. Background Art

[0002] With the growing development of artificial intelligence (AI) technology, computer vision has become a key area of AI application. Computer vision involves the processing and analysis of images and videos, and one of its key tasks is optical character recognition (OCR). OCR technology has a wide range of applications, particularly in logistics weighing processes. AI-based text location and content recognition methods further expand the capabilities of traditional OCR. By combining image preprocessing, multi-scale feature extraction, sequence modeling, and contextual understanding, they enable more accurate text location and content recognition on documents such as ID cards, driver's licenses, vehicle registration certificates, and vehicle manifests, as well as logistics documents, containers, train compartment markings, and surface-printed information on cargo.

[0003] In existing text recognition systems, traditional OCR technology is often affected by factors such as image quality, irregular text arrangement, and background noise, resulting in reduced recognition accuracy. Traditional methods generally perform poorly when dealing with complex scenes such as handwriting, low-contrast text, and distorted or tilted text. Most existing systems lack the ability to process multi-scale and diverse images and have difficulty coping with complex contexts, which in turn affects the overall recognition effect and the breadth of application.

[0004] These current issues and shortcomings primarily stem from the algorithmic limitations of traditional OCR technology, such as single-scale feature extraction and a lack of effective context modeling. These issues not only hinder the system's performance in real-world applications but can also lead to significant errors in recognition results, particularly when processing documents with complex backgrounds or mixed fonts. These errors significantly impact data reliability and consistency. Furthermore, in sectors with extremely high data accuracy requirements, such as industry, healthcare, and finance, low-precision text recognition can lead to information errors, low work efficiency, and even severe economic losses. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides an AI-based text position and content recognition method to solve the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for identifying text position and content based on AI, comprising the following steps:

[0007] S1. Preprocess the input image and output the preprocessed image IMGdenoise. Then, perform multi-scale normalization to obtain the standard image IMGnom. Use the spatial transformer network (STN) variant model to correct the standard image IMGnom and output the corrected image IMGcorr.

[0008] S2. Use OCR text detection technology to detect the text area from the corrected image IMGcorr, then use the multi-scale convolutional neural network MS-CNN to extract the multi-scale feature Fms from the text area of the corrected image IMGcorr, and then perform multi-scale feature fusion based on the extracted multi-scale feature Fms to obtain the multi-scale fusion feature Ffusion;

[0009] S3. Use the bidirectional LSTM model to extract sequence features from the fused multi-scale fusion feature Ffusion, aggregate them with context information, output the aggregated sequence feature Fseq, and introduce the attention mechanism to output the context feature Fcontext;

[0010] S4. Use the linked temporal classification (CTC) algorithm to predict the characters of the context feature Fcontext and output the prediction probability Pctc. At the same time, use the language model LM to decode the prediction probability Pctc and obtain the decoded text sequence Tdecoded.

[0011] S5. Process the decoded text sequence Tdecoded, adaptively adjust the recognition result based on the context, output the recognition result Tfinal, and adaptively correct the recognition result Tfinal to obtain the corrected result Tadaptive;

[0012] S6. Summarize the corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, output the evaluation function fi(Params), and perform comprehensive summary calculation to output the comprehensive evaluation coefficient Sfinal. The preset recognition threshold Threshold is compared with the comprehensive evaluation coefficient Sfinal for evaluation, and the action decision Execfinal is output.

[0013] Preferably, said S1 includes S11, S12 and S13;

[0014] S11. By collecting text image input, performing image denoising and color space conversion on the image, and outputting a pre-processed image IMGdenoise, to ensure that the text in the image can be accurately processed regardless of its tilt, curvature or other deformation. Then, multi-scale normalization processing is performed on the output pre-processed image IMGdenoise to output a standard image IMGnom;

[0015] The pre-processed image IMGdenoise and the standard image IMGnom are processed in the following manner:

[0016] IMGdenoise=Denoise(IMG)IMGcolor=ColorTransform(IMGdenoise)

[0017]

[0018] In the formula, Denoise (IMG) represents the noise reduction processing function, which is used for denoising operation, ColorTransform (IMGdenoise) represents the color space conversion function, which is used for color space conversion operation, and IMGnom (s) Represents the standard image normalized at scale s, IMGdenoise( s ) represents the preprocessed image at scale s, μIMG (s) represents the mean of the image at scale s, σIMG( s ) represents the standard deviation of the image at scale s, and IMGcolor represents the color conversion function;

[0019] S12, the output standard image IMGnom is then rectified using the spatial transformer network STN variant model, and the rectified image IMGcorr is output;

[0020] The corrected image IMGcorr is obtained by outputting the following spatial transformer network STN variant model;

[0021]

[0022] Where S represents the total number of different scales, λs represents the weight coefficient of the sth scale, represents the transformation function of the spatial transformation network variant model at scale s;

[0023] S13. Use the EAST model in OCR to obtain the bounding box of the corrected image IMGcorr through convolutional layers and multi-scale image pyramid processing. In the EAST model, branches 1 and 2 are responsible for detecting text boundaries, while branches 3 and 4 are responsible for detecting the orientation of the text bounding box. The detected area is then scaled using ROIPooling technology to ensure that the feature map maintains high resolution, reduce the impact of text size changes, and output an optimized text area.

[0024] Preferably, said S2 includes S21 and S22;

[0025] S21. Use the multi-scale convolutional neural network MS-CNN to perform convolution operations on the text area of the corrected image IMGcorr at different scales to extract features at different levels, including edge features, texture features, and shape features. At the same time, the convolved features are pooled to optimize the feature dimension while retaining image information and enhancing the model's anti-interference ability. Apply the nonlinear activation function ReLU to perform nonlinear transformation on the convolution result to extract multi-scale features Fms.

[0026] The multi-scale feature Fms is calculated and obtained by the following algorithm formula:

[0027]

[0028] Where, α s represents the weight coefficient of the sth scale, σ represents the nonlinear activation function, Represents the operation of applying a convolutional neural network CNN at scale s, which includes multiple layers of convolution, pooling, and nonlinear activation;

[0029] S22, by performing a pooling operation on the multi-scale feature Fms, optimizing the computational complexity and removing redundant information, and then weighting and summing the pooled features at different scales s to obtain the multi-scale fusion feature Ffusion;

[0030]

[0031] Where, β s represents the fusion weight coefficient of the s-th scale, and Pool represents the pooling operation.

[0032] Preferably, said S3 includes S31 and S32;

[0033] S31. Use the bidirectional LSTM in the CRNN model to extract sequence features from the multi-scale fusion feature Ffusion, and extract the context information of each time step through two LSTM layer models. One of the two LSTM layer models processes the sequence from front to back, and the other processes the sequence from back to front, and outputs the aggregated sequence feature Fseq.

[0034] The aggregate sequence feature Fseq is output by the following model;

[0035] Fseq = hBiLSTM(Ffusion);

[0036] Where hBiLSTM represents a bidirectional LSTM function, which is used to convert the input multi-scale fusion feature Ffusion into sequence features;

[0037] S32, by introducing the output context feature Fcontext of the attention mechanism, dynamically adjust the degree of attention to each time step;

[0038] The context feature Fcontext is obtained by introducing the following attention mechanism and context modeling formula;

[0039]

[0040] Where T represents the number of time steps contained in the sequence feature, t represents the time step, and γ t represents the attention weight of time step t, Attention represents the attention machine function, Fseq (t) Represents the feature vector of the aggregated sequence feature Fseq at time step t.

[0041] Preferably, said S4 includes S41 and S42;

[0042] S41, using the connection temporal classification algorithm CTC, calculate the probability of each character in each time step, perform character prediction on the context feature Fcontext, and output the prediction probability Pctc;

[0043] The predicted probability Pctc is calculated by the following algorithm formula:

[0044]

[0045] In the formula, CSeq represents the character sequence, CTC represents the connection time series classification, and Fcontext (t) represents the feature representation of the context feature vector at time step t.

[0046] Preferably, S42, combining the prediction probability Pctc output by the connection time series classification algorithm CTC with the language model LM, decoding the predicted character sequence through the n-gram model in the language model LM combined with BeamSearch, and obtaining a decoded text sequence Tdecoded;

[0047] The decoded text sequence Tdecoded is obtained by calculating the following language model LM decoding formula;

[0048]

[0049] Where N represents the number of LMs of different n-geam language models, LMdecode represents the decoding function performed by the language model LM, and δ n Represents the weight coefficient of n-gram in the language model, and n-gram represents the phonetic model LM of the next character predicted by the nth preceding character.

[0050] Preferably, said S5 includes S51 and S52;

[0051] S51, further processing the decoded text sequence Tdecoded by using an optimization method based on edit distance, including removing redundant characters and correcting recognition errors, and adaptively adjusting based on context, and outputting a recognition result Tfinal;

[0052] The recognition result Tfinal is obtained by calculating the following edit distance optimization algorithm;

[0053] Tfinal=Optimize(Tdecoded, GT, Sim);

[0054] Where Sim represents the semantic similarity function, Optimize represents the optimization function used to integrate the results of edit distance and semantic similarity, and GT represents the target text.

[0055] Preferably, S52, the obtained recognition result Tfinal is input into the adaptive correction model to adaptively correct the coherence and readability of the text in the context, and output a correction result Tadaptive;

[0056] The correction result Tadaptive is output through the following adaptive correction model;

[0057] Tadaptive=Tfinal+ε·Adapt(Tfinal,Sim);

[0058] Where Adapt represents the adaptive correction function, and ε represents the correction coefficient.

[0059] Preferably, said S6 includes S61 and S62;

[0060] S61, comprehensively summarize the output corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, and perform comprehensive calculation to output evaluation function fi(Params), wherein evaluation function fi(Params)=f{IMGcorr, Efusion, Fcontext, Tdecoded, Tadaptive}(Params), and then perform correlation calculation to output comprehensive evaluation coefficient Sfinal;

[0061] The comprehensive evaluation coefficient Sfinal is calculated and output by the following algorithm formula:

[0062]

[0063] Where η i represents the number of the i-th step, I represents the number of different processing steps, where the processing steps include image correction, feature extraction, decoding and correction, and fi(Params) represents the evaluation function of the i-th step.

[0064] Preferably, S62, the user presets the recognition threshold Threshold based on the recognition requirement, inputs the comprehensive evaluation coefficient Sfinal for comparative evaluation, and outputs the action decision Execfinal according to the evaluation result, outputs the recognition result, generates a report and automatically archives it. The specific evaluation content is as follows;

[0065] When the comprehensive evaluation coefficient Sfinal ≥ the recognition threshold Threshold, it indicates that the current text recognition is normal, the automation task is executed, and the action decision Execfinal is output;

[0066] When the comprehensive evaluation coefficient Sfinal is less than the recognition threshold Threshold, it indicates that the current text recognition is abnormal, triggering the abnormality handling process and prompting manual intervention in the recognition;

[0067] The action decision Execfinal is executed by the following algorithm:

[0068] Execfinal=Action(Sfinal,Threshold)Feedback;

[0069] Where Action represents the execution action function, which performs intelligent operations based on the evaluation results and the set recognition threshold Threshold; Feedback represents the feedback mechanism function, which is used to adjust the model parameters to optimize future recognition performance.

[0070] The present invention provides an AI-based text location and content recognition method, which has the following beneficial effects:

[0071] (1) In the image preprocessing stage, this method performs image denoising and color space conversion on the input image to generate a preprocessed image IMGdenoise, and further obtains the standard image IMGnom through multi-scale normalization. These operations ensure that the text in the image can be accurately processed regardless of its tilt, curvature or other deformation. By introducing the spatial transformer network (STN) variant model, the standard image is corrected and the corrected image IMGcorr is finally output. The optimization of these steps greatly improves the system's ability to process complex and deformed images, reduces recognition errors caused by image distortion, and thus improves recognition accuracy and stability.

[0072] (2) In the feature extraction and fusion stage, this method uses a multi-scale convolutional neural network MS-CNN to extract features from the corrected image IMGcorr at different scales to obtain features Fms at different levels, including edge, texture, and shape features. Subsequently, the feature dimension is optimized through pooling operations, and the nonlinear activation function ReLU is applied to the convolution results to perform nonlinear transformations, and finally multi-scale features are extracted. By pooling and fusing these multi-scale features, multi-scale fusion features Ffusion are obtained. This process ensures that the system can retain important information when faced with complex image features, and enhances the robustness and accuracy of recognition by fusing features at different scales.

[0073] (3) In the recognition stage, this method extracts sequence features Fseq from the multi-scale fusion features Ffusion through a bidirectional LSTM model, aggregates them in combination with contextual information, and introduces an attention mechanism to output contextual features Fcontext. Character prediction is performed on the contextual features through the connected temporal classification CTC algorithm, and the prediction probability Pctc is output. The prediction results are decoded in combination with the language model LM to obtain the decoded text sequence Tdecoded. Furthermore, the decoding results are corrected through the adaptive correction model, and the final recognition result Tadaptive is output. Finally, the results of each step are summarized, and the recognition results are intelligently decided and feedback adjusted through the comprehensive evaluation coefficient Sfinal. This series of optimization measures enables the system to dynamically adapt to different input conditions, which not only improves the recognition accuracy, but also improves the system's adaptability and robustness, thereby maintaining efficient recognition performance in various complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 Schematic diagram of the steps of the AI-based text position and content recognition method of the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0076] Example 1

[0077] See also Figure 1 The present invention provides an AI-based text position and content recognition method. To achieve the above objectives, the present invention is implemented through the following technical solutions, including the following steps:

[0078] S1. Preprocess the input image and output the preprocessed image IMGdenoise. Then, perform multi-scale normalization to obtain the standard image IMGnom. Use the spatial transformer network (STN) variant model to correct the standard image IMGnom and output the corrected image IMGcorr.

[0079] S2. Use OCR text detection technology to detect the text area from the corrected image IMGcorr, then use the multi-scale convolutional neural network MS-CNN to extract the multi-scale feature Fms from the text area of the corrected image IMGcorr, and then perform multi-scale feature fusion based on the extracted multi-scale feature Fms to obtain the multi-scale fusion feature Ffusion;

[0080] S3. Use the bidirectional LSTM model to extract sequence features from the fused multi-scale fusion feature Ffusion, aggregate them with context information, output the aggregated sequence feature Fseq, and introduce the attention mechanism to output the context feature Fcontext;

[0081] S4. Use the linked temporal classification (CTC) algorithm to predict the characters of the context feature Fcontext and output the prediction probability Pctc. At the same time, use the language model LM to decode the prediction probability Pctc and obtain the decoded text sequence Tdecoded.

[0082] S5. Process the decoded text sequence Tdecoded, adaptively adjust the recognition result based on the context, output the recognition result Tfinal, and adaptively correct the recognition result Tfinal to obtain the corrected result Tadaptive;

[0083] S6. Summarize the corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, output the evaluation function fi(Params), and perform comprehensive summary calculation to output the comprehensive evaluation coefficient Sfinal. The preset recognition threshold Threshold is compared with the comprehensive evaluation coefficient Sfinal for evaluation, and the action decision Execfinal is output.

[0084] In this embodiment, in step S1 of the method, through the application of image denoising, multi-scale normalization and spatial transformation network STN, the system effectively corrects the distortion and deformation in the input image and outputs a high-quality corrected image IMGcorr. This lays a solid foundation for subsequent feature extraction. In S2, the multi-scale convolutional neural network MS-CNN is used to extract and fuse multi-scale features Ffusion, which enhances the system's ability to capture image details, especially when processing complex backgrounds or diverse images, significantly improving the accuracy of feature extraction. In S3, the sequence features Fseq are extracted by the bidirectional LSTM model and the context features Fcontext are generated in combination with the attention mechanism. The system can better understand the contextual relationship between characters and improve the accuracy and coherence of character recognition. Subsequently, in S4 and S5, the system uses the connection temporal classification CTC algorithm and the language model LM to predict and decode the characters, and optimizes the recognition results Tadaptive through the adaptive correction mechanism, ensuring the semantic and grammatical consistency of the final output text sequence. Finally, through S6's comprehensive evaluation and intelligent decision-making, the system automatically determines the quality of the recognition results and makes the corresponding execution decision (Execfinal) based on the preset threshold, thereby achieving continuous self-optimization. Compared with traditional technologies, this method has achieved significant breakthroughs in image correction, multi-scale feature extraction, context modeling, and adaptive correction. These improvements significantly enhance the system's recognition performance when handling complex scenes, different fonts, and various text layouts, ultimately significantly improving the accuracy, stability, and intelligence of text recognition.

[0085] Example 2

[0086] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: S1 includes S11, S12 and S13;

[0087] S11. By collecting text image input, performing image denoising and color space conversion on the image, and outputting a pre-processed image IMGdenoise, to ensure that the text in the image can be accurately processed regardless of its tilt, curvature or other deformation. Then, multi-scale normalization processing is performed on the output pre-processed image IMGdenoise to output a standard image IMGnom;

[0088] The preprocessed image IMGdenoise and the standard image IMGnom are processed in the following way;

[0089] IMGdenoise=Denoise(IMG)IMGcolor=ColorTransform(IMGdenoise)

[0090]

[0091] In the formula, Denoise (IMG) represents the noise reduction processing function, which is used for denoising operation, ColorTransform (IMGdenoise) represents the color space conversion function, which is used for color space conversion operation, and IMGnom (s) Represents the standard image normalized at scale s, IMGdenoise( s ) represents the preprocessed image at scale s, μIMG (s) represents the mean of the image at scale s, σIMG( s ) represents the standard deviation of the image at scale s, and IMGcolor represents the color conversion function;

[0092] S12, the output standard image IMGnom is then rectified using the spatial transformer network STN variant model, and the rectified image IMGcorr is output;

[0093] The corrected image IMGcorr is obtained by outputting the following spatial transformer network STN variant model;

[0094]

[0095] Where S represents the total number of different scales, λs represents the weight coefficient of the sth scale, represents the transformation function of the spatial transformation network variant model at scale s;

[0096] S13. Use the EAST model in OCR to obtain the bounding box of the corrected image IMGcorr through convolutional layers and multi-scale image pyramid processing. In the EAST model, branches 1 and 2 are responsible for detecting text boundaries, while branches 3 and 4 are responsible for detecting the orientation of the text bounding box. The detected area is then scaled using ROIPooling technology to ensure that the feature map maintains high resolution, reduce the impact of text size changes, and output an optimized text area.

[0097] In this embodiment, by implementing step S1, the method achieves significant technological advancements in image preprocessing and correction. Specifically, through the image denoising and color space conversion processing in S11, the system can effectively remove noise from the image and generate a standard image IMGnom through multi-scale normalization. These operations ensure that the system can accurately recognize and process the text in the input image regardless of how it is tilted, curved, or otherwise deformed, thereby significantly improving the accuracy and stability of image processing. Then, in S12, the standard image is corrected using a spatial transformer network (STN) variant model, and a corrected image IMGcorr is output. This correction process eliminates geometric distortion in the image, making subsequent feature extraction and recognition more reliable and accurate. Compared with traditional technologies, this method has significantly improved the accuracy of image preprocessing and correction. By introducing multi-scale normalization and the STN model, the system performs better when processing complex and deformed images, greatly reducing recognition errors caused by poor image quality. Ultimately, these improvements effectively improve the overall performance of text recognition, ensure the system's adaptability and robustness in complex scenarios, and significantly improve the accuracy and consistency of recognition results.

[0098] Example 3

[0099] This embodiment is explained in Example 2, please refer to Figure 1 , specifically: S2 includes S21 and S22;

[0100] S21. Use the multi-scale convolutional neural network MS-CNN to perform convolution operations on the text area of the corrected image IMGcorr at different scales to extract features at different levels, including edge features, texture features, and shape features. At the same time, the convolved features are pooled to optimize the feature dimension while retaining image information and enhancing the model's anti-interference ability. Apply the nonlinear activation function ReLU to perform nonlinear transformation on the convolution result to extract multi-scale features Fms.

[0101] The multi-scale feature Fms is calculated using the following algorithm formula:

[0102]

[0103] Where, α s represents the weight coefficient of the sth scale, σ represents the nonlinear activation function, Represents the operation of applying a convolutional neural network CNN at scale s, which includes multiple layers of convolution, pooling, and nonlinear activation;

[0104] S22, by performing a pooling operation on the multi-scale feature Fms, optimizing the computational complexity and removing redundant information, and then weighting and summing the pooled features at different scales s to obtain the multi-scale fusion feature Ffusion;

[0105]

[0106] Where, β s represents the fusion weight coefficient of the s-th scale, and Pool represents the pooling operation.

[0107] In this embodiment, the method achieves significant technical improvements in feature extraction and fusion by implementing step S2. Specifically, in S21, a multi-scale convolutional neural network MS-CNN is used to perform a multi-scale convolution operation on the corrected image IMGcorr. The system can extract multi-level image features from different scales, including edge, texture, and shape features. The extraction of these features not only optimizes the feature dimension and reduces the computational complexity through convolution and pooling operations, but also further enhances the model's adaptability and anti-interference ability to complex images by applying nonlinear activation functions such as ReLU, and ultimately generates multi-scale features Fms. In S22, by performing a pooling operation on these multi-scale features, redundant information is removed, and a multi-scale fusion feature Ffusion is obtained after weighted summation. The system effectively fuses image information at different scales, making the feature representation more comprehensive and robust. This method has significantly improved the accuracy of feature extraction and the efficiency of feature fusion. By introducing MS-CNN and a multi-scale fusion strategy, the system can more accurately capture the detailed information in the image and improve the anti-interference and recognition accuracy of the model through multi-level fusion of features. These improvements enable the system to perform better when processing complex backgrounds and diverse images, ultimately improving the overall effect of text recognition and ensuring the stability and reliability of the recognition results.

[0108] Example 4

[0109] This embodiment is explained in Example 3, please refer to Figure 1 ,Specifically: S3 includes S31 and S32;

[0110] S31. Use the bidirectional LSTM in the CRNN model to extract sequence features from the multi-scale fusion feature Ffusion. Use two LSTM layer models to extract the context information of each time step. One LSTM layer model processes the sequence from front to back, and the other processes the sequence from back to front, and outputs the aggregated sequence feature Fseq.

[0111] The aggregate sequence feature Fseq is output by the following model;

[0112] Fseq = hBiLSTM(Ffusion);

[0113] Where hBiLSTM represents a bidirectional LSTM function, which is used to convert the input multi-scale fusion feature Ffusion into sequence features;

[0114] S32, by introducing the output context feature Fcontext of the attention mechanism, dynamically adjust the degree of attention to each time step;

[0115] The context feature Fcontext is obtained by introducing the following attention mechanism and context modeling formula;

[0116]

[0117] Where T represents the number of time steps contained in the sequence feature, t represents the time step, and γ t represents the attention weight of time step t, Attention represents the attention machine function, Fseq (t) Represents the feature vector of the aggregated sequence feature Fseq at time step t.

[0118] In this embodiment, by implementing step S3, the method achieves significant technological progress in sequence feature extraction and context modeling. In S31, a bidirectional LSTM model is used to extract sequence features from the multi-scale fusion feature Ffusion. The system can simultaneously capture the contextual information of the character sequence from both the forward and backward directions to generate an aggregated sequence feature Fseq. This bidirectional processing method significantly improves the system's ability to understand the dependencies between characters, especially when processing long texts or complex character arrangements, and can better retain contextual information. In S32, an attention mechanism is introduced, allowing the system to dynamically adjust the degree of attention to different time steps and output the contextual feature Fcontext. In this way, the system can focus on the features that are most important to the recognition results, further improving the accuracy and reliability of recognition. This method has obvious advantages in sequence modeling and context understanding. By combining bidirectional LSTM and the attention mechanism, the system can not only more accurately capture the complex relationships between characters, but also optimize the recognition effect by focusing on important features. These improvements enable the system to perform better when processing complex texts, ultimately significantly improving the accuracy, coherence and context consistency of text recognition.

[0119] Example 5

[0120] This embodiment is explained in Example 4. Please refer to Figure 1 , specifically: S4 includes S41 and S42;

[0121] S41, using the connection temporal classification algorithm CTC, calculate the probability of each character in each time step, perform character prediction on the context feature Fcontext, and output the prediction probability Pctc;

[0122] The predicted probability Pctc is calculated using the following algorithm formula;

[0123]

[0124] Where CSeq represents the character sequence, CTC represents the connection time series classification, and Fcontext( t ) represents the feature representation of the context feature vector at time step t.

[0125] S42. Combine the prediction probability Pctc output by the connection time series classification algorithm CTC with the language model LM, and decode the predicted character sequence using the n-gram model in the language model LM in combination with BeamSearch to obtain a decoded text sequence Tdecoded.

[0126] The decoded text sequence Tdecoded is obtained by calculating the following language model LM decoding formula;

[0127]

[0128] Where N represents the number of LMs of different n-geam language models, LMdecode represents the decoding function performed by the language model LM, and δ n Represents the weight coefficient of n-gram in the language model, and n-gram represents the phonetic model LM of the next character predicted by the nth preceding character.

[0129] In this embodiment, the method achieves significant technical improvements in character prediction and decoding by implementing step S4. In S41, the connected temporal classification CTC algorithm is adopted, and the system can calculate the probability of each character in each time step without explicit alignment, perform character prediction on the context feature Fcontext, and output the predicted probability Pctc. The introduction of the CTC algorithm solves the problem of inconsistency between the length of the character sequence and the length of the input sequence, so that the system can flexibly process input data of different lengths and improve the accuracy of character prediction. In S42, by combining the predicted probability Pctc output by CTC with the language model LM, especially using the n-gram model to decode the character sequence, the system can further optimize the decoding process and generate a more accurate and coherent decoded text sequence Tdecoded.

[0130] Example 6

[0131] This embodiment is explained in Example 5, please refer to Figure 1 ,Specifically: S5 includes S51 and S52;

[0132] S51, further processing the decoded text sequence Tdecoded by using an optimization method based on edit distance, including removing redundant characters and correcting recognition errors, and adaptively adjusting based on context, and outputting a recognition result Tfinal;

[0133] The recognition result Tfinal is obtained by calculating the following edit distance optimization algorithm;

[0134] Tfinal=Optimize(Tdecoded, GT, Sim);

[0135] Where Sim represents the semantic similarity function, Optimize represents the optimization function used to integrate the results of edit distance and semantic similarity, and GT represents the target text.

[0136] S52, inputting the obtained recognition result Tfinal into the adaptive correction model to adaptively correct the coherence and readability of the text in the context, and outputting the correction result Tadaptive;

[0137] The correction result Tadaptive is output through the following adaptive correction model;

[0138] Tadaptive=Tfinal+ε·Adapt(Tfinal,Sim);

[0139] Where Adapt represents the adaptive correction function, and ε represents the correction coefficient.

[0140] In this embodiment, the method achieves significant technical improvements in the optimization and correction of recognition results by implementing step S5. In S51, the system further processes the decoded text sequence Tdecoded through an optimization method based on edit distance, removes redundant characters and corrects recognition errors. At the same time, the recognition result is adaptively adjusted in combination with the semantic similarity function Sim, and a more accurate recognition result Tfinal is output. This process ensures the accuracy of the recognition results at the character level and the semantic level, enabling the system to better cope with complex text recognition tasks. In S52, by inputting the recognition result Tfinal into the adaptive correction model, the system further optimizes the coherence and readability of the text in the context, and finally outputs the correction result Tadaptive. This adaptive correction method makes the recognition results more natural and in line with language habits. The method has significantly improved the accuracy of the recognition results and the coherence of the text. By combining the optimization method of edit distance and semantic similarity, as well as the adaptive correction model, the system not only greatly reduces recognition errors, but also significantly improves the readability and semantic consistency of the final output text. These improvements ensure that the system can generate more accurate, consistent, and natural recognition results when processing complex text and diverse inputs, ultimately improving the overall performance and application value of text recognition.

[0141] Example 7

[0142] This embodiment is explained in Example 6, please refer to Figure 1 , specifically: S6 includes S61 and S62;

[0143] S61, comprehensively summarize the output corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, and perform comprehensive calculation to output evaluation function fi(Params), evaluation function fi(Params)=f{IMGcorr, Efusion, Fcontext, Tdecoded, Tadaptive}(Params), and then perform correlation calculation to output comprehensive evaluation coefficient Sfinal;

[0144] The comprehensive evaluation coefficient Sfinal is calculated and output by the following algorithm formula;

[0145]

[0146] Where η i represents the number of the i-th step, I represents the number of different processing steps, where the processing steps include image correction, feature extraction, decoding and correction, and fi(Params) represents the evaluation function of the i-th step.

[0147] S62: The user presets the recognition threshold Threshold based on the recognition requirements, inputs the comprehensive evaluation coefficient Sfinal for comparative evaluation, and outputs the action decision Execfinal based on the evaluation result, outputs the recognition result, generates a report and automatically archives it. The specific evaluation content is as follows;

[0148] When the comprehensive evaluation coefficient Sfinal ≥ the recognition threshold Threshold, it indicates that the current text recognition is normal, the automation task is executed, and the action decision Execfinal is output;

[0149] When the comprehensive evaluation coefficient Sfinal is less than the recognition threshold Threshold, it indicates that the current text recognition is abnormal, triggering the abnormality handling process and prompting manual intervention in the recognition;

[0150] Action decision Execfinal is executed through the following algorithm;

[0151] Execfinal=Action(Sfinal,Threshold)Feedback;

[0152] Where Action represents the execution action function, which performs intelligent operations based on the evaluation results and the set recognition threshold Threshold; Feedback represents the feedback mechanism function, which is used to adjust the model parameters to optimize future recognition performance.

[0153] In this embodiment, the method achieves systematic improvements in comprehensive evaluation and intelligent decision-making by implementing step S6. In S61, the system comprehensively summarizes the corrected image IMGcorr, multi-scale fusion feature Efusion, contextual feature Fcontext, decoded text sequence Tdecoded, and correction result Tadaptive output from the previous steps, and quantitatively evaluates the effectiveness of each step by calculating the evaluation function fi(Params), ultimately outputting a comprehensive evaluation coefficient Sfinal. This process ensures that the system comprehensively evaluates the contribution of each processing step as a whole, ensuring the quality and reliability of the final recognition result. In S62, by comparing and evaluating the comprehensive evaluation coefficient Sfinal with the preset recognition threshold Threshold, the system can make automated decisions (Execfinal) based on actual needs, such as outputting recognition results, generating reports, or automatically archiving them. If the recognition results do not meet the requirements, the system can also trigger an exception handling process and prompt manual intervention, thereby ensuring the flexibility and security of the recognition process. This method has significantly improved the intelligent level of evaluation and decision-making. Through comprehensive summarization and precise quantitative evaluation, the system can fully and accurately grasp the contribution and effect of each processing step. Through intelligent decision-making and feedback mechanisms, the system achieves automated operation and continuous performance optimization. These improvements not only significantly improve the system's efficiency and recognition accuracy, but also enhance its adaptability and reliability, enabling it to maintain efficient and stable performance in various application scenarios.

[0154] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. AI-based text location and content recognition method, characterized by: The following steps are involved: S1. Preprocess the input image and output the preprocessed image IMGdenoise. Then, perform multi-scale normalization to obtain the standard image IMGnom. Use the spatial transformer network (STN) variant model to correct the standard image IMGnom and output the corrected image IMGcorr. S2. Use OCR text detection technology to detect the text area from the corrected image IMGcorr, then use the multi-scale convolutional neural network MS-CNN to extract the multi-scale feature Fms from the text area of the corrected image IMGcorr, and then perform multi-scale feature fusion based on the extracted multi-scale feature Fms to obtain the multi-scale fusion feature Ffusion; S3. Use the bidirectional LSTM model to extract sequence features from the fused multi-scale fusion feature Ffusion, aggregate them with context information, output the aggregated sequence feature Fseq, and introduce the attention mechanism to output the context feature Fcontext; S4. Use the connection temporal classification (CTC) algorithm to predict the characters of the context feature Fcontext and output the prediction probability Pctc. At the same time, use the language model LM to decode the prediction probability Pctc and obtain the decoded text sequence Tdecoded. S5. Process the decoded text sequence Tdecoded, adaptively adjust the recognition result based on the context, output the recognition result Tfinal, and adaptively correct the recognition result Tfinal to obtain the corrected result Tadaptive; S6. Summarize the corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, output the evaluation function fi (Params), and perform comprehensive summary calculation to output the comprehensive evaluation coefficient Sfinal. The preset recognition threshold Threshold is compared with the comprehensive evaluation coefficient Sfinal for evaluation, and the action decision Execfinal is output.

2. The AI-based text position and content recognition method according to claim 1, characterized in that: Said S1 includes S11, S12 and S13; S11, by collecting text image input, performing image denoising and color space conversion processing on the image, outputting a pre-processed image IMGdenoise, and then performing multi-scale normalization processing on the pre-processed image IMGdenoise, outputting a standard image IMGnom; The pre-processed image IMGdenoise and the standard image IMGnom are processed in the following manner: ； ； In the formula, Denoise (IMG) represents the noise reduction processing function, which is used for denoising operation, and ColorTransform (IMGdenoise) represents the color space conversion function, which is used for color space conversion operation. represents the standard image normalized at scale s, represents the preprocessed image at scale s, represents the mean of the image at scale s, Represents the standard deviation of the image at scale s, and IMGcolor represents the color conversion function; S12, the output standard image IMGnom is then rectified using the spatial transformer network STN variant model, and the rectified image IMGcorr is output; The corrected image IMGcorr is obtained by outputting the following spatial transformer network STN variant model; ； Where S represents the total number of different scales, represents the weight coefficient of the s-th scale, represents the transformation function of the spatial transformation network variant model at scale s; S13. Use the EAST model in OCR to obtain the bounding box of the corrected image IMGcorr through convolutional layers and multi-scale image pyramid processing. In the EAST model, branches 1 and 2 are responsible for detecting text boundaries, while branches 3 and 4 are responsible for detecting the orientation of the text bounding box. The detected area is then scaled using ROI Pooling technology to ensure that the feature map maintains high resolution, reduce the impact of text size changes, and output an optimized text area.

3. The AI-based text position and content recognition method according to claim 1, characterized in that: Said S2 includes S21 and S22; S21. Use the multi-scale convolutional neural network MS-CNN to perform convolution operations on the text area of the corrected image IMGcorr at different scales to extract features at different levels, including edge features, texture features, and shape features. At the same time, the convolved features are pooled to optimize the feature dimension while retaining the image information. The nonlinear activation function ReLU is applied to perform nonlinear transformation on the convolution result to extract the multi-scale feature Fms. The multi-scale feature Fms is calculated and obtained by the following algorithm formula: ； Where, represents the weight coefficient of the s-th scale, represents a nonlinear activation function, Represents the operation of applying a convolutional neural network CNN at scale s, which includes multiple layers of convolution, pooling, and nonlinear activation; S22, by performing a pooling operation on the multi-scale feature Fms, optimizing the computational complexity and removing redundant information, and then weighting and summing the pooled features at different scales s to obtain the multi-scale fusion feature Ffusion; ； Where, represents the fusion weight coefficient of the s-th scale, Represents a pooling operation.

4. The AI-based text position and content recognition method according to claim 1, characterized in that: Said S3 includes S31 and S32; S31. Use the bidirectional LSTM in the CRNN model to extract sequence features from the multi-scale fusion feature Ffusion, and extract the context information of each time step through two LSTM layer models. One of the two LSTM layer models processes the sequence from front to back, and the other processes the sequence from back to front, and outputs the aggregated sequence feature Fseq. The aggregate sequence feature Fseq is output by the following model; ； Where hBiLSTM represents a bidirectional LSTM function, which is used to convert the input multi-scale fusion feature Ffusion into sequence features; S32, by introducing the attention mechanism to output the context feature Fcontext, dynamically adjust the degree of attention to each time step; The context feature Fcontext is obtained by introducing the following attention mechanism and context modeling formula; ； In the formula, T represents the number of time steps contained in the sequence feature, t represents the time step, represents the attention weight of time step t, Attention represents the attention machine function, Represents the feature vector of the aggregated sequence feature Fseq at time step t.

5. The AI-based text position and content recognition method according to claim 4, characterized in that: Said S4 includes S41 and S42; S41, using the connection temporal classification algorithm CTC, calculate the probability of each character in each time step, perform character prediction on the context feature Fcontext, and output the prediction probability Pctc; The predicted probability Pctc is calculated by the following algorithm formula: ； In the formula, CSeq represents the character sequence, CTC represents the connection time classification, represents the feature representation of the context feature vector at time step t.

6. The AI-based text position and content recognition method according to claim 5, characterized in that: S42. Combine the prediction probability Pctc output by the connection time series classification algorithm CTC with the language model LM, and decode the predicted character sequence using the n-gram model in the language model LM in combination with BeamSearch to obtain a decoded text sequence Tdecoded. The decoded text sequence Tdecoded is obtained by calculating the following language model LM decoding formula; ； Where N represents the number of LMs of different n-geam language models, LMdecode represents the decoding function performed by the language model LM, Represents the weight coefficient of n-gram in the language model, and n-gram represents the phonetic model LM of the next character predicted by the nth preceding character.

7. The AI-based text position and content recognition method according to claim 1, characterized in that: Said S5 includes S51 and S52; S51, further processing the decoded text sequence Tdecoded by using an optimization method based on edit distance, including removing redundant characters and correcting recognition errors, and adaptively adjusting based on context, and outputting a recognition result Tfinal; The recognition result Tfinal is obtained by calculating the following edit distance optimization algorithm; ； Where Sim represents the semantic similarity function, Optimize represents the optimization function used to integrate the results of edit distance and semantic similarity, and GT represents the target text.

8. The AI-based text position and content recognition method according to claim 7, characterized in that: S52, inputting the obtained recognition result Tfinal into the adaptive correction model to adaptively correct the coherence and readability of the text in the context, and outputting the correction result Tadaptive; The correction result Tadaptive is output through the following adaptive correction model; ； Where Adapt represents the adaptive correction function, Indicates the correction factor.

9. The AI-based text position and content recognition method according to claim 1, characterized in that: Said S6 includes S61 and S62; S61, comprehensively summarize the output corrected image IMGcorr, multi-scale fusion feature Efusion, context feature Fcontext, decoded text sequence Tdecoded and correction result Tadaptive, and perform comprehensive calculation to output an evaluation function fi(Params), wherein the evaluation function fi(Params)=f{IMGcorr, Efusion, Fcontext, Tdecoded, Tadaptive}(Params), and then perform correlation calculation to output a comprehensive evaluation coefficient Sfinal; The comprehensive evaluation coefficient Sfinal is calculated and output by the following algorithm formula: ； Where, represents the number of the i-th step, I represents the number of different processing steps, where the processing steps include image correction, feature extraction, decoding and correction, and fi(Params) represents the evaluation function of the i-th step.

10. The AI-based text position and content recognition method according to claim 9, characterized in that: S62: The user presets the recognition threshold Threshold based on the recognition requirements, inputs the comprehensive evaluation coefficient Sfinal for comparative evaluation, and outputs the action decision Execfinal based on the evaluation result, outputs the recognition result, generates a report and automatically archives it. The specific evaluation content is as follows; When the comprehensive evaluation coefficient Sfinal ≥ the recognition threshold Threshold, it indicates that the current text recognition is normal, the automation task is executed, and the action decision Execfinal is output; When the comprehensive evaluation coefficient Sfinal is less than the recognition threshold Threshold, it indicates that the current text recognition is abnormal, triggering the abnormality handling process and prompting manual intervention in the recognition; The action decision Execfinal is executed by the following algorithm: ； Where Action represents the execution action function, which performs intelligent operations based on the evaluation results and the set recognition threshold Threshold; Feedback represents the feedback mechanism function, which is used to adjust the model parameters to optimize future recognition performance.

Images