Artificial intelligence-based character recognition and error correction methods, devices, equipment, and media

Abstract

This invention relates to the field of financial technology, and more particularly to a character recognition and error correction method, apparatus, device, and medium based on artificial intelligence. The method is applied in the financial field. In this application, the position of the character to be corrected is determined through a character recognition model. A first predicted character and its probability value are identified based on the character recognition model. A preset language model is used to identify a second predicted character and its probability value. The character corresponding to the maximum value of the average of the character probability values ​​predicted by the character recognition model and the preset language model is used as the replacement character. Based on prior knowledge of the character to be corrected, the error correction position is determined for end-to-end error correction processing, thereby improving the error correction accuracy.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a character recognition and error correction method, apparatus, device, and medium based on artificial intelligence. Background Technology

[0002] With the development of internet finance, online financial services (such as payments and wealth management) are gaining increasingly widespread application demand. For individual consumers, the security and convenience of online financial transactions directly affect their user experience. The combination of bank card number and password, due to its high security, has become a widely accepted online transaction method. However, bank card numbers are often quite long, and the background of the card is often complex. Manually entering the bank card number requires users to repeatedly identify and correct it. Therefore, automatically recognizing the bank card number from the image using machine learning has become an effective means to improve the convenience of this online transaction method and has gradually attracted attention. Generally, this involves locating the bank card number line, identifying each character within that line, and finally using optical character recognition (OCR) technology to recognize each character, thus achieving the purpose of bank card recognition. However, due to the inconsistent quality of bank card images uploaded by users, problems such as reflections, blurry photos, camera shake, and diverse backgrounds can interfere with card number recognition, leading to erroneous characters in the recognition results. Therefore, it is necessary to correct these erroneous characters in the OCR recognition results.

[0003] In existing technologies, error correction tasks are generally based on multimodal features. For example, when recalling candidate characters, similar characters are found by using some publicly available dictionaries of similar-looking and near-homophone characters. However, some similar-looking and near-homophone characters that are not publicly available cannot be corrected, resulting in low error correction accuracy. Therefore, how to improve error correction accuracy has become an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide an artificial intelligence-based character recognition and error correction method, device, equipment, and medium to address the aforementioned technical problems and solve the problem of low error correction accuracy.

[0005] A first aspect of this application provides a character recognition error correction method based on artificial intelligence, the character recognition error correction method comprising:

[0006] Obtain the text block to be identified, perform character recognition on the text block to be identified, and obtain the recognition result of each character in the text block to be identified, as well as the N first predicted characters and the prediction probability value of each first predicted character at each time in the corresponding time sequence, where N is an integer greater than 1;

[0007] For any character, the maximum predicted probability value of each first predicted character in the corresponding time sequence is selected as the target probability value of the corresponding first predicted character, and N target probability values ​​corresponding to N first predicted characters are obtained. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained.

[0008] The text block to be identified is identified and predicted using a preset language model, and the M second predicted characters and the prediction probability value of each second predicted character at each character position in the text block to be identified are obtained. Based on the position of the character to be corrected, the M second predicted characters corresponding to the position of the character to be corrected and the N first predicted characters corresponding to the character to be corrected are determined as a candidate character set, where M is an integer greater than 1.

[0009] Calculate the mean of the predicted probability values ​​corresponding to the same character in the candidate character set. If the maximum value of the mean is greater than a second preset threshold, use the character corresponding to the maximum value of the mean as the replacement character and use the replacement character to replace the character to be corrected to obtain the correction result corresponding to the character to be corrected.

[0010] A second aspect of this application provides an artificial intelligence-based character recognition and error correction device, the character recognition and error correction device comprising:

[0011] The acquisition module is used to acquire the text block to be identified, perform character recognition on the text block to be identified using a character recognition model, obtain the recognition result of each character in the text block to be identified, and the N first predicted characters and the prediction probability value of each first predicted character at each time point in the corresponding time series, where N is an integer greater than 1;

[0012] The selection module is used to select the maximum predicted probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character for any character, and obtain N target probability values ​​corresponding to N first predicted characters. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained.

[0013] The prediction module is used to perform recognition and prediction on the text block to be recognized using a preset language model, to obtain M second predicted characters at each character position in the text block to be recognized and the prediction probability value of each second predicted character, and to determine the M second predicted characters corresponding to the character position to be corrected and the N first predicted characters corresponding to the character to be corrected as a candidate character set, where M is an integer greater than 1.

[0014] The calculation module is used to calculate the mean of the predicted probability values ​​corresponding to the same characters in the candidate character set. If the maximum value of the mean is greater than a second preset threshold, the character corresponding to the maximum value of the mean is used as a replacement character, and the replacement character is used to replace the character to be corrected to obtain the correction result corresponding to the character to be corrected.

[0015] Thirdly, embodiments of the present invention provide a terminal device, the terminal device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the character recognition and error correction method based on artificial intelligence as described in the first aspect.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the character recognition and error correction method based on artificial intelligence as described in the first aspect.

[0017] The advantages of this invention compared to the prior art are:

[0018] Obtain the text block to be recognized, and use a character recognition model to recognize characters in the text block to obtain the recognition result of each character in the text block, as well as N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time sequence, where N is an integer greater than 1. For any character, select the maximum prediction probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character, and obtain N target probability values ​​corresponding to N first predicted characters. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained. The method uses a preset language model to predict and identify the text block to be recognized, obtaining M second predicted characters and their corresponding predicted probability values ​​for each character position in the text block. Based on the position of the character to be corrected, the M second predicted characters and N first predicted characters corresponding to the position of the character to be corrected are determined as a candidate character set, where M is an integer greater than 1. The mean of the predicted probability values ​​corresponding to the same characters in the candidate character set is calculated. The character with the maximum mean value greater than a second preset threshold is used as the replacement character. The replacement character is used to replace the character to be corrected, thus obtaining the correction result corresponding to the character to be corrected. In this application, the position of the character to be corrected is determined by the character recognition model, and the character corresponding to the maximum value of the average of the character probability values ​​predicted by the character recognition model and the preset language model is used as the replacement character. Based on the prior knowledge of the character to be corrected, the correction position is determined to perform end-to-end error correction processing, thereby improving the error correction accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for a character recognition and error correction method based on artificial intelligence provided in an embodiment of the present invention;

[0021] Figure 2 This is a flowchart illustrating a character recognition and error correction method based on artificial intelligence, provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of a character recognition and error correction device based on artificial intelligence according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0026] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0028] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0031] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0032] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0033] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0034] An embodiment of the present invention provides a character recognition and error correction method based on artificial intelligence, which can be applied to, for example... Figure 1In this application environment, the client communicates with the server. Clients include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0035] See Figure 2 This is a flowchart illustrating an artificial intelligence-based character recognition and error correction method according to an embodiment of the present invention. The aforementioned artificial intelligence-based character recognition and error correction method can be applied to... Figure 1 The server in the above-mentioned configuration connects to the corresponding client, such as... Figure 2 As shown, the AI-based character recognition and error correction method may include the following steps.

[0036] S201: Obtain the text block to be recognized, perform character recognition on the text block to be recognized, obtain the recognition result of each character in the text block to be recognized, and the N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time sequence.

[0037] In step S201, the text block to be identified is an image block after being segmented in the image. Character recognition is performed on the text block to be identified, and the N first predicted characters and the prediction probability value of each first predicted character at each time in the corresponding time sequence are identified, so as to obtain the corresponding recognition result based on the corresponding first predicted character and the prediction probability value of each first predicted character.

[0038] In this embodiment, text boxes in the image to be detected are detected and cropped to obtain text blocks to be recognized. The image to be detected can be a scanned bank card image. An OCR character recognition model based on CTC (greedy algorithm) loss is used to recognize characters in the text blocks to be recognized, obtain the CTC decoded character sequence and the probability value of each character in the CTC decoded character sequence, and obtain the N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time sequence.

[0039] For example, if the detected text block to be identified is "leaf award text", the recognition result corresponding to "leaf" obtained by decoding through the CTC decoder includes multiple characters, such as "leaf", "ten", and "what". The prediction of "leaf" includes the predicted characters at three time points. The probability values ​​of the recognition results at the first time point are 0.96 for "leaf", 0.23 for "ten", and 0.11 for "what". The probability values ​​of the recognition results at the second time point are 0.9 for "leaf", 0.2 for "ten", and 0.17 for "what". The probability values ​​of the recognition results at the third time point are 0.8 for "leaf", 0.3 for "ten", and 0.1 for "what".

[0040] Optionally, character recognition is performed on the text block to be recognized to obtain the recognition result for each character in the text block, including:

[0041] Image features are extracted from the text block to be recognized to obtain the corresponding image features;

[0042] The image features are predicted to obtain the corresponding predicted characters in the text block to be recognized. Adjacent identical predicted characters are merged to obtain the recognition result for each character in the text block to be recognized.

[0043] In this embodiment, a convolutional neural network is used to extract image features from the text block to be recognized, and the corresponding image features are obtained. A simplified deep convolutional neural network model based on VGGNet16 is selected, using the 3*3 convolutional kernel size in VGGNet16. The simplified network layer includes 8 Conv convolutional layers, 8 ReLU activation layers, and 3 max pooling layers. The reduced number of convolutional kernels are 16, 32, 64, and 128, respectively. Then, image features are extracted from the text block to be recognized to obtain a sequence feature map x = {x1, x2, ..., xT} containing temporal information, where each column xT is a time series, and t starts from t = 1, i.e., 1 ≤ t ≤ T. m and T refer to the height and width of the text block to be recognized, respectively. A bidirectional long short-short memory network is used, with a stacked structure of three layers of bidirectional long short-short memory networks. The sequence feature map x = {x1, x2, ..., xT} is taken as input, and a softmax classifier is connected after each time slice of the bidirectional long short-short memory network to obtain the output y = {y1, y2, ..., yT}, where y is a posterior probability matrix, and each column yt of y is n, representing the length of the text to be recognized.

[0044] It should be noted that in this embodiment, the number of bidirectional long short-term memory network hidden units (i.e., the number of hidden layer nodes) can be increased, so as to further improve the recognition of the bidirectional long short-term memory network in text recognition applications, which in turn helps to improve the character recognition performance and the recognition accuracy is higher. There is a Projection network in the bidirectional long short-term memory network, and the Projection network is used to reduce the number of parameters. In the specific implementation process, due to the increase in the number of hidden units in the bidirectional long short-term memory network, the number of parameters and the amount of calculation increase, resulting in an increase in the amount of calculation and slowing down the calculation speed. In response to this, a Projection network (i.e., introducing a Projection operation) is set inside the bidirectional long short-term memory network, which can reduce the number of parameters, thereby reducing the model calculation amount, improving the calculation speed, and further improving the text recognition speed.

[0045] Optionally, performing character recognition on the text block to be recognized, and obtaining N first predicted characters of each character at each moment in the corresponding time sequence and the predicted probability value of each first predicted character, including:

[0046] Performing character recognition on the text block to be recognized, and obtaining the initial predicted character of each character in the text block to be recognized at each moment in the corresponding time sequence and the predicted probability value corresponding to each initial predicted character;

[0047] Sorting the predicted probability values from largest to smallest, and selecting the initial predicted characters with the first N predicted probability values as the first predicted characters, to obtain N first predicted characters and the predicted probability value of each first predicted character.

[0048] In this embodiment, the initial predicted characters of each character at each moment decoded by the CTC decoder are multiple. To improve the calculation efficiency, N corresponding predicted characters are selected. When selecting, they are selected according to the predicted probability values in the initial predicted characters. Sort the predicted probability values from largest to smallest, and select the initial predicted characters with the first N predicted probability values as the first predicted characters, to obtain N first predicted characters and the predicted probability value of each first predicted character. For example, for the prediction of "叶", the initial predicted characters are characters such as 叶, 十, and 什. The probability value of "叶" at the first moment is 0.96, the probability value of "十" is 0.23, and the probability value of "什" is 0.11. If 2 first predicted characters are selected, then select the first 2 characters corresponding to the predicted probability values as 十 and 什.

[0049] S202: For any character, select the maximum value of the predicted probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character, to obtain N target probability values corresponding to N first predicted characters. If the maximum value of the N target probability values is less than the first preset threshold, determine the character as a character to be corrected, and obtain the position of the character to be corrected.

[0050] In step S202, when any character corresponds to multiple prediction times, each prediction time corresponds to the prediction probability values of N first predicted characters. The maximum value of the prediction probability values of each first predicted character in the corresponding time sequence is selected as the target probability value of the corresponding first predicted character, and N target probability values corresponding to N first predicted characters are obtained. Whether the character is an error-prone character is determined according to the N target probability values, so as to determine the position of the error-prone character, so as to perform error correction processing on the error position in the subsequent error correction.

[0051] In this embodiment, for any character, the maximum value of the prediction probability values of each first predicted character in the corresponding time sequence is selected as the target probability value of the corresponding first predicted character. For example, the detected text block to be recognized is the text of the leaf award, and the recognition results corresponding to "leaf" decoded by the CTC decoder include multiple characters, such as, leaf, ten, and shi. The prediction of "leaf" includes the predicted characters at 3 times, and the obtained recognition results are that the probability value of "leaf" at the first time is 0.96, the probability value of "ten" is 0.23, and the probability value of "shi" is 0.11. The recognition result is that the probability value of "leaf" at the second time is 0.9, the probability value of "ten" is 0.2, and the probability value of "shi" is 0.17. The recognition result is that the probability value of "leaf" at the third time is 0.8, the probability value of "ten" is 0.3, and the probability value of "shi" is 0.1. When the maximum value of the prediction probability values of each first predicted character in the corresponding time sequence is selected as the target probability value of the corresponding first predicted character, the maximum value of the prediction probability value corresponding to "leaf" is 0.96, and 0.96 is used as the target probability value of the first predicted character of "leaf". The maximum value of the prediction probability value corresponding to "ten" is 0.3, and 0.3 is used as the target probability value of the first predicted character of "ten". The maximum value of the prediction probability value corresponding to "shi" is 0.17, and 0.17 is used as the target probability value of the first predicted character of "shi". For any character, including N first predicted characters, N corresponding target probability values are obtained.

[0052] After obtaining N target probability values, it is judged whether the maximum value of the N target probability values is less than the first preset threshold. If the maximum value of the N target probability values is less than the first preset threshold, it is determined that the character is an error-prone character, and the position of the error-prone character is obtained.

[0053] In this embodiment, whether the character is a misrecognized character is judged by judging the maximum value of the prediction probability values, and the position of the misrecognized character is determined.

[0054] S203: Use a preset language model to perform recognition prediction on the text block to be recognized, obtain M second predicted characters at each character position in the text block to be recognized and the prediction probability values of each second predicted character, and determine, according to the position of the character to be corrected, the M second predicted characters corresponding to the position of the character to be corrected and the N first predicted characters corresponding to the character to be corrected as the candidate character set.

[0055] In step S203, use a preset language model to perform recognition prediction on the text block to be recognized, obtain M second predicted characters at each character position in the text block to be recognized and the prediction probability values of each second predicted character, and obtain the recognition result of the preset language model for the text block to be recognized, so as to perform error correction processing on the character to be corrected according to the second predicted characters of the recognition result. M is an integer greater than 1.

[0056] In this embodiment, the preset language model is a convolutional neural network model that can recognize the text block to be recognized. Use the preset language model to perform recognition prediction on the text block to be recognized, and obtain M second predicted characters at each character position in the text block to be recognized and the prediction probability values of each second predicted character. For example, the recognition results corresponding to "叶" obtained through the preset language model include multiple characters, such as 叶, 十, and 什. The prediction of "叶" includes the predicted characters at 3 moments. The obtained recognition result is that the probability value of "叶" at the first moment is 0.97, the probability value of "十" is 0.22, and the probability value of "什" is 0.12. The recognition result is that the probability value of "叶" at the second moment is 0.91, the probability value of "十" is 0.22, and the probability value of "什" is 0.18. The recognition result is that the probability value of "叶" at the third moment is 0.82, the probability value of "十" is 0.31, and the probability value of "什" is 0.11.

[0057] According to the position of the character to be corrected, determine the M second predicted characters corresponding to the position of the character to be corrected and the N first predicted characters corresponding to the character to be corrected as the candidate character set. For example, if the recognition result of "叶" is an incorrect character, "叶" is the character to be corrected, and the N first predicted characters corresponding to "叶" and the M second predicted characters corresponding to the position of the character to be corrected are combined as the candidate character set, so as to select a suitable character from the candidate character set to replace the original predicted character "叶".

[0058] Optionally, before using the preset language model to perform recognition prediction on the text block to be recognized, it further includes:

[0059] Obtain an initial language model and training samples of image data containing text;

[0060] Use the training samples to perform unsupervised training on the initial language model to obtain a trained language model, and use the trained language model as the preset language model.

[0061] In this embodiment, an initial language model and training samples of image data containing text are obtained. The image data in the training samples is segmented to obtain text blocks corresponding to the text. The initial language model is unsupervised-trained using the text blocks to obtain a trained language model, and the trained language model is used as the preset language model.

[0062] Optionally, unsupervised-training the initial language model using the training samples to obtain a trained language model includes:

[0063] Using a character recognition model to recognize the characters in the training samples to obtain a text recognition result;

[0064] Performing masking processing on the text recognition result to obtain masked characters corresponding to the masking processing result;

[0065] Using similar-shaped characters corresponding to the masked characters to replace the masked characters, and pre-training the initial language model to obtain a trained language model.

[0066] In this embodiment, a character recognition model is used to recognize the characters in the training samples. The character recognition model is the character recognition model OCR model. The OCR model is used to recognize the text blocks in the training samples to obtain corresponding text recognition results. The text recognition results are sorted by default to generate picture text, and image text data containing prediction labels is obtained. Masking processing is performed on the picture text data to obtain masked characters corresponding to the masking processing result. Using similar-shaped characters corresponding to the masked characters to replace the masked characters, and pre-training the initial language model to obtain a trained language model.

[0067] Among them, when performing masking processing, it is set that 25% of the characters are masked, and 75% of the characters remain unchanged. 90% of the masked characters in the 25% of the characters being masked are replaced with similar-shaped characters. When replacing, if the same character is masked multiple times, the same character is replaced with different similar-shaped characters according to the corresponding probability values. For example, the similar-shaped characters of the character "叶" are "十" with a target probability value of 0.6 and "什" with a target probability value of 0.3. Then, when replacing the character "叶", there is a probability of 0.6 / (0.6 + 0.3) to be replaced with "十", and a probability of 0.3 / (0.6 + 0.3) to be replaced with "什". 10% of the masked characters in the 25% of the characters being masked are only replaced with random characters.

[0068] Predicting the masked characters based on the context information of 75% of the characters, calculating the corresponding loss between the prediction result of the masked characters and the corresponding similar-shaped characters, and pre-training the initial language model according to the corresponding loss to obtain a trained language model.

[0069] It should be noted that after obtaining the trained language model through pre-training, the trained language model is fine-tuned. The text in the training samples is manually annotated to generate corresponding image text, which is then used to obtain the image text with the corresponding real labels. During pre-training, the image text is masked to obtain the mask characters corresponding to the masking results. The mask characters are then predicted to obtain the predicted characters. The predicted characters are compared with the image text with the corresponding labels of the mask characters, and the corresponding loss is calculated. Based on the loss, the trained language model is fine-tuned in the first stage to obtain the fine-tuned language model.

[0070] The fine-tuned language model undergoes a second stage of fine-tuning. Through active learning, character recognition and prediction are performed on the text in the training samples. Predicted characters whose predicted probability values ​​are less than a first preset threshold are replaced, using the most similar-looking character with the highest similarity score. This yields the final trained language model.

[0071] Optionally, before replacing the mask character with a similar-looking character, the method also includes:

[0072] The character recognition model is used to recognize characters in the text of the training samples, and N predicted characters and the prediction probability value of each predicted character are obtained for each character in the text at each time step in the corresponding time series.

[0073] For any given character, the maximum predicted probability value of each predicted character in the corresponding time series is selected as the probability value of the target for each predicted character, thus obtaining N target probability values ​​corresponding to N predicted characters;

[0074] Select the predicted character that is equal to the mask character as the keyword, calculate the ratio of the target probability value between the remaining predicted characters and the keyword, and determine the predicted character corresponding to the maximum value of the ratio as the similar character to the mask character.

[0075] In this embodiment, when determining the similarity score of corresponding similar-looking Chinese characters, a character recognition model is used to recognize the text in the training samples, and N predicted characters and their corresponding predicted probability values at each moment in the corresponding time sequence for each character in the text are obtained. For any character, the maximum predicted probability value of each predicted character in the corresponding time sequence is selected as the target probability value of each predicted character, and N target probability values corresponding to the N predicted characters are obtained. For example, the recognition results corresponding to "叶" include multiple characters, such as "叶", "十", and "什". The prediction of "叶" includes predicted characters at 3 moments. The recognition result shows that the probability value of "叶" at the first moment is 0.96, the probability value of "十" is 0.23, and the probability value of "什" is 0.11. The recognition result shows that the probability value of "叶" at the second moment is 0.9, the probability value of "十" is 0.2, and the probability value of "什" is 0.17. The recognition result shows that the probability value of "叶" at the third moment is 0.8, the probability value of "十" is 0.3, and the probability value of "什" is 0.1. When selecting the maximum predicted probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character, the maximum predicted probability value corresponding to "叶" is 0.96, and 0.96 is used as the target probability value of the first predicted character of "叶". The maximum predicted probability value corresponding to "十" is 0.3, and 0.3 is used as the target probability value of the first predicted character of "十". The maximum predicted probability value corresponding to "什" is 0.17, and 0.17 is used as the target probability value of the first predicted character of "什".

[0076] Select the predicted character equal to the masked character as the keyword, and calculate the ratio of the target probability values between the remaining predicted characters and the keyword. For example, when masking the character "叶", select the predicted character "叶" in the prediction result as the keyword, and calculate the ratio of the target probability values between the predicted characters "叶" and "十" and "什", that is, the similarity score between the predicted character "十" and the predicted character "叶" is 0.3 / 0.96, and the similarity score between the predicted character "什" and the predicted character "叶" is 0.17 / 0.96. Determine the similar-looking character corresponding to the masked character as the predicted character corresponding to the maximum ratio, that is, determine the predicted character "十" as the similar-looking character of the predicted character "叶".

[0077] S204: Calculate the mean value of the predicted probability values corresponding to the same characters in the candidate character set. If the maximum mean value is greater than the second preset threshold, use the character corresponding to the maximum mean value as the replacement character, and use the replacement character to replace the character to be corrected to obtain the correction result corresponding to the character to be corrected.

[0078] In step S204, according to the characters in the candidate character set, correct the character to be corrected at the position corresponding to the character to be corrected, and replace the character to be corrected with a similar-looking Chinese character corresponding to the character to be corrected.

[0079] In this embodiment, the mean of the predicted probability values ​​corresponding to the same characters in the candidate character set is calculated. That is, the mean between the target probability value corresponding to the first predicted character and the predicted probability values ​​in the second predicted character is calculated. The character corresponding to the maximum mean is used as the replacement character, and the character to be corrected is replaced with the replacement character to obtain the correction result corresponding to the character to be corrected. Using the probability values ​​corresponding to the predicted characters obtained by different methods for calculation improves the prediction efficiency of the predicted characters.

[0080] Obtain the text block to be recognized, and use a character recognition model to recognize characters in the text block to obtain the recognition result of each character in the text block, as well as N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time sequence, where N is an integer greater than 1. For any character, select the maximum prediction probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character, and obtain N target probability values ​​corresponding to N first predicted characters. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained. The method uses a preset language model to predict and identify the text block to be recognized, obtaining M second predicted characters and their corresponding predicted probability values ​​for each character position in the text block. Based on the position of the character to be corrected, the M second predicted characters and N first predicted characters corresponding to the position of the character to be corrected are determined as a candidate character set, where M is an integer greater than 1. The mean of the predicted probability values ​​corresponding to the same characters in the candidate character set is calculated. The character with the maximum mean value greater than a second preset threshold is used as the replacement character. The replacement character is used to replace the character to be corrected, thus obtaining the correction result corresponding to the character to be corrected. In this application, the position of the character to be corrected is determined by the character recognition model, and the character corresponding to the maximum value of the average of the character probability values ​​predicted by the character recognition model and the preset language model is used as the replacement character. Based on the prior knowledge of the character to be corrected, the correction position is determined to perform end-to-end error correction processing, thereby improving the error correction accuracy.

[0081] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a character recognition and error correction device based on artificial intelligence provided in an embodiment of the present invention. In this embodiment, the terminal includes units used for performing... Figure 2 The steps in the corresponding embodiments. Please refer to the details. Figure 2 as well as Figure 2 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 3 As shown, the character recognition and error correction device 30 includes: an acquisition module 31, a selection module 32, a prediction module 33, and a calculation module 34.

[0082] The acquisition module 31 is used to acquire the text block to be recognized, use the character recognition model to perform character recognition on the text block to be recognized, and obtain the recognition result of each character in the text block to be recognized, as well as the N first predicted characters and the prediction probability value of each first predicted character at each time in the corresponding time series, where N is an integer greater than 1.

[0083] The selection module 32 is used to select the maximum predicted probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character for any character, and obtain N target probability values ​​corresponding to N first predicted characters. If the maximum value of the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected and the position of the character to be corrected is obtained.

[0084] The prediction module 33 is used to perform recognition and prediction on the text block to be recognized using a preset language model, to obtain the M second prediction characters at each character position in the text block to be recognized and the prediction probability value of each second prediction character, and to determine the M second prediction characters corresponding to the character position to be corrected and the N first prediction characters corresponding to the character to be corrected as the candidate character set according to the position of the character to be corrected, where M is an integer greater than 1.

[0085] The calculation module 34 is used to calculate the mean of the predicted probability values ​​of the same characters in the candidate character set. If the maximum mean value is greater than the second preset threshold, the character corresponding to the maximum mean value is used as the replacement character. The replacement character is used to replace the character to be corrected, and the correction result corresponding to the character to be corrected is obtained.

[0086] Optionally, the acquisition module 31 includes:

[0087] The extraction unit is used to extract image features from the text block to be recognized, and obtain the corresponding image features.

[0088] The merging unit is used to predict image features using a character recognition model with a greedy algorithm loss, obtain the predicted characters in the text block to be recognized, merge adjacent identical predicted characters, and obtain the recognition result for each character in the text block to be recognized.

[0089] Optionally, the acquisition module 31 includes:

[0090] The recognition unit is used to perform character recognition on the text block to be recognized, and to obtain the initial predicted character and the prediction probability value of each character in the text block to be recognized at each time step in the corresponding time sequence.

[0091] The sorting unit is used to sort the predicted probability values ​​from largest to smallest, select the first predicted character with the top N predicted probability values ​​as the first predicted character, and obtain N first predicted characters and the predicted probability value of each first predicted character.

[0092] Optionally, the character recognition and error correction device 30 further includes:

[0093] The training sample acquisition module is used to acquire training samples of the initial language model and image data containing text;

[0094] The training module is used to perform unsupervised training on the initial language model using training samples to obtain a trained language model, which is then used as the preset language model.

[0095] Optionally, the above training module includes:

[0096] The unit is used to perform character recognition on the text in the training samples using a character recognition model, and obtain the text recognition result.

[0097] The masking unit is used to perform masking processing on the text recognition results to obtain the masking characters corresponding to the masking results.

[0098] The replacement unit is used to replace the mask character with a similar-looking character corresponding to the mask character, and to pre-train the initial language model to obtain a trained language model.

[0099] Optionally, the above training module also includes:

[0100] The predictive character recognition unit is used to perform character recognition on the text in the training samples using a character recognition model, and to obtain N predicted characters and the prediction probability value of each predicted character at each time step in the corresponding time series for each character in the text.

[0101] The target probability value determination unit is used to select the maximum predicted probability value of each predicted character in the corresponding time sequence as the probability value of the target of each predicted character for any given character, thereby obtaining N target probability values ​​corresponding to N predicted characters;

[0102] The calculation unit is used to select the predicted character that is equal to the mask character as the keyword, calculate the ratio of the target probability value between the remaining predicted characters and the keyword, and determine the predicted character corresponding to the maximum value of the ratio as the similar character to the mask character.

[0103] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0104] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Figure 4 As shown, the terminal device of this embodiment includes: at least one processor ( Figure 4Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, wherein the processor, when executing the computer program, implements any of the above-described steps of the AI-based character recognition and error correction method.

[0105] The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. A terminal device may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as network interfaces, displays, and input devices.

[0106] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0107] The memory includes readable storage media, internal memory, etc., wherein the internal memory can be the main memory of the terminal device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of the terminal device, or in some embodiments, it can be an external storage device of the terminal device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital Card (SD), or a Flash Card. Furthermore, the memory can include both internal storage units and external storage devices of the terminal device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0108] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0109] The present invention can implement all or part of the processes in the above embodiments of the method, or it can be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps in the above method embodiments.

[0110] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0112] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A character recognition and error correction method based on artificial intelligence, characterized in that, The character recognition and error correction method includes: A text block to be recognized is obtained, and an OCR character recognition model based on a greedy algorithm loss is used to recognize the characters in the text block to be recognized, so as to obtain the recognition result of each character in the text block to be recognized, and the N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time series, where N is an integer greater than 1; the OCR character recognition model based on a greedy algorithm loss includes a convolutional neural network and a 3-layer bidirectional long short-term memory network stacked structure, and the bidirectional long short-term memory network is internally set with a projection network; For any character, the maximum predicted probability value of each first predicted character in the corresponding time sequence is selected as the target probability value of the corresponding first predicted character, and N target probability values ​​corresponding to N first predicted characters are obtained. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained. The text block to be identified is identified and predicted using a preset language model, and the M second predicted characters and the prediction probability value of each second predicted character at each character position in the text block to be identified are obtained. Based on the position of the character to be corrected, the M second predicted characters corresponding to the position of the character to be corrected and the N first predicted characters corresponding to the character to be corrected are determined as a candidate character set, where M is an integer greater than 1. Before using a preset language model to perform recognition and prediction on the text block to be recognized, the method further includes: Obtain the initial language model and training samples of image data containing text; The character recognition model is used to perform character recognition on the text in the training samples to obtain the text recognition results. The text recognition result is masked to obtain the masked character corresponding to the masking result; The mask character is replaced with a similar-looking character corresponding to the mask character, and the initial language model is pre-trained to obtain a trained language model. The trained language model is then used as the preset language model. Calculate the mean of the predicted probability values ​​corresponding to the same character in the candidate character set. If the maximum value of the mean is greater than a second preset threshold, use the character corresponding to the maximum value of the mean as the replacement character and use the replacement character to replace the character to be corrected to obtain the correction result corresponding to the character to be corrected.

2. The character recognition and error correction method as described in claim 1, characterized in that, The step of performing character recognition on the text block to be recognized, to obtain the recognition result of each character in the text block to be recognized, includes: Image features are extracted from the text block to be identified to obtain the corresponding image features; The image features are predicted to obtain the corresponding predicted characters in the text block to be recognized. Adjacent identical predicted characters are merged to obtain the recognition result for each character in the text block to be recognized.

3. The character recognition and error correction method as described in claim 1, characterized in that, The step of performing character recognition on the text block to be recognized, obtaining N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time sequence, includes: Character recognition is performed on the text block to be recognized to obtain the initial predicted character and the prediction probability value of each character in the text block at each time step in the corresponding time sequence. Sort the predicted probability values ​​from largest to smallest, select the first N predicted characters as the first predicted characters, and obtain N first predicted characters and the predicted probability value of each first predicted character.

4. The character recognition and error correction method as described in claim 1, characterized in that, Before replacing the mask character with a similar-looking character corresponding to the mask character, the method further includes: The character recognition model is used to recognize characters in the text of the training samples, and N predicted characters and the prediction probability value of each predicted character are obtained for each character in the text at each time step in the corresponding time series. For any given character, the maximum predicted probability value of each predicted character in the corresponding time series is selected as the probability value of the target for each predicted character, thus obtaining N target probability values ​​corresponding to N predicted characters; Select the predicted character that is equal to the mask character as the keyword, calculate the ratio of the target probability value between the remaining predicted characters and the keyword, and determine the predicted character corresponding to the maximum value of the ratio as the similar character to the mask character.

5. A character recognition and error correction device based on artificial intelligence, characterized in that, The character recognition and error correction device includes: The acquisition module is used to acquire a text block to be recognized, and to perform character recognition on the text block using an OCR character recognition model based on a greedy algorithm loss, to obtain the recognition result of each character in the text block, and the N first predicted characters and the prediction probability value of each first predicted character at each time step in the corresponding time series, where N is an integer greater than 1; the OCR character recognition model based on a greedy algorithm loss includes a convolutional neural network and a 3-layer bidirectional long short-term memory network stacked structure, and the bidirectional long short-term memory network internally sets a projection network; The selection module is used to select the maximum predicted probability value of each first predicted character in the corresponding time sequence as the target probability value of the corresponding first predicted character for any character, and obtain N target probability values ​​corresponding to N first predicted characters. If the maximum value among the N target probability values ​​is less than a first preset threshold, the character is determined to be a character to be corrected, and the position of the character to be corrected is obtained. The prediction module is used to perform recognition prediction on the text block to be recognized using a preset language model, to obtain M second predicted characters at each character position in the text block to be recognized and the prediction probability value of each second predicted character, and to determine the M second predicted characters corresponding to the character position to be corrected and the N first predicted characters corresponding to the character to be corrected as a candidate character set, where M is an integer greater than 1; before performing recognition prediction on the text block to be recognized using the preset language model, the module further includes: Obtain the initial language model and training samples of image data containing text; The character recognition model is used to perform character recognition on the text in the training samples to obtain the text recognition results. The text recognition result is masked to obtain the masked character corresponding to the masking result; The mask character is replaced with a similar-looking character corresponding to the mask character, and the initial language model is pre-trained to obtain a trained language model. The trained language model is then used as the preset language model. The calculation module is used to calculate the mean of the predicted probability values ​​corresponding to the same characters in the candidate character set. If the maximum value of the mean is greater than a second preset threshold, the character corresponding to the maximum value of the mean is used as a replacement character, and the replacement character is used to replace the character to be corrected to obtain the correction result corresponding to the character to be corrected.

6. The character recognition and error correction device as described in claim 5, characterized in that, The acquisition module includes: The extraction unit is used to extract image features from the text block to be identified to obtain corresponding image features; The merging unit is used to predict the image features using a character recognition model with a greedy algorithm loss, obtain the predicted character in the text block to be recognized, merge adjacent identical predicted characters, and obtain the recognition result for each character in the text block to be recognized.

7. A terminal device, characterized in that, The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the character recognition and error correction method as described in any one of claims 1 to 4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the character recognition and error correction method as described in any one of claims 1 to 4.

Images