Text correction methods, training methods and related equipment based on RPA and AI

Abstract

The present disclosure proposes a text error correction method, training method and related equipment based on RPA and AI, which relates to the field of AI and RPA, wherein the method includes: based on the OCR model, character recognition is performed on the image to be recognized, and the confidence of each predicted character in the predicted text is obtained; according to the confidence of each predicted character, the character to be processed is determined from each predicted character; the character to be processed in the predicted text is masked; the predicted text after masking is predicted using the prediction model to predict the characters, and at least one replacement character corresponding to the character to be processed is obtained; according to the similarity between at least one replacement character and the character to be processed, the target character is determined from at least one replacement character; the target character is used to replace the character to be recognized in the predicted text to obtain the recognized text. Therefore, after the text information is recognized based on the OCR technology, the characters in the text information are corrected, which can improve the accuracy and reliability of the text information recognition results.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence (AI) and robotic process automation (RPA), and in particular to a text error correction method, a training method and related equipment based on RPA and AI. Background Art

[0002] RPA uses specific "robot software" to simulate human operations on computers and automatically execute process tasks according to rules.

[0003] AI is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0004] For enterprises, it may be necessary to recognize and extract character information from images or PDF (Portable Document Format) documents, for example, to recognize and extract invoice information, order information, etc. In related technologies, character information from images or documents can be extracted based on optical character recognition (OCR) technology.

[0005] However, due to the size of the image, the accuracy of the OCR recognition results may not be guaranteed. For example, if the image size is small, the text "Account" in the image may be mistakenly recognized as "Account". In addition, if there is a seal on the image, the accuracy of the OCR recognition results cannot be guaranteed. For example, the word "car" in the image may be mistakenly recognized as "military". Summary of the invention

[0006] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0007] To this end, the present disclosure proposes a text error correction method, training method and related equipment based on RPA and AI, so as to correct the characters in the text information after the text information is recognized based on OCR technology, thereby improving the accuracy and reliability of the text information recognition results.

[0008] The first aspect of the present disclosure provides a text error correction method based on RPA and AI, the method comprising:

[0009] Based on an optical character recognition (OCR) model, character recognition is performed on the image to be recognized to obtain a predicted text and a confidence level of each predicted character in the predicted text;

[0010] Determining a character to be processed from the predicted characters according to the confidence level of each predicted character;

[0011] Masking the to-be-processed character in the predicted text, and using a prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the to-be-processed character;

[0012] According to the similarity between the at least one replacement character and the character to be processed, a target character is determined from the at least one replacement character, and the character to be recognized in the predicted text is replaced by the target character to obtain a recognized text.

[0013] The second aspect of the present disclosure provides a method for training the prediction model described in the first aspect of the present disclosure, including:

[0014] Get sample text;

[0015] Masking at least one sample character in the sample text to obtain a masked sample text;

[0016] Inputting the masked sample text into an initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain an output text;

[0017] According to the difference between the sample text and the output text, the model parameters in the prediction model are adjusted.

[0018] The third aspect of the present disclosure provides a text error correction device based on RPA and AI, including:

[0019] A recognition module, used for performing character recognition on the image to be recognized based on an optical character recognition (OCR) model to obtain a predicted text and a confidence level of each predicted character in the predicted text;

[0020] A determination module, used for determining a character to be processed from each of the predicted characters according to the confidence level of each of the predicted characters;

[0021] A masking module, used for masking the characters to be processed in the predicted text;

[0022] A prediction module, used for performing character prediction on the masked predicted text using a prediction model to obtain at least one replacement character corresponding to the character to be processed;

[0023] The replacement module is used to determine a target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and use the target character to replace the character to be recognized in the predicted text to obtain a recognized text.

[0024] The fourth aspect of the present disclosure provides a device for training the prediction model described in the third aspect of the present disclosure, including:

[0025] An acquisition module is used to acquire sample text;

[0026] A masking module, used for masking at least one sample character in the sample text to obtain a masked sample text;

[0027] An input module, used for inputting the masked sample text into an initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain an output text;

[0028] The adjustment module is used to adjust the model parameters in the prediction model according to the difference between the sample text and the output text.

[0029] An embodiment of the fifth aspect of the present disclosure proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in the embodiment of the first aspect of the present disclosure, or implements the method described in the embodiment of the second aspect of the present disclosure.

[0030] The sixth aspect embodiment of the present disclosure proposes a non-temporary computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method described in the first aspect embodiment of the present disclosure, or implements the method described in the second aspect embodiment of the present disclosure.

[0031] The seventh aspect embodiment of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect embodiment of the present disclosure, or implements the method described in the second aspect embodiment of the present disclosure.

[0032] The technical solution provided by the embodiments of the present disclosure has the following beneficial effects:

[0033] Based on the OCR model, character recognition is performed on the image to be recognized to obtain the predicted text and the confidence of each predicted character in the predicted text; according to the confidence of each predicted character, the character to be processed is determined from each predicted character; the character to be processed in the predicted text is masked, and the predicted model is used to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the character to be processed; according to the similarity between at least one replacement character and the character to be processed, the target character is determined from at least one replacement character, and the target character is used to replace the character to be recognized in the predicted text to obtain the recognized text. Therefore, after the text information is recognized based on the OCR technology, the characters in the text information are corrected, which can improve the accuracy and reliability of the text information recognition results. In addition, there is no need to manually correct the characters in the text information, which can release human resources, reduce labor costs, and improve the applicability of the method.

[0034] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0036] Figure 1 Indicates the image to be recognized Figure 1 ;

[0037] Figure 2 Indicates the image to be recognized Figure 2 ;

[0038] Figure 3 A flowchart of a text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0039] Figure 4 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0040] Figure 5 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0041] Figure 6 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0042] Figure 7 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0043] Figure 8 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0044] Fig. 9 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure;

[0045] Fig.10 Schematic diagram of the OCR recognition process in the embodiment of the present disclosure;

[0046] Fig.11 This is a schematic diagram of the text line content recognition process in the embodiment of the present disclosure;

[0047] Fig.12 is a schematic diagram of the structure of the prediction model in the embodiment of the present disclosure;

[0048] Fig.13 is a schematic diagram of character similarity in an embodiment of the present disclosure;

[0049] Fig.14 Indicates the image to be recognized Figure 3 ;

[0050] Fig.15 Indicates the image to be recognized Figure 4 ;

[0051] Fig.16 A flowchart of a training method provided by an embodiment of the present disclosure;

[0052] Fig.17 A structural diagram of a text error correction device based on RPA and AI provided in an embodiment of the present disclosure;

[0053] Fig.18 A schematic diagram of the structure of a training device provided by an embodiment of the present disclosure;

[0054] Fig.19 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0055] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0056] In the related art, although the comprehensive indicators of the Optical Character Recognition (OCR) model (such as the F1 indicator, where the F1 indicator is the harmonic mean of the precision rate and the recall rate) have achieved relatively high values, due to the defects of the OCR model in visual deep learning, there are still problems with low accuracy of recognition results in some long-tail problems. Typical scenarios include the following aspects:

[0057] First, stain interference, such as red seals, ink stains, etc.;

[0058] Second, similar-looking characters, such as "戍" and "戌, 成, 戊";

[0059] Third, font deformation caused by image deformation.

[0060] For example, as Figure 1 shown, when the characters in the image are too small, the character "账" is more likely to be misrecognized as the similar-looking character "帐"; for another example, as Figure 2 shown, when there is red seal interference in the image, the character "车" in the image is more likely to be misrecognized as "军".

[0061] To address the above problems, in the related art, the common solutions mainly include the following:

[0062] First, add semantic information during the OCR decoding process to ensure that not only image features are used during decoding, but also semantic alignment is performed with the previous character.

[0063] Second, add multi-tasks for predicting semantic information. That is, randomly occlude characters on the image in a mosaic manner. The model finally has two prediction networks (i.e., the head network). One prediction network is used to predict the characters shown on the image, and the other prediction network is used to predict the occluded characters on the image. Thus, the model can learn semantic-based error correction information.

[0064] Regarding the above methods, after multiple tests, the inventor found that: in order to improve the prediction effect of the model, in a large number of synthesized image samples, it is necessary to increase the exposure times of some long-tail characters. For example, some rare characters need to appear in the image samples enough times, that is, it is necessary to add random texts containing rare characters in the image samples. And, in order to enable the model to distinguish some similar-looking characters, it is necessary to deliberately let multiple similar-looking characters appear in one image sample, which will result in the lack of semantic coherence of the text in the synthesized image samples. Therefore, the first method above will not be applicable.

[0065] In addition, in real image samples, in order to reduce the labeling cost of samples, the position of a single character is generally not labeled, which will result in the inability to accurately mask the characters at a certain position in the real image samples. Therefore, the second method mentioned above will not be applicable.

[0066] In response to the above problems, the present disclosure proposes a text error correction method, a training method and related equipment based on RPA and AI.

[0067] The following describes the text error correction method, training method and related devices based on RPA and AI in the embodiments of the present disclosure with reference to the accompanying drawings. Before describing the embodiments of the present disclosure in detail, for ease of understanding, the commonly used technical terms are first introduced:

[0068] "RPA" is the abbreviation of Robotic Process Automation, which provides professional and comprehensive process automation solutions for enterprises and individuals. RPA simulates human operations on computers through specific "robot software" and automatically executes process tasks according to rules. That is, RPA robots can quickly and accurately collect data from user operation interfaces by simulating user mouse and keyboard operations, process these data based on clear logical rules, and then quickly and accurately enter them into another system or interface. In this way, the investment in labor costs can be greatly reduced, the existing office efficiency can be effectively improved, and work can be completed accurately, stably and quickly.

[0069] "AI" is the abbreviation of Artificial Intelligence, which is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. AI hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; AI software technologies mainly include computer vision technology, speech recognition technology, natural language processing (NLP) technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology and other major directions.

[0070] "OCR" refers to the process in which an electronic device examines characters printed on paper, determines their shapes by detecting dark and light patterns, and then uses character recognition methods to translate the shapes into computer text; that is, for printed characters, an optical method is used to convert the text in a paper document into a black and white dot matrix image file, and recognition software is used to convert the text in the image into a text format for further editing and processing by word processing software.

[0071] The “OCR model” is a pre-trained model that has learned the correspondence between the input image and the output text.

[0072] “Image to be identified” refers to any image that needs to be identified. For example, the image to be identified may be an image containing invoice information, an image containing order information, and so on.

[0073] "Predicted text" refers to the text information or OCR recognition results obtained by the OCR model when performing character recognition on the image to be recognized.

[0074] "Confidence", also known as recognition probability or classification probability, refers to the probability value output by the OCR model.

[0075] The “prediction model” refers to a model obtained after training, and the prediction model is used to predict characters of input text.

[0076] The "mask character" refers to a character used to mask text information. The mask character may be a pre-set character, or the mask character may be a random character, for example, a character may be randomly obtained from a set dictionary as the mask character.

[0077] The "set dictionary" refers to a preset dictionary, for example, the set dictionary may include common or frequently used characters. For example, taking the set dictionary including Chinese characters as an example, the set dictionary may include 3900 common Chinese characters.

[0078] "Target text" refers to the text information output by the prediction model.

[0079] “Sample text” refers to the text information used to train the prediction model.

[0080] “Specific characters” refer to pre-set special characters. For example, the specific characters may be OOV (Out of Vocabulary) characters, or other characters. The present disclosure does not impose any limitation on this.

[0081] “Set encoding algorithm” refers to a pre-set encoding algorithm. For example, the set encoding algorithm may be a four-corner encoding algorithm, or may be other encoding algorithms. The present disclosure does not impose any limitation on this.

[0082] “Confidence threshold” refers to a pre-set threshold.

[0083] Figure 3 A flowchart of a text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0084] In a possible implementation method provided by an embodiment of the present disclosure, the present disclosure takes the RPA and AI-based text correction method as an example configured in a text correction device, and the text correction device can be applied to any electronic device with computing capabilities.

[0085] The electronic device may be a personal computer, a mobile terminal, etc. The mobile terminal may be, for example, a mobile phone, a tablet computer, a personal digital assistant, or other hardware device with various operating systems.

[0086] In another possible implementation of the embodiments of the present disclosure, the text error correction method based on RPA and AI can be applied to an RPA robot, wherein the RPA robot can run in any electronic device with computing capabilities.

[0087] like Figure 3 As shown, the text error correction method based on RPA and AI may include the following steps:

[0088] Step 101 : Based on the OCR model, character recognition is performed on the image to be recognized to obtain a predicted text and the confidence of each predicted character in the predicted text.

[0089] In the disclosed embodiments, the image to be identified may be directly acquired, for example, the image to be identified containing text information may be directly acquired. Alternatively, the image to be identified may be indirectly acquired, for example, a document in a format such as PDF or PSD (PSD is a special format for Adobe's graphic design software Photoshop) containing text information may be acquired, and the image to be identified containing text information may be extracted or intercepted from the above document.

[0090] The format of the image to be identified may be JPG (or JPEG (Joint Photographic Experts Group)), PNG (Portable Network Graphics) or other image formats, and the present disclosure does not impose any limitation on this.

[0091] In the embodiments of the present disclosure, an OCR model may be used to perform character recognition on an image to be recognized, and a predicted text and a confidence level of each predicted character in the predicted text may be obtained.

[0092] Step 102: determining a character to be processed from the predicted characters according to the confidence level of each predicted character.

[0093] In the disclosed embodiment, the characters to be processed may be determined from the predicted characters according to the confidence of each predicted character, wherein the number of the characters to be processed may be at least one.

[0094] In a possible implementation of the embodiment of the present disclosure, the confidence of each predicted character may be compared with a set confidence threshold, and the predicted characters with confidence higher than the confidence threshold are used as characters to be processed.

[0095] In another possible implementation of the embodiment of the present disclosure, the predicted character with the highest confidence level may be used as the character to be processed.

[0096] In another possible implementation of the embodiment of the present disclosure, the predicted characters may be sorted from large to small according to the confidence values, and a target number of predicted characters ranked first are selected as characters to be processed.

[0097] The value of the target number is positively correlated with the length of the predicted text, that is, the longer the length of the predicted text is, the larger the value of the target number is.

[0098] As an example, considering that in most OCR recognition error scenarios, a sentence has only one incorrect character, therefore, in the present disclosure, the number of sentences contained in the predicted text can be determined, and the value of the target number can be determined according to the number of sentences. Among them, the target number is positively correlated with the number of sentences, that is, the more sentences there are, the larger the value of the target number is, and conversely, the fewer sentences there are, the smaller the value of the target number is.

[0099] In summary, the character to be processed can be determined from the predicted characters based on different methods, which can improve the flexibility and applicability of the method.

[0100] Step 103 , masking the characters to be processed in the predicted text, and using the prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the characters to be processed.

[0101] In an embodiment of the present disclosure, the characters to be processed in the predicted text can be masked to obtain the masked predicted text, and the masked predicted text can be input into the prediction model to use the prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the characters to be processed.

[0102] Step 104, determining a target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and using the target character to replace the character to be recognized in the predicted text to obtain a recognized text.

[0103] In the disclosed embodiment, the similarity between each replacement character and the character to be processed can be calculated, and based on the similarity between each replacement character and the character to be processed, a target character can be determined from each replacement character, and the target character can be used to replace the character to be recognized in the predicted text to obtain the recognized text.

[0104] The similarity may be character similarity (or character shape similarity). That is, in the present disclosure, considering that the main reason for OCR recognition error is: interference between similar characters or characters, the character similarity (or character shape similarity) between each replacement character and the character to be processed may be calculated, and according to the character similarity (or character shape similarity), a target character may be determined from each replacement character, and the target character may be used to replace the character to be recognized in the predicted text.

[0105] As a possible implementation method, each replacement character can be encoded based on a set encoding algorithm to obtain a first encoding value of each replacement character, and the character to be recognized can be encoded based on the set encoding algorithm to obtain a second encoding value of the character to be recognized. Therefore, in the present disclosure, the similarity between each replacement character and the character to be recognized can be determined based on the difference between the first encoding value and the second encoding value of each replacement character.

[0106] As an example, the characters are Chinese characters for exemplary explanation, and the encoding algorithm can be set as a four-corner encoding algorithm. It can be understood that the more similar the Chinese characters are in terms of glyphs, the closer the four-corner encoding values ​​are. Based on the above characteristics, the difference between the first encoding value and the second encoding value of each replacement character can be calculated, and the similarity between each replacement character and the character to be recognized can be determined based on the difference between the first encoding value and the second encoding value of each replacement character. Among them, the similarity and the difference are inversely related, that is, the smaller the difference, the higher the similarity, and conversely, the larger the difference, the lower the similarity.

[0107] It should be noted that the above only takes the four-corner coding algorithm as an example. In actual application, the coding algorithm may also be other coding algorithms, and the present disclosure does not impose any limitation on this.

[0108] As another possible implementation, for each replacement character, a first image can be obtained by drawing according to the replacement character. For example, the replacement character can be drawn on a blank image to obtain the first image, and a second image can be obtained by drawing according to the character to be recognized. For example, the character to be recognized can also be drawn on a blank image to obtain the second image. In the present disclosure, the similarity between the first image and the second image can be calculated, and the similarity between the replacement character and the character to be recognized can be determined based on the similarity between the first image and the second image.

[0109] Among them, the similarity between the replacement character and the character to be recognized is positively correlated with the similarity between the first image and the second image, that is, the higher the similarity between the first image and the second image, the higher the similarity between the replacement character and the character to be recognized. Conversely, the lower the similarity between the first image and the second image, the lower the similarity between the replacement character and the character to be recognized.

[0110] As another possible implementation, for each replacement character, based on a feature extraction algorithm, feature extraction can be performed on the replacement character to obtain a feature vector corresponding to the replacement character, and feature extraction can be performed on the character to be recognized to obtain a feature vector of the character to be recognized. In the present disclosure, the similarity between the replacement character and the character to be recognized can be determined according to the similarity between the feature vector of the replacement character and the feature vector of the character to be recognized.

[0111] Among them, the similarity between the replacement character and the character to be recognized is in a positive relationship with the similarity between the feature vector of the replacement character and the feature vector of the character to be recognized.

[0112] It should be noted that the above calculation method of similarity is only an exemplary illustration, but the present disclosure is not limited thereto, that is, the present disclosure does not limit the calculation method of similarity, as long as the similarity between each replacement character and the character to be recognized can be obtained.

[0113] In the embodiments of the present disclosure, after determining the similarity between the replacement character and the character to be processed, the target character can be determined from each replacement character according to the similarity between each replacement character and the character to be processed. For example, the replacement character with the largest similarity can be used as the target character, so that the target character can be used to replace the character to be recognized in the predicted text to obtain the recognized text.

[0114] For example, taking the predicted text as "There is a peach tree at the south corner of the garden of Le" and the character to be processed as "Le" for illustration, after masking the character "Le" with the mask character "Kai", the masked predicted text obtained is "There is a peach tree at the south corner of the garden of Kai". After the prediction model performs character prediction on the masked predicted text, the replacement characters obtained are "Dong" and "Xi". Since the character similarity between "Dong" and "Le" is higher than the character similarity between "Xi" and "Le", "Dong" can be used as the target character, and the target character is used to replace the "character to be processed" in the predicted text, and the recognized text obtained is "There is a peach tree at the southeast corner of the garden".

[0115] The text error correction method based on RPA and AI according to the embodiments of the present disclosure performs character recognition on the image to be recognized based on an OCR model to obtain the predicted text and the confidence of each predicted character in the predicted text; determines the characters to be processed from each predicted character according to the confidence of each predicted character; masks the characters to be processed in the predicted text, and uses a prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the character to be processed; determines the target character from at least one replacement character according to the similarity between at least one replacement character and the character to be processed, and uses the target character to replace the character to be recognized in the predicted text to obtain the recognized text. Thus, after recognizing the text information based on the OCR technology, correcting the characters in the text information can improve the accuracy and reliability of the recognition result of the text information. In addition, there is no need to manually correct the characters in the text information, which can release human resources, reduce labor costs, and improve the applicability of this method.

[0116] To clearly illustrate how to obtain at least one replacement character corresponding to the character to be processed in the above embodiments of the present disclosure, the present disclosure also proposes a text error correction method based on RPA and AI.

[0117] Figure 4 It is a schematic flowchart of another text error correction method based on RPA and AI provided by the embodiments of the present disclosure.

[0118] Such as Figure 4 shown, the text error correction method based on RPA and AI may include the following steps:

[0119] Step 201, based on an OCR model, perform character recognition on the image to be recognized to obtain the predicted text and the confidence of each predicted character in the predicted text.

[0120] Step 202, determine the characters to be processed from each predicted character according to the confidence of each predicted character.

[0121] The execution processes of steps 201 to 202 can refer to the execution processes of any embodiment of the present disclosure, which will not be elaborated here.

[0122] Step 203, determine the target position of the character to be processed in the predicted text.

[0123] In the embodiments of the present disclosure, the position where the character to be processed is located in the predicted text can be determined, which is denoted as the target position in the present disclosure.

[0124] For example, taking the predicted text as "There is a peach tree at the south corner of the garden" and the character to be processed as "乐" for example, the target position can be: the fourth character position.

[0125] Step 204: Obtain a masking character, and use the masking character to replace the character to be processed at the target position in the predicted text, so as to obtain a masked predicted text.

[0126] In an embodiment of the present disclosure, the masking character may be a preset fixed character, or the masking character may also be a random character. The present disclosure does not limit this.

[0127] In an embodiment of the present disclosure, the character to be processed at the target position in the predicted text may be replaced with a masking character to obtain a masked predicted text.

[0128] Still taking the above example for illustration, assuming the masking character is "Kai", then the character "Le" can be masked with the masking character "Kai", and the masked predicted text obtained is "There is a peach tree at the Kai south corner of the garden".

[0129] Step 205: Input the masked predicted text into a prediction model, and use the prediction model to perform character prediction on the masked predicted text, so as to obtain at least one target text.

[0130] In an embodiment of the present disclosure, the masked predicted text may be input into a prediction model, and the prediction model may perform character prediction on the masked predicted text to obtain at least one target text. That is to say, in the present disclosure, the prediction model may predict all characters in the entire text in a manner similar to machine translation to obtain at least one target text.

[0131] Still taking the above example for illustration, the target texts output by the prediction model may be "There is a peach tree at the southeast corner of the garden" and "There is a peach tree at the southwest corner of the garden".

[0132] Step 206: Use the characters at the target positions in at least one target text as at least one replacement character.

[0133] In an embodiment of the present disclosure, the characters at the target positions in at least one target text may be used as at least one replacement character.

[0134] Still taking the above example for illustration, the characters at the fourth character position in each target text may be used as replacement characters. For example, the replacement characters may be "East" and "West".

[0135] Step 207: Determine a target character from at least one replacement character according to the similarity between at least one replacement character and the character to be processed, and use the target character to replace the character to be recognized in the predicted text, so as to obtain a recognized text.

[0136] The execution process of Step 207 may refer to the execution process of any embodiment of the present disclosure, and will not be elaborated here.

[0137] The text error correction method based on RPA and AI in the disclosed embodiment determines the target position of the character to be processed in the predicted text; obtains the mask character, and uses the mask character to replace the character to be processed at the target position in the predicted text to obtain the masked predicted text; inputs the masked predicted text into the prediction model, and uses the prediction model to perform character prediction on the masked predicted text to obtain at least one target text; and uses the character at the target position in the at least one target text as at least one replacement character. Therefore, using deep learning technology to predict at least one replacement character can improve the accuracy and reliability of the prediction result.

[0138] In order to clearly illustrate how the prediction model in the above embodiment is trained, the present disclosure also proposes a text error correction method.

[0139] Figure 5 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0140] like Figure 5 As shown, based on the embodiment shown in 4, the text error correction method may further include the following steps:

[0141] Step 301, obtaining sample text.

[0142] In the embodiments of the present disclosure, sample text can be obtained from an existing training set, or sample text can be collected online, such as by using web crawler technology to collect sample text online, or sample text can be collected offline, such as by capturing images of paper text content, and then using OCR technology to identify each character in the image to obtain sample text, or sample text can be synthesized artificially, etc. The embodiments of the present disclosure do not limit this.

[0143] Step 302: Mask at least one sample character in the sample text to obtain the masked sample text.

[0144] In the embodiment of the present disclosure, a mask character may be used to mask at least one sample character in the sample text to obtain the masked sample text.

[0145] As an example, at least one sample character in the sample text may be replaced with a random character with a set first random probability, and / or at least one sample character in the sample text may be replaced with a fixed character with a set second random probability. The first random probability and the second random probability may be the same or different, and the present disclosure does not limit this.

[0146] For example, it can be determined whether the current random probability matches the first random probability. If the current random probability matches the first random probability, at least one sample character in the sample text is replaced with a random character, and it can be determined whether the current random probability matches the second random probability. If the current random probability matches the second random probability, at least one sample character in the sample text is replaced with a fixed character. If the current random probability does not match the second random probability, no processing is performed.

[0147] If the current random probability does not match the first random probability, then continue to determine whether the current random probability matches the second random probability. If the current random probability matches the second random probability, then replace at least one sample character in the sample text with a fixed character. If the current random probability does not match the second random probability, then no processing is performed.

[0148] Take the second random probability and the first random probability as 10 / 100=10% for example, and assume that the currently selected random number hits 30-40 of 0-100, then determine that the current random probability matches the first random probability and the second random probability. When masking the sample text, a random number can be randomly selected, and it can be determined whether the random number is between 30 and 40. If so, it is determined that the current random probability matches the first random probability, and at least one sample character in the sample text can be replaced with a random character, and continue to select a random number, and determine whether the random number is between 30 and 40. If so, it is determined that the current random probability matches the second random probability, and at least one sample character in the sample text can continue to be replaced with a fixed character.

[0149] Step 303: input the masked sample text into the initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain output text.

[0150] In the embodiment of the present disclosure, the masked sample text may be input into an initial prediction model, so that the prediction model performs character prediction on the masked sample text to obtain an output text.

[0151] Step 304: adjusting the model parameters in the prediction model according to the difference between the sample text and the output text.

[0152] In the disclosed embodiment, the model parameters in the prediction model may be adjusted according to the difference between the sample text and the output text.

[0153] As an example, a target loss value can be generated based on the difference between the sample text and the output text, wherein the target loss value is positively correlated with the above difference, that is, the smaller the difference, the smaller the target loss value, and conversely, the larger the difference, the larger the target loss value.

[0154] Therefore, in the present disclosure, the prediction model can be trained according to the target loss value, that is, the model parameters in the prediction model can be adjusted. For example, the prediction model can be trained according to the target loss value to minimize the value of the target loss value. It should be noted that the above example only uses the termination condition of model training as the minimization of the value of the target loss value. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set number threshold, etc., and the present disclosure does not limit this.

[0155] The text error correction method based on RPA and AI of the embodiment of the present disclosure obtains a sample text; masks at least one sample character in the sample text to obtain a masked sample text; inputs the masked sample text into the initial prediction model to use the prediction model to perform character prediction on the masked sample text to obtain an output text; and adjusts the model parameters in the prediction model according to the difference between the sample text and the output text. Thus, by training the prediction model in advance, the prediction effect of the prediction model can be improved, that is, using the trained prediction model to perform character prediction on the masked prediction text can improve the accuracy and reliability of the target text prediction result.

[0156] In order to clearly illustrate how the prediction model is trained in any embodiment of the present disclosure, the present disclosure also proposes a text error correction method.

[0157] Figure 6 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0158] like Figure 6 As shown, based on the embodiment shown in 4, the text error correction method may further include the following steps:

[0159] Step 401, obtaining sample text.

[0160] Step 402: Mask at least one sample character in the sample text to obtain the masked sample text.

[0161] Step 403: input the masked sample text into the initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain output text.

[0162] The execution process of steps 401 to 403 may refer to the execution process of any embodiment of the present disclosure, and will not be described in detail here.

[0163] Step 404: Generate a first loss value according to the difference in confidence distribution of characters between the sample text and the output text.

[0164] In the embodiment of the present disclosure, the difference in confidence distribution (or probability distribution) of characters between the sample text and the output text may be determined, and a first loss value may be generated according to the confidence distribution difference (or probability distribution difference).

[0165] The first loss value is positively correlated with the confidence distribution difference, that is, the smaller the confidence distribution difference is, the smaller the first loss value is, and conversely, the larger the confidence distribution difference is, the larger the first loss value is. For example, the first loss value may be a cross entropy loss Loss.

[0166] Step 405: determine the first position of at least one sample character to be masked in the sample text.

[0167] In an embodiment of the present disclosure, the position of at least one sample character to be masked in the sample text may be determined, which is recorded as the first position in the present disclosure.

[0168] Step 406: Generate a second loss value according to the difference between the sample character at the first position in the sample text and the output character at the first position in the output text.

[0169] In the embodiment of the present disclosure, the difference between the sample character at the first position in the sample text and the output character at the first position in the output text may be determined, and a second loss value may be generated according to the difference.

[0170] The second loss value is positively correlated with the above difference, that is, the smaller the difference, the smaller the second loss value, and vice versa, the larger the difference, the larger the second loss value. For example, the second loss value can also be a cross entropy Loss.

[0171] Step 407: Generate a target loss value according to the first loss value and the second loss value.

[0172] In the embodiment of the present disclosure, a target loss value may be generated according to the first loss value and the second loss value, wherein the target loss value is in a positive relationship with the first loss value, and the target loss value is also in a positive relationship with the second loss value.

[0173] As an example, the first loss value and the second loss value can be weighted according to the first weight of the first loss value and the second weight of the second loss value to obtain a weighted result, and the target loss value is determined according to the weighted result, wherein the target loss value is positively correlated with the weighted result.

[0174] It should be noted that, in the training process of the prediction model, more attention is generally paid to the loss value of the masked characters. Therefore, in the present disclosure, the second weight may be greater than the first weight.

[0175] Step 408: Adjust the model parameters in the prediction model according to the target loss value.

[0176] In the embodiments of the present disclosure, the prediction model can be trained according to the target loss value, that is, the model parameters in the prediction model can be adjusted. For example, the prediction model can be trained according to the target loss value to minimize the value of the target loss value. It should be noted that the above example only uses the termination condition of model training as the minimization of the value of the target loss value. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set number threshold, etc., and the present disclosure does not limit this.

[0177] The text error correction method based on RPA and AI in the embodiment of the present disclosure can improve the prediction effect of the prediction model by pre-training the prediction model, that is, using the trained prediction model to predict characters of the masked prediction text, which can improve the accuracy and reliability of the target text prediction results.

[0178] In a possible implementation of the embodiment of the present disclosure, in order to improve the accuracy of the model prediction results, the numbers, English characters, punctuation marks, and Chinese characters with strong semantic commonality such as the quantifiers "thousand" and "hundred" in the sample text can be replaced with specific characters (such as OOV characters), so that the model does not need to pay attention to these specific characters. Specifically, for step 205 in the above embodiment, a set dictionary can be obtained, and it can be determined whether each character in the masked sample text is located in the set dictionary. If each character in the masked sample text is located in the set dictionary, the masked sample text can be directly input into the initial prediction model to use the prediction model to perform character prediction on the masked sample text to obtain the output text. At this time, the target loss value can be a weighted result of the first loss value and the second loss value, that is, the first loss value and the second loss value can be weighted according to the first weight of the first loss value and the second weight of the second loss value to obtain the target loss value.

[0179] If the first character in the masked sample text is not in the set dictionary, the first character in the masked sample text can be replaced with a specific character, and the replaced sample text is input into the prediction model to use the prediction model to predict the characters of the replaced sample text to obtain the output text. At this time, the target loss value is not only obtained based on the weighted result of the first loss value and the second loss value, but also based on the specific character. Figure 7 , the above process is explained in detail.

[0180] Figure 7 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0181] like Figure 7 As shown, based on the embodiment shown in Figure 6, step 407 may include the following steps:

[0182] Step 501, determining a second position of a first character in a sample text.

[0183] In an embodiment of the present disclosure, when there is a first character in the masked sample text that is not in the set dictionary, the position of the first character in the sample text can be determined, which is recorded as the second position in the present disclosure.

[0184] Step 502: Generate a third loss value according to the difference between the specific character and the output character located at the second position in the output text.

[0185] In the embodiment of the present disclosure, a third loss value may be generated according to the difference between the specific character and the output character at the second position in the output text, wherein the third loss value is positively correlated with the difference, that is, the smaller the difference, the smaller the value of the third loss value, and conversely, the larger the difference, the larger the value of the third loss value.

[0186] Step 503: weighting the first loss value and the second loss value according to a first weight of the first loss value and a second weight of the second loss value to obtain a fourth loss value, wherein the second weight is greater than the first weight.

[0187] In the embodiment of the present disclosure, the first loss value and the second loss value may be weighted according to a first weight of the first loss value and a second weight of the second loss value to obtain a weighted result, and the weighted result may be used as the fourth loss value.

[0188] Step 504: Generate a target loss value according to the difference between the fourth loss value and the third loss value.

[0189] In the embodiment of the present disclosure, a target loss value can be generated based on the difference between the fourth loss value and the third loss value, wherein the target loss value is positively correlated with the above difference, that is, the smaller the difference is, the smaller the target loss value is, and conversely, the larger the difference is, the larger the target loss value is.

[0190] As an example, the third loss value may be subtracted from the fourth loss value to obtain a fifth loss value, and the fifth loss value may be amplified to obtain a target loss value. That is, the third loss value corresponding to a specific character that the model does not need to pay attention to may be removed from the fourth loss value, and in order to avoid the situation where there are fewer valid characters in the output text, resulting in a lower target loss value, the fifth loss value may be amplified in the present disclosure.

[0191] The text error correction method based on RPA and AI in the embodiment of the present disclosure can improve the prediction effect of the prediction model by pre-training the prediction model, that is, using the trained prediction model to predict characters of the masked prediction text, which can improve the accuracy and reliability of the target text prediction results.

[0192] In order to clearly explain how at least one replacement character corresponding to the character to be processed is obtained in the above embodiment of the present disclosure, the present disclosure also proposes a text error correction method based on RPA and AI.

[0193] Figure 8 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0194] like Figure 8 As shown, the text error correction method based on RPA and AI may include the following steps:

[0195] Step 601 : Based on the OCR model, character recognition is performed on the image to be recognized to obtain a predicted text and the confidence of each predicted character in the predicted text.

[0196] Step 602: Determine the character to be processed from the predicted characters according to the confidence level of each predicted character.

[0197] The execution process of steps 601 to 602 may refer to the execution process of any embodiment of the present disclosure, and will not be described in detail here.

[0198] Step 603: Use the mask character to replace the characters to be processed in the predicted text to obtain the masked predicted text.

[0199] In the embodiment of the present disclosure, the mask character may be a preset fixed character, or the mask character may also be a random character, and the present disclosure does not impose any limitation on this.

[0200] In the embodiment of the present disclosure, mask characters may be used to replace characters to be processed in the predicted text, so as to obtain masked predicted text.

[0201] Step 604: input the masked predicted text into a prediction model, so as to use the prediction model to predict the masked characters in the masked predicted text to obtain at least one replacement character.

[0202] In the embodiment of the present disclosure, the masked predicted text can be input into the prediction model, so that the prediction model can predict the masked characters in the masked predicted text to obtain at least one replacement character. That is, in the present disclosure, the prediction model can only predict the masked characters, which is similar to the task of cloze test.

[0203] In a possible implementation of the embodiment of the present disclosure, the prediction model can be trained in the following manner: obtaining a sample text, in order to distinguish it from the sample text in the above embodiment, Figures 5 to 7 The sample text in can be marked as the first sample text, the sample text in this embodiment can be marked as the second sample text, at least one second sample character in the second sample text can be masked to obtain the masked second sample text, and the masked second sample text can be input into the initial prediction model so that the prediction model can predict the at least one masked second sample character to obtain at least one recognized character, so that in the present disclosure, the model parameters in the prediction model can be adjusted according to the difference between at least one recognized character and at least one first sample character.

[0204] As an example, a target loss value can be generated according to the difference between at least one recognized character and at least one first sample character, wherein the target loss value is positively correlated with the above-mentioned difference, that is, the smaller the difference, the smaller the target loss value, and conversely, the larger the difference, the larger the target loss value. Therefore, in the present disclosure, the prediction model can be trained according to the target loss value, that is, the model parameters in the prediction model can be adjusted. For example, the prediction model can be trained according to the target loss value to minimize the value of the target loss value. It should be noted that the above only uses the termination condition of model training as the minimization of the value of the target loss value as an example. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set threshold, and so on. The present disclosure does not limit this.

[0205] Step 605: determine a target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and use the target character to replace the character to be recognized in the predicted text to obtain the recognized text.

[0206] The execution process of step 605 can refer to the execution process of any embodiment of the present disclosure, and will not be described in detail here.

[0207] The text error correction method based on RPA and AI in the embodiment of the present disclosure replaces the characters to be processed in the predicted text with mask characters to obtain the masked predicted text; the masked predicted text is input into the prediction model to use the prediction model to predict the mask characters in the masked predicted text to obtain at least one replacement character. Thus, the accuracy and reliability of the prediction results can be improved by using deep learning technology to predict at least one replacement character. Figure 2 The different ways of the illustrated embodiments predict replacement characters, which can enhance the flexibility and applicability of the method.

[0208] In order to clearly explain how character recognition is performed on the image to be recognized in the above embodiments of the present disclosure, the present disclosure also proposes a text error correction method.

[0209] Fig. 9 A flowchart of another text error correction method based on RPA and AI provided in an embodiment of the present disclosure.

[0210] like Fig. 9 As shown, the text error correction method based on RPA and AI may include the following steps:

[0211] Step 701: Use the feature extraction branch in the OCR model to extract features from the image to be recognized to obtain a first feature map.

[0212] In the disclosed embodiment, the feature extraction branch in the OCR model can be used to extract features of the image to be identified to obtain a first feature map. For example, the feature extraction branch can be a backbone network such as CNN (Convolutional Neural Network) and VIT (Vision Transformer), and the feature extraction of the image to be identified is performed through the backbone network to obtain the first feature map.

[0213] It should be noted that the image to be recognized may be tilted, deformed or flipped to a certain extent, and the above situations will affect the reliability of the subsequent model recognition results. Therefore, in a possible implementation of the embodiment of the present disclosure, in order to improve the accuracy of the subsequent model recognition results, after obtaining the image to be recognized, the tilt angle of the image to be recognized can be corrected.

[0214] Specifically, the angle of the image to be recognized may be predicted to determine the tilt angle of the document image to be processed.

[0215] As an example, when the tilt angle of the image to be identified is large, an image classification model may be used to perform angle prediction on the image to be identified to determine the tilt angle of the image to be identified.

[0216] As another example, when the tilt angle of the image to be recognized is small, the angle of the image to be recognized can be predicted based on a corner detection algorithm to determine the tilt angle of the image to be recognized.

[0217] It should be understood that when the image to be identified is tilted, what needs to be detected is an irregular quadrilateral, so the conventional target detection algorithm is invalid. The key point detection algorithm can be used to detect the four corner points of the irregular quadrilateral, and then extract the quadrilateral based on the four corner points, so that the tilt angle of the image to be identified can be determined based on the extracted quadrilateral.

[0218] It should be noted that, in actual application, the above two examples can also be combined to predict the tilt angle of the image to be recognized.

[0219] In the embodiment of the present disclosure, after determining the tilt angle of the image to be identified, the image to be identified can be rotated according to the tilt angle. Thus, the image to be identified that is tilted or flipped can be angle corrected to improve the reliability of subsequent image recognition results.

[0220] Step 702, using the fusion branch in the OCR model, fuses the first feature map with the position map to obtain a second feature map, wherein each element in the position map corresponds one-to-one to each element in the first feature map, and the elements in the position map are used to indicate the coordinates of the corresponding elements in the first feature map in the image to be recognized.

[0221] In the disclosed embodiment, the image to be identified can be position-encoded to obtain a position map, wherein each element in the position map corresponds one-to-one to each element in the first feature map, and each element in the position map is used to indicate the coordinates of the corresponding element in the first feature map in the image to be identified.

[0222] In the disclosed embodiment, the fusion branch in the OCR model can be used to fuse the first feature map with the position map to obtain a second feature map. For example, the first feature map can be spliced ​​with the corresponding position map to obtain the second feature map. Alternatively, the first feature map can be spliced ​​with the corresponding position map to obtain a spliced ​​feature map, and the spliced ​​feature map can be input into the convolution layer to fuse to obtain the second feature map.

[0223] Step 703: Use the feature transformation branch in the OCR model to perform feature transformation on the second feature map to obtain a third feature map.

[0224] In the disclosed embodiment, the feature transformation branch in the OCR model may be used to perform feature transformation on the second feature map to obtain a third feature map.

[0225] Step 704: Use the prediction branch in the OCR model to decode the third feature map to obtain the predicted text and the confidence level corresponding to each predicted character in the predicted text.

[0226] In the disclosed embodiment, the prediction branch in the OCR model may be used to decode the third feature map to obtain the predicted text and the confidence level corresponding to each predicted character in the predicted text.

[0227] Step 705: Determine the character to be processed from the predicted characters according to the confidence level of each predicted character.

[0228] Step 706 , masking the characters to be processed in the predicted text, and using the prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the characters to be processed.

[0229] Step 707: determine a target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and use the target character to replace the character to be recognized in the predicted text to obtain the recognized text.

[0230] In the embodiment of the present disclosure, the execution process of steps 705 to 707 can refer to the execution process of any embodiment of the present disclosure, and will not be repeated here.

[0231] As an example, error correction can be used as a post-processing model that is not coupled with OCR recognition. Fig.10 The OCR recognition process shown in FIG. 1 is used to exemplify the recognition of character information in an image. Fig.10 As shown, the OCR recognition process mainly includes: rotating the image; detecting the text rows (columns) on the image; performing character recognition on the content of the text rows (columns); restoring the coordinate information of each recognized character; and outputting each character, the coordinates of each character and the confidence of each character.

[0232] After analysis, it can be determined that the main reason for long-tail recognition errors is that Fig.10 Therefore, a post-processing module can be added after the text line content recognition stage and before the coordinate information restoration stage, and the text line content can be corrected by using the post-processing module.

[0233] The model structure used in the text line content recognition stage can be as follows Fig.11As shown, feature extraction can be performed on the image to be recognized through a feature extraction branch, such as a backbone network such as CNN or VIT, to obtain a first feature map; the extracted first feature map is fused with the position map through a fusion branch to obtain a second feature map; the second feature map is subjected to feature transformation through a feature transformation branch to obtain a third feature map. For example, the feature transformation branch may include a conversion branch (the conversion branch is used to convert the second feature map into a one-dimensional feature (Reshape to 1-D) or a two-dimensional feature (Reshape to 2-D)), a Transformer (used to continue to perform feature transformation on the converted feature to obtain a feature sequence sequence feature) and an MLP (Multi Layer Perceptron) (used to perform feature transformation on the feature sequence to obtain a third feature map); the third feature map is decoded through a prediction branch, such as CTC (Connectionist temporal classification, temporal classification based on neural networks), to obtain a predicted text and the confidence corresponding to each predicted character in the predicted text.

[0234] Optionally, in the final stage of model training, you can enable multi-task training, such as center-loss and rdrop (Regularized Dropout, a regularization method), to improve the accuracy of model recognition.

[0235] The text error correction method based on RPA and AI in the embodiment of the present disclosure can improve the accuracy and reliability of the recognition result by adopting deep learning technology to perform OCR recognition on the image to be recognized.

[0236] It should be noted that in the related technologies, text error correction can be achieved based on dictionary search algorithms, traditional Bayesian machine learning algorithms, and deep learning algorithms. In the broad sense of text error correction, there are generally three types of problems that need to be solved:

[0237] The first type is to replace the substitution. For example, replace "I feel extremely happy today" with "I feel very happy today".

[0238] The second category is Insert & Delete. For example, "I feel very happy today" is added to "I feel very happy today"; for another example, "I feel very happy today" is deleted to "I feel very happy today".

[0239] The third type is rewriting Local Paraphrasing (minimal), for example, rewriting "I feel very happy today" to "I feel very happy today".

[0240] However, considering that the main reasons for OCR recognition errors are: interference or misrecognition caused by similar characters, the problems that need to be solved are mainly the first type of problems mentioned above. In addition, since error correction is a common scenario, the dictionary-based solution is not applicable, and a model-based solution can be used.

[0241] If a supervised model is used to correct text errors, then before the correction, a large number of labeled sample pairs need to be used to train the supervised model, where the labeled sample pairs include the erroneous results and accurate results of OCR recognition. Specifically, the training process is: according to the difference between the erroneous results and the accurate results in the labeled sample pairs, the supervised model is trained.

[0242] However, in the above method, in order to improve the prediction effect of the model, it is necessary to generate enough labeled sample pairs to train the model, and the labeling cost is high. Therefore, the supervised model trained with labeled sample pairs is not applicable.

[0243] from Fig.10 It can be seen that the OCR recognition process includes multiple stages, each of which requires a GPU (graphics processing unit). In order to save expensive GPUs, the post-processing module (or error correction model) can be a lightweight model that can be executed on a CPU (Central Processing Unit). Therefore, a lightweight unsupervised model and solution can be used to correct the OCR recognition results.

[0244] The specific implementation plan is:

[0245] Considering that the commonly used models for text error correction in related technologies are basically divided into the following two categories:

[0246] The first category is sequence tasks, such as machine translation. Simply put, it is to translate a sentence with typos into an accurate sentence.

[0247] The second type is error detection and correction. In simple terms, it detects possible erroneous words in a sentence, and corrects them if they are detected.

[0248] In a single error correction task, the second type has better error correction effect. For example, the soft-masked BERT (Bidirectional Encoder Representations from Transformers) model can achieve better error correction effect. The reason is that the additional typo detection task can effectively alleviate the problem of false recall in error correction.

[0249] Therefore, in this disclosure, the second type of design model mentioned above can be adopted to divide the entire error correction task into three parts:

[0250] The first part is: error word detection.

[0251] In the OCR task, the last layer of the OCR model is an N classification task (N is the number of recognized characters). From the perspective of the loss function of the classification network, the more accurate the predicted character is, the higher the confidence (or softmax recognition probability) p of the character in the classification task will be.

[0252] The inventors used the OCR model to test the test text in the test set and obtained: the confidence or recognition probability of each character, when the confidence threshold (or probability threshold) is gradually reduced from 1.0, the recognition accuracy of the remaining characters can be improved. From this, it can be inferred that the optimal confidence threshold (or optimal probability threshold) f, that is, below the optimal confidence threshold (or optimal probability threshold) f, the recognition accuracy of the remaining characters is not greatly improved.

[0253] In the present disclosure, in the OCR error correction scenario, the confidence (or recognition probability) p of the character is used as prior knowledge to detect the wrong word, and no additional error detection model is required. Specifically, when the confidence (or recognition probability) p of the character is higher than the optimal confidence threshold (or called the optimal probability threshold) f, it is considered that the character recognition is correct. On the contrary, when the confidence (or recognition probability) p of the character is not higher than the optimal confidence threshold (or called the optimal probability threshold) f, it is considered that the character recognition is incorrect.

[0254] In the case where a character is recognized incorrectly, the character can be recalled as a corrected word.

[0255] The second part is: Correction word recall.

[0256] Corrected word recall can be illustrated from offline model training and online model calling respectively.

[0257] Among them, offline model training:

[0258] Model design: After statistical analysis, in the vast majority of OCR recognition error scenarios, a sentence has only one incorrect character, and due to the lack of supervised annotated sample pairs, the present disclosure can adopt the MLM (Mask Language Modeling) approach to implement model training based on self-supervised training tasks.

[0259] For model design, two options can be included: the first is to predict only the masked characters, similar to the cloze task; the second is to predict all the characters in the whole sentence, similar to machine translation. After experiments, it can be found that the second prediction effect is better. For example, for the training corpus "There is a peach tree in the southeast corner of the garden" (referred to as sample text in this disclosure), the following tasks can be designed (the left side is the model input, and the right side is the model output):

[0260] There is a peach tree in the south corner of the garden -> There is a peach tree in the southeast corner of the garden;

[0261] There is a peach tree in the south corner of the garden. There is a peach tree in the southeast corner of the garden.

[0262] In the model architecture design, in order to reduce the output delay, non-autoregressive tasks can be used, the decoding Decoder part of the BERT model is discarded, and only the encoding Encoder part is used. The prediction performance and prediction effect are combined. In this disclosure, Fig.12 As shown, the prediction model can use the standard six-layer TransformerEncoderLayer stack, the maximum character length of the model input is 32 bits, the model parameter head_num is selected as 6, and the dimension of the character feature vector embedding is 128, that is, the prediction model (such as Fig.12 The right part, Multi-Head Attention) can include 6 layers of attention encoding layers. The encoding sequence (up to 32 bits) is input into the prediction model. The last layer of the prediction model can output a 32×128 encoding vector. Finally, the output features of the 6th attention encoding layer can be classified by softmax on 32 feature vectors to obtain 32 predicted characters.

[0263] Among them, Multi-Head Attention projects Q, K, and V through multiple (e.g., h) different linear transformations, and then splices the different attention results; the spliced ​​results are calculated using scaled dot product (Scaled Dot-Product Attention); the results of the attention calculation are processed by nonlinear transformation through the connection layer (Concat) and the linear layer (Liner). Among them, the structure of Scaled Dot-Product Attention can be as follows Fig.12As shown on the left. In Scaled Dot-Product Attention, Q and K are matrix-multiplied (MatMul), the result of the matrix multiplication is scaled (Scale) and masked (Mask), the masked result is processed by the activation function SoftMax, and the result of the activation function is matrix-multiplied with V, which can be expressed as the following formula:

[0264]

[0265] Among them, Q, K, and V are three matrices obtained by performing matrix operations on the input of the attention layer and the model parameters. k represents the normalization factor, and T represents the transpose operation of the matrix. k When d is small, the result is similar to the dot product. k When larger, it performs better without scaling, but the dot product is faster to compute and scaling reduces the impact.

[0266] It should be noted that the reason why the prediction model uses the Transformer as the basic structure is that the advantage of the Transformer is that it captures the correlation between long-distance characters through the Multi-Head Self-Attention mechanism in the Multi-Head Attention mechanism. For example, the Multi-Head Attention mechanism can be Fig.12 Shown on the right.

[0267] Training data: In terms of training corpus, multiple news data can be captured from multiple data sources, and the multiple news data can be split into more than 200 million training corpus (referred to as sample text in this disclosure). Different from the method of masking the sample text before model training, in this disclosure, the sample text can be randomly masked during the training process, and any character in the sample text can be replaced with a random character with a first random probability (e.g., 10%), and any character in the sample text can be replaced with a mask character (i.e., a fixed character) with a second random probability (e.g., 10%).

[0268] In addition, on the sample text, in order to reduce the size of the model, the 3,900 most common Chinese characters can be used to form a set dictionary; in order to improve the accuracy of the model's prediction results, the numbers, English characters, punctuation marks, and Chinese characters with strong semantic universality such as the quantifiers "thousand" and "hundred" in the sample text can be replaced with specific characters, such as the OOV character.

[0269] Loss function design: adopt the cross entropy loss function, wherein the value of the loss function can be determined by the following method: the cross entropy Loss consists of two parts, one part is the cross entropy Loss of all predicted characters in the whole sentence (recorded as the first loss value in this disclosure), and the other part is the cross entropy Loss of the masked characters (recorded as the second loss value in this disclosure). Because the second part is more concerned in the final use, the weight of the second part Loss will be higher.

[0270] In the Loss calculation process, the Loss of OOV character prediction (referred to as the third loss value in this disclosure) needs to be removed, and the remaining Loss is proportionally amplified according to the number of valid characters, mainly to avoid the situation where there are fewer valid characters, resulting in lower Loss.

[0271] Among them, online call: When calling online, the characters with confidence or recognition probability p lower than f in the OCR recognition result can be replaced with OOV characters. If the number of characters contained in the sentence exceeds 32, the sentence can be truncated at the beginning and end to ensure that the masked characters are located in the middle of the sentence. Although the OCR recognition result contains all predicted characters, the present disclosure only uses low-probability or low-confidence predicted characters as recall, and ignores predicted characters in other positions, such as recalling 20 (Top20) predicted characters.

[0272] The third part is: correcting word order.

[0273] As can be seen from the above, the entire recall part is a purely semantic model. Using only the Top1 recalled by the model for correction cannot guarantee the accuracy of the correction result. For example, the input of the model is: "There is a peach tree in the south corner of the garden [mask]", and the predicted characters "East" and "West" may be wrong. Therefore, a sorting module with prior knowledge is needed to select the most accurate character.

[0274] As mentioned above, the main reason for OCR recognition errors is similar characters or interference between characters. Therefore, in the present disclosure, character similarity can be used as a sorting indicator.

[0275] Among them, the character similarity can be calculated by the following three methods:

[0276] The first one is the Chinese character four-corner encoding algorithm. Based on the Chinese character four-corner encoding algorithm, the encoding value of the character is calculated, and the similarity between the characters is determined according to the encoding value of the character.

[0277] Second, image similarity. Fonts such as KaiTi and SongTi can be used to draw each character fully onto a 128*128 white-background image, and then calculate the image similarity pairwise as the character similarity. Alternatively, feature extraction can be performed on the image, and based on the extracted feature vectors, the image similarity can be calculated. Thus, the character similarity can be determined according to the image similarity.

[0278] Third, OCR feature vectors. The feature vectors of each character can be determined according to the softmax matrix of the last layer in the OCR model, and thus similarity calculation can be performed based on the feature vectors of the characters. The above matrix is an N×D matrix, where N is the number of characters and D is the vector dimension. It can be considered that each row of D-dimensional vectors in this matrix is the representation vector or feature vector of the corresponding character. Through the third method, the character similarity as shown in Fig.13 can be obtained. Among them, Fig.13 the third column is the character shape similarity.

[0279] Through experiments, it can be known that the effect of the third method is higher than that of the first and second methods. The character similarity between the recalled characters and the characters to be corrected can be determined according to the character shape similarity. If the character similarity is higher than the set threshold, the character with the highest character similarity can be selected as the target character.

[0280] After the model is launched, the inventor tests the model on a large-scale test set and obtains that on the basis that the F1 index is already very high, it is still possible to achieve an increase in the F1 index of more than 0.03%.

[0281] As an example, as shown in Fig.14 , when detecting the text line in the image, if a partial area of the text line characters is cropped, resulting in "提" being misrecognized as "摆", then through the above model, normal correction can be achieved.

[0282] As another example, as shown in Fig.15 , when there is a red seal interference in the image, resulting in "通用" being misrecognized as "涌用", through the above model, normal correction can be achieved.

[0283] The above are the respective embodiments corresponding to the text error correction method. The present disclosure also proposes a method for training the prediction model in any of the above method embodiments.

[0284] Fig.16 It is a schematic flowchart of a training method provided by an embodiment of the present disclosure.

[0285] As shown in Fig.16 , the training method may include the following steps:

[0286] Step 801, obtain sample texts.

[0287] Step 801: mask at least one sample character in a sample text to obtain a masked sample text.

[0288] Step 802: input the masked sample text into the initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain output text.

[0289] Step 803: Adjust the model parameters in the prediction model according to the difference between the sample text and the output text.

[0290] In a possible implementation of the embodiments of the present disclosure, a first loss value can be generated based on the difference in confidence distribution of characters between a sample text and an output text; a first position of at least one sample character to be masked in the sample text is determined; a second loss value is generated based on the difference between the sample character at the first position in the sample text and the output character at the first position in the output text; a target loss value is generated based on the first loss value and the second loss value; and model parameters in the prediction model are adjusted based on the target loss value.

[0291] In a possible implementation of the embodiments of the present disclosure, a set dictionary can be obtained; it is determined whether each character in the masked sample text is located in the set dictionary; if there is a first character in the masked sample text that is not in the set dictionary, the first character in the masked sample text is replaced with a specific character; and a prediction model is used to perform character prediction on the replaced sample text to obtain an output text.

[0292] In a possible implementation of the embodiments of the present disclosure, the second position of the first character in the sample text can be determined; a third loss value can be generated based on the difference between the specific character and the output character located at the second position in the output text; the first loss value and the second loss value are weighted according to a first weight of the first loss value and a second weight of the second loss value to obtain a fourth loss value, wherein the second weight is greater than the first weight; and a target loss value is generated based on the difference between the fourth loss value and the third loss value.

[0293] It should be noted that the explanation of the text error correction method based on RPA and AI in any of the aforementioned embodiments is also applicable to this embodiment, and the implementation principle is similar, which will not be repeated here.

[0294] The training method of the embodiment of the present disclosure obtains a sample text; masks at least one sample character in the sample text to obtain the masked sample text; inputs the masked sample text into the initial prediction model to use the prediction model to perform character prediction on the masked sample text to obtain the output text; and adjusts the model parameters in the prediction model according to the difference between the sample text and the output text. Thus, by training the prediction model in advance, the prediction effect of the prediction model can be improved.

[0295] With the above Figures 3 to 9 Corresponding to the text error correction method based on RPA and AI provided in the embodiment, the present disclosure also provides a text error correction device based on RPA and AI. Figures 3 to 9 The text correction method based on RPA and AI provided in the embodiment corresponds to each other, so the implementation of the text correction method based on RPA and AI is also applicable to the text correction device based on RPA and AI provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0296] Fig.17 A structural diagram of a text error correction device based on RPA and AI provided in an embodiment of the present disclosure.

[0297] like Fig.17 As shown, the text error correction device 1700 based on RPA and AI may include: an identification module 1710, a determination module 1720, a mask module 1730, a prediction module 1740 and a replacement module 1750.

[0298] The recognition module 1710 is used to perform character recognition on the image to be recognized based on an optical character recognition (OCR) model to obtain a predicted text and the confidence level of each predicted character in the predicted text.

[0299] The determination module 1720 is used to determine the character to be processed from the predicted characters according to the confidence of each predicted character.

[0300] The masking module 1730 is used to mask the characters to be processed in the predicted text.

[0301] The prediction module 1740 is used to perform character prediction on the masked predicted text using a prediction model to obtain at least one replacement character corresponding to the character to be processed.

[0302] The replacement module 1750 is used to determine a target character from at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and use the target character to replace the character to be recognized in the predicted text to obtain the recognized text.

[0303] In a possible implementation of the embodiment of the present disclosure, the RPA and AI-based text error correction device 1700 can be applied to an RPA robot.

[0304] In a possible implementation of the embodiment of the present disclosure, the mask module 1730 is used to: determine the target position of the character to be processed in the predicted text; obtain the mask character, and use the mask character to replace the character to be processed at the target position in the predicted text to obtain the masked predicted text.

[0305] The prediction module 1740 is used to: input the masked predicted text into the prediction model, use the prediction model to perform character prediction on the masked predicted text to obtain at least one target text; and use the character at the target position in the at least one target text as at least one replacement character.

[0306] In a possible implementation of the embodiment of the present disclosure, the prediction model is obtained by training the following modules:

[0307] The acquisition module is used to obtain sample text.

[0308] The masking module 1730 is further used to mask at least one sample character in the sample text to obtain the masked sample text.

[0309] The input module is used to input the masked sample text into the initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain output text.

[0310] The adjustment module is used to adjust the model parameters in the prediction model according to the difference between the sample text and the output text.

[0311] In a possible implementation of the embodiments of the present disclosure, an adjustment module is used to: generate a first loss value based on the difference in confidence distribution of characters between a sample text and an output text; determine a first position in the sample text of at least one sample character to be masked; generate a second loss value based on the difference between the sample character at the first position in the sample text and the output character at the first position in the output text; generate a target loss value based on the first loss value and the second loss value; and adjust model parameters in the prediction model based on the target loss value.

[0312] In a possible implementation of the embodiment of the present disclosure, the input module is used to: obtain a set dictionary; determine whether each character in the masked sample text is in the set dictionary; if there is a first character in the masked sample text that is not in the set dictionary, replace the first character in the masked sample text with a specific character; input the replaced sample text into the prediction model to use the prediction model to perform character prediction on the replaced sample text to obtain an output text.

[0313] In a possible implementation of the embodiment of the present disclosure, the adjustment module is used to: determine the second position of the first character in the sample text; generate a third loss value based on the difference between the specific character and the output character located at the second position in the output text; weight the first loss value and the second loss value according to a first weight of the first loss value and a second weight of the second loss value to obtain a fourth loss value, wherein the second weight is greater than the first weight; and generate a target loss value based on the difference between the fourth loss value and the third loss value.

[0314] In a possible implementation of the embodiment of the present disclosure, the text error correction device 1700 based on RPA and AI may further include:

[0315] The first processing module is used to: encode at least one replacement character based on a set encoding algorithm to obtain a first encoding value of the at least one replacement character; encode the character to be recognized based on the set encoding algorithm to obtain a second encoding value of the character to be recognized; and determine the similarity between at least one replacement character and the character to be recognized based on the difference between the first encoding value and the second encoding value of the at least one replacement character.

[0316] In a possible implementation of the embodiment of the present disclosure, the text error correction device 1700 based on RPA and AI may further include:

[0317] The second processing module is used to: for each replacement character, draw according to the replacement character to obtain a first image; draw according to the character to be recognized to obtain a second image; and determine the similarity between the replacement character and the character to be recognized based on the similarity between the first image and the second image.

[0318] In a possible implementation of the embodiment of the present disclosure, the text error correction device 1700 based on RPA and AI may further include:

[0319] The third processing module is used to: perform feature extraction on at least one replacement character to obtain a feature vector of at least one replacement character; perform feature extraction on the character to be recognized to obtain a feature vector of the character to be recognized; and determine the similarity between at least one replacement character and the character to be recognized based on the similarity between the feature vector of at least one replacement character and the feature vector of the character to be recognized.

[0320] In a possible implementation of the embodiment of the present disclosure, the recognition module 1710 is used to: use the feature extraction branch in the OCR model to extract features from the image to be recognized to obtain a first feature map; use the fusion branch in the OCR model to fuse the first feature map with the position map to obtain a second feature map, wherein each element in the position map corresponds one-to-one to each element in the first feature map, and the elements in the position map are used to indicate the coordinates of the corresponding elements in the first feature map in the image to be recognized; use the feature transformation branch in the OCR model to transform the second feature map to obtain a third feature map; use the prediction branch in the OCR model to decode the third feature map to obtain predicted text and the confidence corresponding to each predicted character in the predicted text.

[0321] In a possible implementation of the embodiment of the present disclosure, the text error correction device 1700 based on RPA and AI may further include:

[0322] The fourth processing module is used to perform angle prediction on the image to be identified and determine the tilt angle of the image to be identified; and perform rotation processing on the image to be identified according to the tilt angle.

[0323] In a possible implementation of the embodiment of the present disclosure, the determination module 1720 is used to: use the predicted characters with confidence higher than the confidence threshold as the characters to be processed; or, use the predicted characters with the highest confidence as the characters to be processed; or, sort the predicted characters from large to small according to the confidence values, and select the target number of predicted characters in the front sort as the characters to be processed, wherein the value of the target number is positively correlated with the length of the predicted text.

[0324] The text error correction device based on RPA and AI of the disclosed embodiment performs character recognition on the image to be recognized based on the OCR model to obtain the predicted text and the confidence of each predicted character in the predicted text; determines the characters to be processed from each predicted character according to the confidence of each predicted character; masks the characters to be processed in the predicted text, and uses the prediction model to perform character prediction on the masked predicted text to obtain at least one replacement character corresponding to the characters to be processed; determines the target character from at least one replacement character according to the similarity between at least one replacement character and the characters to be processed, and uses the target character to replace the characters to be recognized in the predicted text to obtain the recognized text. Thus, after the text information is recognized based on the OCR technology, the characters in the text information are corrected, which can improve the accuracy and reliability of the text information recognition results. In addition, there is no need to manually correct the characters in the text information, which can release human resources, reduce labor costs, and improve the applicability of the method.

[0325] With the above Fig.16 Corresponding to the training method provided in the embodiment, the present disclosure also provides a training device. Fig.16 The training method provided in the embodiment corresponds to the above, so the implementation of the training method is also applicable to the training device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0326] Fig.18 A schematic diagram of the structure of a training device provided in an embodiment of the present disclosure.

[0327] like Fig.18 As shown, the training device 1800 may include: an acquisition module 1810 , a mask module 1820 , an input module 1830 and an adjustment module 1840 .

[0328] The acquisition module 1810 is used to acquire sample text.

[0329] The masking module 1820 is used to mask at least one sample character in the sample text to obtain the masked sample text.

[0330] The input module 1830 is used to input the masked sample text into the initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain output text.

[0331] The adjustment module 1840 is used to adjust the model parameters in the prediction model according to the difference between the sample text and the output text.

[0332] In a possible implementation of the embodiments of the present disclosure, the adjustment module 1840 is used to: generate a first loss value based on the difference in confidence distribution of characters between the sample text and the output text; determine the first position of at least one sample character to be masked in the sample text; generate a second loss value based on the difference between the sample character at the first position in the sample text and the output character at the first position in the output text; generate a target loss value based on the first loss value and the second loss value; and adjust the model parameters in the prediction model based on the target loss value.

[0333] In a possible implementation of the embodiment of the present disclosure, the input module 1830 is used to: obtain a set dictionary; determine whether each character in the masked sample text is in the set dictionary; if there is a first character in the masked sample text that is not in the set dictionary, replace the first character in the masked sample text with a specific character; input the replaced sample text into the prediction model to use the prediction model to perform character prediction on the replaced sample text to obtain an output text.

[0334] In a possible implementation of the embodiment of the present disclosure, the adjustment module 1840 is used to: determine the second position of the first character in the sample text; generate a third loss value based on the difference between the specific character and the output character located at the second position in the output text; weight the first loss value and the second loss value according to a first weight of the first loss value and a second weight of the second loss value to obtain a fourth loss value, wherein the second weight is greater than the first weight; and generate a target loss value based on the difference between the fourth loss value and the third loss value.

[0335] The text error correction device based on RPA and AI in the embodiment of the present disclosure obtains a sample text; masks at least one sample character in the sample text to obtain a masked sample text; inputs the masked sample text into the initial prediction model to use the prediction model to perform character prediction on the masked sample text to obtain an output text; and adjusts the model parameters in the prediction model according to the difference between the sample text and the output text. Thus, by training the prediction model in advance, the prediction effect of the prediction model can be improved.

[0336] In order to implement the above embodiments, the embodiments of the present disclosure also propose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the text error correction method based on RPA and AI as described in any of the above method embodiments, or implements the training method as described in any of the above method embodiments.

[0337] In order to implement the above embodiments, the embodiments of the present disclosure also propose a non-temporary computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the text error correction method based on RPA and AI as described in any of the above method embodiments, or implements the training method as described in any of the above method embodiments.

[0338] In order to implement the above embodiments, the embodiments of the present disclosure also propose a computer program product. When the instruction processor in the computer program product is executed, it implements the text error correction method based on RPA and AI as described in any of the above method embodiments, or implements the training method as described in any of the above method embodiments.

[0339] Fig.19 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Fig.19 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0340] like Fig.19 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the memory 28 and the processing unit 16).

[0341] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0342] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0343] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.19 not shown, usually called a "hard drive"). Although Fig.19 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0344] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0345] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0346] The processing unit 16 executes various functional applications and data processing by running the programs stored in the memory 28, such as implementing the methods mentioned in the above embodiments.

[0347] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0348] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0349] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0350] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0351] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0352] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0353] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0354] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for training a prediction model, characterized in that: The method comprises: Get sample text; Masking at least one sample character in the sample text to obtain a masked sample text; Inputting the masked sample text into an initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain an output text; Generate a first loss value based on the difference between the sample text and the output text; determine the first position of the at least one sample character to be masked in the sample text; generate a second loss value based on the difference between the sample character at the first position in the sample text and the output character at the first position in the output text; generate a target loss value based on the first loss value and the second loss value; and adjust the model parameters in the prediction model based on the target loss value.

2. The method according to claim 1, characterized in that The step of inputting the masked sample text into an initial prediction model, using the prediction model to perform character prediction on the masked sample text to obtain output text, comprises: Get the settings dictionary; Determine whether each character in the masked sample text is in the set dictionary; When there is a first character in the masked sample text that is not in the set dictionary, replacing the first character in the masked sample text with a specific character; The replaced sample text is input into the prediction model, so as to use the prediction model to perform character prediction on the replaced sample text to obtain the output text.

3. The method according to claim 2, characterized in that The generating a target loss value according to the first loss value and the second loss value comprises: Determining a second position of the first character in the sample text; generating a third loss value according to a difference between the specific character and an output character located at the second position in the output text; weighting the first loss value and the second loss value according to a first weight of the first loss value and a second weight of the second loss value to obtain a fourth loss value, wherein the second weight is greater than the first weight; The target loss value is generated according to the difference between the fourth loss value and the third loss value.

4. A text error correction method based on robotic process automation (RPA) and artificial intelligence (AI), characterized in that: The method comprises: Based on an optical character recognition (OCR) model, character recognition is performed on the image to be recognized to obtain a predicted text and a confidence level of each predicted character in the predicted text; Determining a character to be processed from the predicted characters according to the confidence level of each predicted character; Masking the to-be-processed character in the predicted text, and using a prediction model trained by any training method according to claims 1 to 3 to perform character prediction on the masked predicted text, so as to obtain at least one replacement character corresponding to the to-be-processed character; According to the similarity between the at least one replacement character and the character to be processed, a target character is determined from the at least one replacement character, and the character to be processed in the predicted text is replaced by the target character to obtain a recognized text.

5. The method according to claim 4, characterized in that The method is performed by an RPA robot.

6. The method according to claim 4, characterized in that The step of masking the characters to be processed in the predicted text and using a prediction model to perform character prediction on the masked predicted text includes: Determining a target position of the character to be processed in the predicted text; Obtaining a mask character, and using the mask character to replace the to-be-processed character at the target position in the predicted text, so as to obtain a masked predicted text; Inputting the masked predicted text into the prediction model, so as to use the prediction model to perform character prediction on the masked predicted text to obtain at least one target text; The character at the target position in the at least one target text is used as the at least one replacement character.

7. The method according to claim 6, characterized in that The prediction model is trained by the following steps: Get sample text; Masking at least one sample character in the sample text to obtain a masked sample text; Inputting the masked sample text into an initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain an output text; According to the difference between the sample text and the output text, the model parameters in the prediction model are adjusted.

8. The method according to any one of claims 4 to 7, characterized in that: Before determining the target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, the method further includes: Based on a set encoding algorithm, encode the at least one replacement character to obtain a first encoding value of the at least one replacement character; Based on the set encoding algorithm, encode the character to be processed to obtain a second encoding value of the character to be processed; The similarity between the at least one replacement character and the character to be processed is determined according to the difference between the first encoding value and the second encoding value of the at least one replacement character.

9. The method according to any one of claims 4 to 7, characterized in that: Before determining the target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, the method further includes: For each of the replacement characters, drawing is performed according to the replacement character to obtain a first image; Drawing is performed according to the character to be processed to obtain a second image; The similarity between the replacement character and the character to be processed is determined according to the similarity between the first image and the second image.

10. The method according to any one of claims 4 to 7, characterized in that: Before determining the target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, the method further includes: Performing feature extraction on the at least one replacement character to obtain a feature vector of the at least one replacement character; Extracting features of the character to be processed to obtain a feature vector of the character to be processed; The similarity between the at least one replacement character and the character to be processed is determined according to the similarity between the feature vector of the at least one replacement character and the feature vector of the character to be processed.

11. The method according to any one of claims 4 to 7, characterized in that: The character recognition of the image to be recognized based on the optical character recognition (OCR) model includes: Using the feature extraction branch in the OCR model, extracting features from the image to be recognized to obtain a first feature map; Using the fusion branch in the OCR model, the first feature map and the position map are fused to obtain a second feature map, wherein each element in the position map corresponds to each element in the first feature map one by one, and the elements in the position map are used to indicate the coordinates of the corresponding elements in the first feature map in the image to be recognized; Using the feature transformation branch in the OCR model, the second feature map is subjected to feature transformation to obtain a third feature map; The prediction branch in the OCR model is used to decode the third feature map to obtain the predicted text and the confidence corresponding to each predicted character in the predicted text.

12. The method according to claim 11, characterized in that Before extracting features of the image to be recognized by using the feature extraction branch in the OCR model to obtain a first feature map, the method further includes: Performing angle prediction on the image to be identified to determine the tilt angle of the image to be identified; The image to be recognized is rotated according to the tilt angle.

13. The method according to any one of claims 4 to 7, characterized in that: Determining the character to be processed from the predicted characters according to the confidence of each predicted character includes: The predicted characters whose confidence level is higher than the confidence level threshold are used as the characters to be processed; or, The predicted character with the highest confidence is used as the character to be processed; or, The predicted characters are sorted from large to small according to the confidence value, and a target number of predicted characters ranked first are selected as the characters to be processed, wherein the value of the target number is positively correlated with the length of the predicted text.

14. A text error correction device based on robotic process automation (RPA) and artificial intelligence (AI), characterized in that: include: A recognition module, used for performing character recognition on the image to be recognized based on an optical character recognition (OCR) model to obtain a predicted text and a confidence level of each predicted character in the predicted text; A determination module, used for determining a character to be processed from each of the predicted characters according to the confidence level of each of the predicted characters; A masking module, used for masking the characters to be processed in the predicted text; A prediction module, configured to use a prediction model trained by any training method according to claims 1 to 3 to perform character prediction on the masked predicted text, so as to obtain at least one replacement character corresponding to the character to be processed; The replacement module is used to determine a target character from the at least one replacement character according to the similarity between the at least one replacement character and the character to be processed, and use the target character to replace the character to be processed in the predicted text to obtain the recognized text.

15. The device according to claim 14, characterized in that The device is applied to an RPA robot.

16. A device for training a prediction model, characterized in that: The device comprises: An acquisition module is used to acquire sample text; A masking module, used for masking at least one sample character in the sample text to obtain a masked sample text; An input module, used for inputting the masked sample text into an initial prediction model, so as to use the prediction model to perform character prediction on the masked sample text to obtain an output text; An adjustment module is used to generate a first loss value according to the difference between the sample text and the output text; determine the first position of the at least one sample character to be masked in the sample text; generate a second loss value according to the difference between the sample character at the first position in the sample text and the output character at the first position in the output text; generate a target loss value according to the first loss value and the second loss value; and adjust the model parameters in the prediction model according to the target loss value.

17. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 13 is implemented.

18. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images