Character recognition model training method and device, storage medium and electronic device

Abstract

The present disclosure relates to the field of image processing technology, and provides a training method and device for a character recognition model, a computer-readable storage medium, and an electronic device. The method includes: inputting a sequence of character image blocks to be recognized corresponding to a routing trace image sample into a long short-term memory network to obtain a recognized text based on a template character set; recognizing the word image blocks to be recognized corresponding to the routing trace image sample according to an adaptive classifier and a template character set to obtain a character recognition result; merging the recognized text and the character recognition result to determine a target character recognition result; determining the correct character corresponding to the incorrectly recognized character to update the template character set; and retraining the long short-term memory network and the adaptive classifier based on the updated template character set, respectively, until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier. The present disclosure can improve the accuracy of character recognition.

Description

Background Art

[0002] The work order system is an important means of network monitoring and operation for telecom operators. There are many professional work orders circulating in the work order system. During the processing of website access failure work orders (such as packet loss, high latency, etc.), the route trace provided by the user will appear, that is, a screenshot of the route trace information. Recognizing the characters in the screenshot can greatly improve work efficiency.

[0003] In the related art, a general character recognition system is directly used to perform OCR (Optical Character Recognition) on the route trace screenshot.

[0004] However, this method of directly performing character recognition using a general character recognition framework has poor adaptability and low recognition accuracy for character recognition scenarios in work order screenshots.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide a method and device for training a character recognition model, a computer-readable storage medium, and an electronic device, thereby improving the problem of low character recognition accuracy in routing trace images at least to a certain extent.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a training method for a character recognition model is provided, comprising: obtaining a sequence of character image blocks to be recognized corresponding to a routing trace image sample, inputting the sequence of character image blocks to be recognized into a long short-term memory network, so as to obtain a recognition text of the routing trace image sample based on a template character set; obtaining a word image block to be recognized corresponding to the routing trace image sample, recognizing the word image block to be recognized according to an adaptive classifier and a template character set, so as to obtain a character recognition result of the word image block to be recognized; merging the recognition text and the character recognition result to determine a character recognition result of the routing trace image sample; determining a correct character corresponding to an incorrectly recognized character in the character recognition result, so as to update the template character set; and retraining the long short-term memory network and the adaptive classifier based on the routing trace image sample and the updated template character set, respectively, until a first preset condition is met, so as to obtain a target long short-term memory network and a target adaptive classifier.

[0009] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, merging the recognized text and the character recognition result to determine the character recognition result of the routing trace image sample includes: matching the character recognition result with the character in the corresponding position in the recognized text, and when the match is successful, determining the recognized character as the first target character; when the match fails, selecting a character with a large first similarity and determining it as the recognized second target character, wherein the first similarity is used to characterize the degree of similarity between the recognized character and its candidate characters in the template character set; and merging the first target character and the second target character according to the position of the character in the character sequence to determine the character recognition result of the trace image sample.

[0010] In an exemplary embodiment of the present disclosure, based on the aforementioned solution, the first preset condition includes that the recognition accuracy of the long short-term memory network and the adaptive classifier reaches a first preset value.

[0011] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the method further includes: when the recognition accuracy of the long short-term memory network reaches a first preset value and the recognition accuracy of the adaptive classifier does not reach the first preset value, obtaining the recognition text of the routing trace image sample based on the long short-term memory network, splitting the recognition text into single characters, and using the single characters as training labels of the adaptive classifier to perform supervised learning training on the adaptive classifier model; when the recognition accuracy of the adaptive classifier reaches the first preset value and the recognition accuracy of the long short-term memory network does not reach the first preset value, generating a character sequence based on the character recognition result corresponding to the routing trace image sample obtained by the adaptive classifier, and using the character sequence as the training label of the long short-term memory network to perform supervised learning training on the long short-term memory network.

[0012] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the word image block to be recognized is recognized according to the adaptive classifier and the template character set to obtain the character recognition result of the word image block to be recognized, including: performing a first recognition on the word image block to be recognized according to the adaptive classifier and the template character set to obtain an initial character recognition result of each word image block to be recognized; determining the word image block to be recognized whose initial character recognition result meets a second preset condition as a training sample, determining the character recognition result of the word image block to be recognized as a label of the training sample, and retraining the adaptive classifier; performing a second recognition on the word image block to be recognized based on the template character set and the retrained adaptive classifier to obtain the character recognition result of the word image block to be recognized.

[0013] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the second preset condition includes that the similarity between each character in the character recognition result corresponding to the word image block to be recognized and its candidate character in the template character set is greater than a second preset value.

[0014] In an exemplary embodiment of the present disclosure, based on the aforementioned scheme, the method further includes: preprocessing the routing trace image samples; performing character connected area detection and character segmentation on the preprocessed routing trace image samples to obtain character image blocks to be recognized or word image blocks to be recognized corresponding to the routing trace image samples.

[0015] According to a second aspect of the present disclosure, a training device for a character recognition model is provided, comprising: a text segment recognition module, configured to obtain a sequence of character image blocks to be recognized corresponding to a routing trace image sample, and input the sequence of character image blocks to be recognized into a long short-term memory network to obtain a recognition text of the routing trace image sample based on a template character set; a word character recognition module, configured to obtain a word image block to be recognized corresponding to the routing trace image sample, and recognize the word image block to be recognized according to an adaptive classifier and a template character set to obtain a character recognition result of the word image block to be recognized; a character recognition result determination module, configured to merge the recognition text and the character recognition result to determine a character recognition result of the routing trace image sample; a template character set update module, configured to determine a correct character corresponding to an incorrectly recognized character in the character recognition result to update the template character set; and a retraining module, configured to retrain the long short-term memory network and the adaptive classifier based on the routing trace image sample and the updated template character set, respectively, until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier. 。

[0016] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the training method of the character recognition model as described in the first aspect of the above embodiment is implemented.

[0017] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the character recognition model training method as described in the first aspect of the above embodiment.

[0018] It can be seen from the above technical solutions that the character recognition model training method, character recognition model training device, and computer-readable storage medium and electronic device for implementing the character recognition model training method in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0019] In the technical solutions provided by some embodiments of the present disclosure, the trained character recognition model is used for character recognition in a routing trace image. First, a sequence of character image blocks to be recognized corresponding to a routing trace image sample is obtained, and the sequence of character image blocks to be recognized is input into a long short-term memory network to obtain a recognition text of the routing trace image sample based on a template character set; then, a word image block to be recognized corresponding to the routing trace image sample is obtained, and the word image block to be recognized is recognized according to an adaptive classifier and a template character set to obtain a character recognition result of the word image block to be recognized; the recognition text and the character recognition result are merged to determine a character recognition result of the routing trace image sample; the characters that are recognized incorrectly in the character recognition result are determined, and the characters that are recognized incorrectly are re-marked as correct characters to update the template character set; the long short-term memory network and the adaptive classifier are retrained based on the routing trace image sample and the updated template character set, respectively, until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier. Compared with the related art, on the one hand, the present invention obtains a character recognition model based on the joint training of a long short-term memory network and an adaptive classifier, which can improve the recognition accuracy of the trained character recognition model; on the other hand, the present invention re-labels the incorrectly recognized characters for retraining, which can improve the model training efficiency while ensuring the recognition accuracy of the model.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0022] Figure 1 A schematic diagram showing a flow chart of a method for training a character recognition model in an exemplary embodiment of the present disclosure;

[0023] Figure 2A schematic diagram showing a flow chart of a method for secondary recognition based on an adaptive classifier in an exemplary embodiment of the present disclosure;

[0024] Figure 3 A schematic flow chart showing a method for determining a character recognition result in an exemplary embodiment of the present disclosure;

[0025] Figure 4A A screenshot of a work order to be identified in an exemplary embodiment of the present disclosure is shown;

[0026] Figure 4B The character recognition model pair obtained by training based on the character recognition model training method disclosed in the present invention is shown. Figure 4A The recognition result after character recognition on the work order screenshot in;

[0027] Figure 5 A schematic diagram showing a flow chart of a character recognition method in an exemplary embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram showing the structure of a training device for a character recognition model in an exemplary embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram showing the structure of a computer storage medium in an exemplary embodiment of the present disclosure; and

[0030] Figure 8 A schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0032] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0034] The work order system is an important means of network monitoring and operation for telecom operators. There are many professional work orders circulating in the work order system. During the processing of website access failure work orders (such as packet loss, high latency, etc.), the route trace provided by the user will appear, that is, a screenshot of the route trace information. Recognizing the characters in the screenshot can greatly improve work efficiency.

[0035] In the related art, the tesseract (a kind of character recognition software) framework is directly used to perform OCR (Optical Character Recognition) on the route trace screenshot.

[0036] However, this method of directly performing character recognition using a general character recognition framework has poor adaptability and low recognition accuracy for character recognition scenarios in work order screenshots.

[0037] In the embodiments of the present disclosure, a method for training a character recognition model is first provided, which improves the defects existing in the above-mentioned related technologies to at least a certain extent.

[0038] Figure 1 A flow chart showing a method for training a character recognition model in an exemplary embodiment of the present disclosure is shown. The character recognition model in the present disclosure is used to perform character recognition on an image showing routing trace information. Figure 1 , the method comprising:

[0039] Step S110, obtaining a sequence of character image blocks to be recognized corresponding to the routing trace image sample, and inputting the sequence of character image blocks to be recognized into a long short-term memory network to obtain a recognized text of the routing trace image sample based on a template character set;

[0040] Step S120, obtaining a word image block to be recognized corresponding to the routing trace image sample, and recognizing the word image block to be recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized;

[0041] Step S130, merging the recognition text and the character recognition result to determine the character recognition result of the routing trace image sample;

[0042] Step S140, determining the characters that are incorrectly recognized in the character recognition result, and re-marking the characters that are incorrectly recognized as correct characters to update the template character set;

[0043] Step S150, retraining the long short-term memory network and the adaptive classifier based on the routing trace image sample and the updated template character set respectively until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier.

[0044] exist Figure 1 In the technical solution provided by the illustrated embodiment, first, a sequence of character image blocks to be recognized corresponding to the routing trace image sample is obtained, and the sequence of character image blocks to be recognized is input into a long short-term memory network to obtain a recognition text of the routing trace image sample based on a template character set; then, a word image block to be recognized corresponding to the routing trace image sample is obtained, and the word image block to be recognized is recognized according to an adaptive classifier and a template character set to obtain a character recognition result of the word image block to be recognized; the recognition text and the character recognition result are merged to determine a character recognition result of the routing trace image sample; an incorrectly recognized character in the character recognition result is determined, and the incorrectly recognized character is re-marked as a correct character to update the template character set; and the long short-term memory network and the adaptive classifier are retrained based on the routing trace image sample and the updated template character set, respectively, until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier. Compared with the related art, on the one hand, the present invention obtains a character recognition model based on the joint training of a long short-term memory network and an adaptive classifier, which can improve the recognition accuracy of the trained character recognition model; on the other hand, the present invention re-labels the incorrectly recognized characters for retraining, which can improve the model training efficiency while ensuring the recognition accuracy of the model.

[0045] The following Figure 1 The specific implementation methods of each step in the embodiment shown are described in detail:

[0046] In step S110, a sequence of character image blocks to be recognized corresponding to the routing trace image sample is obtained, and the sequence of character image blocks to be recognized is input into a long short-term memory network to obtain a recognized text of the routing trace image sample based on a template character set.

[0047] The character recognition model in the present disclosure is used for character recognition in a routing trace image. The routing trace image can be understood as an image showing routing trace information, such as a screenshot of routing trace information provided by a user during network operation. The routing trace can be understood as information representing an electronic path, such as a storage location and an access address.

[0048] You can get a screenshot of the existing routing trace information and use it as a routing trace image sample to train the character recognition model.

[0049] Exemplarily, a specific implementation of generating a character image block to be recognized corresponding to a routing trace image sample may include: preprocessing the routing trace image sample; performing character connected area detection and character segmentation on the preprocessed routing trace image sample to obtain a character image block to be recognized corresponding to the routing trace image sample. The preprocessing may include processes such as image adaptive magnification and sharpening.

[0050] For example, the routing trace image sample can be enlarged to a preset size and sharpened to obtain a preprocessed routing trace image. Then, the preprocessed routing trace image is subjected to connected area detection to detect the overall character area (i.e., outline shape) and sub-outlines. Among them, the overall character area can be understood as the entire connected area showing character information in the routing trace image sample, and the sub-outline can be understood as the connected area corresponding to each character in the routing trace image sample. Punctuation, other symbols and noise points can be filtered out by comparing the sub-outline height and the overall character area height, and then the text line is determined by least squares fitting. Next, whether the characters are of equal width can be determined by whether the width of the connected domain on the same line of text image is approximately consistent. For equal-width text, this line of text can be directly divided into single characters by this width to obtain the character image block to be identified corresponding to the routing trace image sample. For text of unequal width, character segmentation can be performed by a certain interval and fuzzy interval to obtain a character image block.

[0051] After the character image blocks to be recognized are generated, a sequence of character image blocks to be recognized can be generated according to the front-to-back position relationship between each character image block to be recognized, and then the sequence of character image blocks to be recognized is input into the long short-term memory network, so as to recognize the sequence of character image blocks to be recognized based on the long short-term memory network and the template character set, so as to recognize the text corresponding to the input sequence of character image blocks to be recognized. Among them, the template character set can include characters commonly used in routing trace information and their corresponding templates, and the character template can be understood as the feature vector of the character.

[0052] For example, the feature vector of each character image block to be recognized can be identified through a long short-term memory network, and then the recognized feature vectors are compared with the feature vectors in the template character set for similarity, and the template character with the greatest similarity is selected as the recognition result of the current character block to be recognized, and finally the recognition text corresponding to the input sequence is obtained.

[0053] Since the routing trace information is of indefinite length, it is difficult to recognize it. The long short-term memory network, i.e., the LSTM (Long short-term memory) network, has a certain memory capacity. It can process information of any length in a time sequence. When LSTM is applied to text recognition tasks, each character in a word is an input time series, and the input at time t is the character vector of the t-th character in the word. Alternatively, the sequence composed of each word in a sentence is an input time series, and the input at time t is the t-th word vector of the sentence. Therefore, in the present disclosure, the recognized text of the routing trace image sample can also be obtained based on the sequence of word image blocks to be recognized corresponding to the routing trace image sample. This exemplary embodiment does not specifically limit this.

[0054] Next, in step S120, a word image block to be recognized corresponding to the routing trace image sample is obtained, and the word image block to be recognized is recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized.

[0055] For example, the word image blocks to be recognized corresponding to the routing trace image samples can be generated based on the above-mentioned method of generating the character image blocks to be recognized. For example, after segmenting the single characters, the text lines can be space recognized to perform word segmentation based on the continuous single characters between adjacent spaces, thereby determining multiple word image blocks to be recognized corresponding to the routing trace image samples based on the segmentation results.

[0056] The specific implementation of step S120 can refer to Figure 2 . For example, Figure 2 A schematic diagram of a method for secondary recognition based on an adaptive classifier in an exemplary embodiment of the present disclosure is shown. Figure 2, the method may include steps S210 to S230. Wherein:

[0057] In step S210, the word image blocks to be recognized are first recognized according to the adaptive classifier and the template character set to obtain an initial character recognition result for each word image block to be recognized.

[0058] For example, the adaptive classifier updated after the previous training process can be used to perform character recognition on each word image block to be recognized, so as to determine the initial character recognition result of each character in each word image block to be recognized. For example, the feature vector of each character in the word image block to be recognized and the template vector of each template character in the template character set can be calculated for similarity, and the template character with the greatest similarity can be determined as the recognition result of the current character.

[0059] Next, in step S220, the word image block to be recognized whose initial character recognition result meets the second preset condition is determined as a training sample, the character recognition result of the word image block to be recognized is determined as a label of the training sample, and the adaptive classifier is retrained.

[0060] In an exemplary embodiment, the second preset condition includes that the similarity between each character in the character recognition result corresponding to the to-be-recognized word image block and its candidate character in the template character set is greater than a second preset value.

[0061] The candidate character corresponding to each character in the template character set can be understood as the template character in the template character set that has the greatest similarity with the character.

[0062] For example, the reliably recognized words in the initial recognition results can be determined as training samples. The reliably recognized words may include words in which the similarity between each character and the corresponding candidate character is greater than a second preset value. Alternatively, when the ratio of the number of reliably recognized characters in the word to the total number of characters in the word is greater than a certain threshold, the word is determined to be a training sample. Reliable recognition here can be understood as the similarity between the character and its corresponding candidate character is greater than a second preset value. The determined training samples and their corresponding labels are then used to retrain the adaptive classifier to update the adaptive classifier. In this way, the recognition accuracy of the adaptive classifier can be improved.

[0063] Then, in step S230, the word image block to be recognized is recognized for the second time based on the template character set and the retrained adaptive classifier to obtain a character recognition result of the word image block to be recognized.

[0064] After retraining the adaptive classifier based on the reliable words, secondary character recognition can be performed on the image block of the word to be recognized based on the retrained adaptive classifier to determine the character recognition result of the image block of the word to be recognized recognized by the adaptive classifier.

[0065] Through the above steps S210 to S220, words with higher accuracy are first identified and used as training samples to perform secondary training on the adaptive classifier, so that the adaptive classifier after secondary training can effectively improve the recognition accuracy of complex characters.

[0066] In an exemplary embodiment, for step S110 and step S120, the case of the character can also be identified based on information such as the stroke height of the character, so as to determine the text recognition result of the long short-term network and the character recognition result of the adaptive classifier based on the identified character case.

[0067] Next, in step S130, the recognition text and the character recognition result are combined to determine the character recognition result of the routing trace image sample.

[0068] Reference below Figure 3 The specific implementation of step S130 is described. For example, Figure 3 A flow chart showing a method for determining a character recognition result of a routing trace image sample in an exemplary embodiment of the present disclosure is shown. Figure 3 , the method may include steps S310 to S330.

[0069] In step S310, the character recognition result is matched with the character in the corresponding position in the recognized text, and when the match is successful, the recognized character is determined to be the first target character.

[0070] Exemplarily, the character recognition result of each word image block to be recognized can be matched with the character at the corresponding position in the recognized text of the routing trace image sample according to the segmentation order of the segmented word image blocks to be recognized. When the match is successful, the recognized character at the position is determined to be the first target character.

[0071] Next, in step S320, when the matching fails, a character with a large first similarity is selected and determined as the recognized second target character, wherein the first similarity is used to characterize the degree of similarity between the recognized character and its candidate characters in the template character set.

[0072] For example, the similarity between the current recognized character in the character recognition result and its candidate character in the template character set can be obtained and used as the first similarity corresponding to the current recognized character in the character recognition result, and the similarity between the recognized character in the corresponding position of the recognized text and its candidate character in the template character set can be obtained and used as the first similarity of the current recognized character in the recognized text. Then, the first similarity corresponding to the current recognized character in the character recognition result and the first similarity corresponding to the current recognized character in the corresponding position of the recognized text are compared to determine the larger value between the two, and then the current recognized character corresponding to the larger value is determined as the recognized second target character.

[0073] In step S330, the first target character and the second target character are combined according to the positions of the characters in the character sequence to determine a character recognition result of the trace image sample.

[0074] For example, the characters in the recognized text recognized by the long short-term memory network can be determined as the first recognized characters, and the character recognition results of the image blocks of the words to be recognized recognized by the adaptive classifier can be determined as the second recognized characters. Then, the first recognized characters and the second recognized characters in the same position are matched. If the match is successful, it means that the recognition results are consistent, then the first recognized character or the second recognized character can be used as the final recognized character of the position. If the match fails, the recognized character with high recognition reliability is selected as the final recognized character of the position, and then according to the position of each character in the character sequence, the final recognized characters corresponding to each position are spliced ​​to determine the character recognition result of the trace image sample. Among them, the reliability can be measured by the above-mentioned first similarity.

[0075] Through the above-mentioned steps S310 to S330, the recognition accuracy of the final character recognition result can be improved. In this way, in the subsequent step S140, the recognition results that need to be corrected can be reduced, and the training speed of the model can be improved.

[0076] After the character recognition result of the routing trace image sample is determined, in step S140, the correct character corresponding to the incorrectly recognized character in the character recognition result is determined to update the template character set.

[0077] Exemplarily, the incorrectly recognized characters in the character recognition results can be manually determined based on the final character recognition results, and the correct characters corresponding to the incorrectly recognized characters can be determined, and then the correct characters can be added to the template character set to update the template character set.

[0078] Next, in step S150, the long short-term memory network and the adaptive classifier are retrained based on the routing trace image sample and the updated template character set respectively until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier.

[0079] In an exemplary embodiment, the first preset condition includes that the recognition accuracy of the long short-term memory network and the adaptive classifier reaches a first preset value.

[0080] For example, after determining the incorrectly recognized characters in the character recognition results, the accuracy of the current character recognition results can also be determined based on the incorrectly recognized characters. If the accuracy of the current character recognition results is greater than or equal to a first preset value, the training can be stopped. If it is less than the first preset value, the correct character corresponding to the incorrectly recognized character can be determined, and then the character is added to the template character set to update the template character set, and steps S110 to S130 are re-executed based on the updated template character set to re-stock the long short-term memory network and the adaptive classifier, and this cycle is repeated until the recognition accuracy of the character recognition result determined in step S130 is greater than or equal to the first preset value, and then the training is stopped.

[0081] By re-labeling the correct characters for the recognition errors to update the template character set, and then retraining based on the updated template character set, the recognition capabilities of the long short-term memory network and the adaptive classifier can be continuously improved, thereby improving the final recognition accuracy of the model.

[0082] In an exemplary embodiment, the recognition accuracy of the long short-term memory network and the adaptive classifier can be determined respectively according to their respective recognition results. When the recognition accuracy of the long short-term memory network reaches a first preset value and the recognition accuracy of the adaptive classifier does not reach the first preset value, the recognition text of the routing trace image sample is obtained based on the long short-term memory network, the recognition text is split into single characters, and the single characters are used as training labels of the adaptive classifier to perform supervised learning training on the adaptive classifier model; when the recognition accuracy of the adaptive classifier reaches the first preset value and the recognition accuracy of the long short-term memory network does not reach the first preset value, a character sequence is generated based on the character recognition result corresponding to the routing trace image sample obtained by the adaptive classifier, and the character sequence is used as the training label of the long short-term memory network to perform supervised learning training on the long short-term memory network.

[0083] For example, the recognition accuracy of the long short-term memory network and the character recognition results of the adaptive classifier can be determined respectively. When the recognition accuracy of the long short-term memory network is greater than the first preset value but the recognition accuracy of the adaptive classifier is not greater than the first preset value, the adaptive classifier can be trained separately. For example, the sequence of character image blocks to be recognized corresponding to the routing trace image sample can be input into the long short-term memory network to obtain the recognition text, and then the recognition text is split into characters, and the split characters are used as the true values ​​of the character image blocks at the corresponding positions, and then the adaptive classifier is trained separately with supervised learning based on the true value to make it converge as quickly as possible and achieve the expected recognition effect.

[0084] Similarly, when the recognition accuracy of the adaptive classifier reaches the first preset value but the recognition accuracy of the long short-term memory network does not reach the first preset value, the long short-term memory network can be trained separately. For example, the word image block to be recognized corresponding to the routing trace image sample is input into the adaptive classifier, and the character recognition result of each word block is obtained based on the adaptive classifier. Then, the character recognition result of each word image block can also be used as the true value of the corresponding position in the long short-term memory network. Based on the true value, the long short-term memory network is trained with supervised learning separately to make it converge as quickly as possible and achieve the expected recognition result.

[0085] In an exemplary application scenario, after the training of the long short-term memory network and the adaptive classifier is completed, they can be applied to the character recognition in other routing trace images. Specifically, the routing trace image to be processed can be obtained, and then the sequence of character image blocks to be recognized and the word image blocks to be recognized corresponding to the routing trace image to be processed can be obtained respectively, and they are input into the trained long short-term memory network and the adaptive classifier respectively, and then the recognition text and character recognition results are obtained respectively. The first recognition character in the recognition text at the same position and the second recognition character in the character recognition result are matched. When the two are matched successfully, the character recognized at the position is determined to be the first recognition character or the second recognition character. When the two fail to match, the character recognized at the position is determined to be a recognition character with a greater similarity to the corresponding candidate character, and then the characters in each position are spliced ​​in order to determine the final recognition character.

[0086] In the present disclosure, the sequence of character image blocks to be recognized in the routing trace image is used as the input time series of the long short-term memory network, which can better combine context information for character recognition. Through secondary recognition by the adaptive classifier, more complex texts can be recognized. The combination of the two can effectively improve the accuracy of routing trace image recognition. Figure 4A A screenshot of a work order to be identified in an exemplary embodiment of the present disclosure is shown. Figure 4B The character recognition model pair obtained by training based on the character recognition model training method disclosed in the present invention is shown. Figure 4A The recognition result after character recognition on the work order screenshot in Figure 4B and Figure 4A It can be seen from the comparison results that the character recognition model trained by the character recognition model training method disclosed in the present invention has a higher recognition accuracy rate for the routing trace image.

[0087] Figure 5 A flow chart of a character recognition method in an exemplary embodiment of the present disclosure is shown. Figure 5 , the method may include steps S510 to S590. In step S510, obtain a sample image corresponding to the screenshot of the sample work order trace information; in step S520, pre-process the sample image; in step S530, perform character recognition based on the adaptive classifier and the long short-term memory network; in step S540, determine whether the model recognition accuracy is greater than the preset value? If yes, go to step S590; if not, go to step S550; in step S550, obtain a sample image corresponding to the labeled screenshot of the sample routing trace information; in step S560, train the adaptive classifier; in step S570, train the long short-term memory network; in step S580, the model is output, and go to step S530; in step S590, perform character recognition on other images to be processed based on the trained model.

[0088] For example, after obtaining the sample image, the sample image can be preprocessed to enhance the clarity of the image, such as adaptively enlarging and sharpening the image. Then, a connected region analysis is performed to detect the overall character area (contour shape) and sub-contours, and the area surrounded by the contour line is integrated into a block area. Then, the text line can be derived from the character contour and the block area, and the word can be recognized by the space. Fixed-width text is segmented into single characters by character units, while non-fixed-width text lines can be segmented by certain intervals and fuzzy intervals. Then, long short-term memory networks and adaptive classifiers are used for character recognition respectively. Among them, the adaptive classifier analyzes and recognizes the words twice, that is, the words that meet the conditions for the first recognition accuracy are used as training samples, the adaptive classifier is trained, and then the trained adaptive classifier is used for secondary recognition. Next, lowercase letters are recognized based on stroke height to obtain the final recognition result. Finally, the characters that are misrecognized are manually re-labeled and added to the template character set, thereby updating the template character set to generate labeled training sample images, and the above steps S560 and S570 are executed to retrain, and character recognition is re-performed based on the retrained model until the final recognition accuracy of the model is greater than the preset value. After the training is completed, the trained model can be applied to the character recognition scenario in the actual route trace image.

[0089] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, the above functions defined by the above method provided by the present invention are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk.

[0090] In addition, it should be noted that the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0091] Figure 6 FIG. 1 is a schematic diagram showing the structure of a training device for a character recognition model in an exemplary embodiment of the present disclosure. Figure 6 The device 600 may include a text segment recognition module 610, a word character recognition module 620, a character recognition result determination module 630, a template character set update module 640, and a retraining module 650. Among them:

[0092] The text segment recognition module 610 is configured to obtain a sequence of character image blocks to be recognized corresponding to the routing trace image sample, and input the sequence of character image blocks to be recognized into a long short-term memory network to obtain a recognized text of the routing trace image sample based on a template character set;

[0093] The word character recognition module 620 is configured to obtain a word image block to be recognized corresponding to the routing trace image sample, and recognize the word image block to be recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized;

[0094] A character recognition result determination module 630 is configured to combine the recognition text and the character recognition result to determine a character recognition result of the routing trace image sample;

[0095] The template character set updating module 640 is configured to determine the correct character corresponding to the incorrectly recognized character in the character recognition result to update the template character set;

[0096] The retraining module 650 is configured to retrain the long short-term memory network and the adaptive classifier based on the routing trace image sample and the updated template character set respectively until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier.

[0097] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the above-mentioned character recognition result determination module 630 can be specifically configured as: matching the character recognition result with the character in the corresponding position in the recognized text, and when the match is successful, determining the recognized character as the first target character; when the match fails, selecting a character with a large first similarity and determining it as the recognized second target character, wherein the first similarity is used to characterize the degree of similarity between the recognized character and its candidate characters in the template character set; and merging the first target character and the second target character according to the position of the character in the character sequence to determine the character recognition result of the trace image sample.

[0098] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the first preset condition includes that the recognition accuracy of the long short-term memory network and the adaptive classifier reaches a first preset value.

[0099] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the device 600 further includes a first supervised learning training module and a second supervised learning training module. The first supervised learning training module is configured to: when the recognition accuracy of the long short-term memory network reaches a first preset value and the recognition accuracy of the adaptive classifier does not reach the first preset value, obtain the recognition text of the routing trace image sample based on the long short-term memory network, split the recognition text into single characters, and use the single characters as training labels of the adaptive classifier to perform supervised learning training on the adaptive classifier model. The second supervised learning training module is configured to: when the recognition accuracy of the adaptive classifier reaches the first preset value and the recognition accuracy of the long short-term memory network does not reach the first preset value, generate a character sequence based on the character recognition result corresponding to the routing trace image sample obtained by the adaptive classifier, and use the character sequence as the training label of the long short-term memory network to perform supervised learning training on the long short-term memory network.

[0100] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the word character recognition module 620 can be specifically configured as follows: performing a first recognition on the word image block to be recognized according to the adaptive classifier and the template character set to obtain an initial character recognition result for each word image block to be recognized; determining the word image block to be recognized whose initial character recognition result meets a second preset condition as a training sample, determining the character recognition result of the word image block to be recognized as a label of the training sample, and retraining the adaptive classifier; performing a second recognition on the word image block to be recognized based on the template character set and the retrained adaptive classifier to obtain a character recognition result of the word image block to be recognized.

[0101] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the second preset condition includes that the similarity between each character in the character recognition result corresponding to the word image block to be recognized and its candidate character in the template character set is greater than a second preset value.

[0102] In some exemplary embodiments of the present disclosure, based on the aforementioned embodiments, the device further includes a preprocessing module, and the preprocessing module is configured to: preprocess the routing trace image samples; perform character connected area detection and character segmentation on the preprocessed routing trace image samples to obtain character image blocks to be recognized or word image blocks to be recognized corresponding to the routing trace image samples.

[0103] The specific details of each unit in the above-mentioned character recognition model training device have been described in detail in the corresponding character recognition model training method, so they will not be repeated here.

[0104] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0105] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0106] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0107] In the exemplary embodiments of the present disclosure, a computer storage medium capable of implementing the above method is also provided. A program product capable of implementing the above method of this specification is stored thereon. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to perform the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0108] refer to Figure 7 As shown, a program product 700 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0109] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0110] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0111] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0112] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0113] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0114] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0115] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0116] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0117] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 1 The steps shown in .

[0118] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0119] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0120] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0121] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0122] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0123] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0124] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for training a character recognition model, wherein the character recognition model is used to perform character recognition on an image showing routing trace information, characterized in that: include: Obtaining a sequence of character image blocks to be recognized corresponding to a routing trace image sample, and inputting the sequence of character image blocks to be recognized into a long short-term memory network to obtain a recognized text of the routing trace image sample based on a template character set; Acquire a word image block to be recognized corresponding to the routing trace image sample, and recognize the word image block to be recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized; Combining the recognition text and the character recognition result to determine a character recognition result of the routing trace image sample; Determining a correct character corresponding to an incorrectly recognized character in the character recognition result to update the template character set; The long short-term memory network and the adaptive classifier are retrained based on the routing trace image sample and the updated template character set respectively until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier.

2. The character recognition model training method according to claim 1, characterized in that: The step of combining the recognition text and the character recognition result to determine the character recognition result of the routing trace image sample includes: Matching the character recognition result with the character in the corresponding position in the recognized text, and when the match is successful, determining that the recognized character is the first target character; When the match fails, a character with a large first similarity is selected and determined as the recognized second target character, wherein the first similarity is used to represent the degree of similarity between the recognized character and its candidate characters in the template character set; The first target character and the second target character are combined according to the position of the character in the character sequence to determine a character recognition result of the trace image sample.

3. The character recognition model training method according to claim 1, characterized in that: The first preset condition includes that the recognition accuracy of the long short-term memory network and the adaptive classifier reaches a first preset value.

4. The character recognition model training method according to claim 3, characterized in that: The method further comprises: When the recognition accuracy of the long short-term memory network reaches a first preset value and the recognition accuracy of the adaptive classifier does not reach the first preset value, obtaining a recognition text of the routing trace image sample based on the long short-term memory network, splitting the recognition text into single characters, and using the single characters as training labels of the adaptive classifier to perform supervised learning training on the adaptive classifier model; When the recognition accuracy of the adaptive classifier reaches the first preset value and the recognition accuracy of the long short-term memory network does not reach the first preset value, a character sequence is generated based on the character recognition result corresponding to the routing trace image sample obtained by the adaptive classifier, and the character sequence is used as a training label of the long short-term memory network to perform supervised learning training on the long short-term memory network.

5. The character recognition model training method according to claim 1, characterized in that: The step of recognizing the word image block to be recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized includes: Performing a first recognition on the word image block to be recognized according to the adaptive classifier and the template character set to obtain an initial character recognition result of each word image block to be recognized; Determine the to-be-recognized word image block whose initial character recognition result meets the second preset condition as a training sample, determine the character recognition result of the to-be-recognized word image block as a label of the training sample, and retrain the adaptive classifier; The word image block to be recognized is recognized for the second time based on the template character set and the retrained adaptive classifier to obtain a character recognition result of the word image block to be recognized.

6. The character recognition model training method according to claim 5, characterized in that: The second preset condition includes that the similarity between each character in the character recognition result corresponding to the word image block to be recognized and its candidate character in the template character set is greater than a second preset value.

7. The character recognition model training method according to claim 1, characterized in that: The method further comprises: Preprocessing the routing trace image sample; The pre-processed routing trace image samples are subjected to character connected area detection and character segmentation to obtain to-be-recognized character image blocks or to-be-recognized word image blocks corresponding to the routing trace image samples.

8. A training device for a character recognition model, the character recognition model being used for character recognition in a routing trace image, characterized in that: The device comprises: A text segment recognition module is configured to obtain a sequence of character image blocks to be recognized corresponding to a routing trace image sample, and input the sequence of character image blocks to be recognized into a long short-term memory network to obtain a recognized text of the routing trace image sample based on a template character set; A word character recognition module is configured to obtain a word image block to be recognized corresponding to the routing trace image sample, and recognize the word image block to be recognized according to the adaptive classifier and the template character set to obtain a character recognition result of the word image block to be recognized; a character recognition result determination module, configured to merge the recognition text and the character recognition result to determine a character recognition result of the routing trace image sample; A template character set updating module is configured to determine a correct character corresponding to an incorrectly recognized character in the character recognition result, so as to update the template character set; The retraining module is configured to retrain the long short-term memory network and the adaptive classifier based on the routing trace image sample and the updated template character set respectively until a first preset condition is met to obtain a target long short-term memory network and a target adaptive classifier.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method according to any one of claims 1 to 7.

Images