Artificial intelligence-based text recognition method and device, computer device and medium

Abstract

The application relates to the technical field of artificial intelligence, in particular to a text recognition method and device based on artificial intelligence, computer equipment and a medium. The method extracts a character bounding box and a reference bounding box of a to-be-recognized image and a template image, calculates an intersection and union ratio of the character bounding box and the closest reference bounding box, takes the average of all intersection and union ratios as a matching parameter, if the matching parameter meets a preset condition, corrects the to-be-recognized image according to a rotation parameter of the template image relative to a standard image, inputs a target image obtained through correction into a text recognition model, outputs recognized text, performs semantic association on the recognized text according to the association relationship of a standard text box in the standard image, corrects the recognized text according to a semantic association result, corrects the image according to the rotation parameter of the template image, and corrects the to-be-recognized text in combination with the semantic association result, so that the accuracy of text recognition is improved, and the processing accuracy of a financial service platform for financial text data is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a text recognition method and device based on artificial intelligence, a computer device and a medium. BACKGROUND

[0002] With the rise of artificial intelligence technology, text recognition technology based on an artificial intelligence model has been widely applied in financial service platforms, which can be insurance systems, bank systems, transaction systems, order systems, etc. The financial service platform can support functions such as shopping, socializing, interactive games, and resource transfer, and can also have functions such as applying for a loan, a credit card, or purchasing insurance, financial products, etc.

[0003] The text recognition task can provide a basis for intelligent financial text analysis and certificate information input in the financial service platform, and the financial text can include insurance documents, contracts, transaction records, etc., thereby effectively reducing labor costs and avoiding the decline in recognition accuracy caused by manual work.

[0004] However, the text recognition task can only recognize characters in natural text, and when an error occurs, it cannot achieve intelligent discovery and intelligent correction of the error recognition result, resulting in low accuracy of text recognition, and still requiring manual intervention for verification. The accuracy of text recognition will also be affected by subjective factors, resulting in low accuracy, therefore, how to improve the accuracy of text recognition has become a problem to be solved. SUMMARY

[0005] Therefore, the embodiments of the present application provide a text recognition method and device based on artificial intelligence, a computer device and a medium to solve the problem of low accuracy of text recognition.

[0006] In a first aspect, the embodiments of the present application provide a text recognition method based on artificial intelligence, which comprises:

[0007] Obtaining a to-be-recognized image, inputting the to-be-recognized image into a trained character positioning model, and outputting a character bounding box to which each recognized character in the to-be-recognized image belongs;

[0008] Obtaining N preset template images, for any template image, obtaining at least one reference bounding box of the template image, determining the reference bounding box closest to each character bounding box according to the distance between the character bounding box and the reference bounding box, calculating the intersection over union of each character bounding box and the reference bounding box closest to it, and taking the average of all intersection over unions as the matching parameter of the corresponding template image;

[0009] If the maximum matching parameter is greater than a preset threshold, a template image corresponding to the maximum matching parameter is taken as a reference image, the to-be-identified image is corrected according to a rotation parameter of the reference image relative to a preset standard image, and a correction result is determined as a target image;

[0010] The target image is input into a trained text recognition model, and at least one recognized text and a recognized text box thereof are output, the recognized texts in the recognized text boxes of each corresponding standard text box are semantically associated according to an association relationship between each standard text box in the standard image;

[0011] Each recognized text is corrected according to a semantic association result of each recognized text, and a corrected recognized text is obtained.

[0012] In a second aspect, an embodiment of the present application provides a text recognition device based on artificial intelligence, and the text recognition device comprises:

[0013] A character positioning module is configured to acquire a to-be-identified image, input the to-be-identified image into a trained character positioning model, and output a character bounding box to which each recognized character in the to-be-identified image belongs.

[0014] A bounding box matching module is configured to acquire N preset template images, acquire at least one reference bounding box of each template image, determine, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the character bounding box and the reference bounding box, calculate an intersection-over-union of each character bounding box and the reference bounding box closest to the character bounding box, and take an average of all intersection-overs-union as a matching parameter of the corresponding template image.

[0015] An image correction module is configured to, if the maximum matching parameter is greater than a preset threshold, take a template image corresponding to the maximum matching parameter as a reference image, correct the to-be-identified image according to a rotation parameter of the reference image relative to a preset standard image, and determine a correction result as a target image.

[0016] A semantic association module is configured to input the target image into a trained text recognition model, output at least one recognized text and a recognized text box thereof, and perform semantic association on recognized texts in recognized text boxes of each corresponding standard text box according to an association relationship between each standard text box in the standard image.

[0017] A text correction module is configured to correct each recognized text according to a semantic association result of each recognized text, and obtain a corrected recognized text.

[0018] In a third aspect, an embodiment of the present application provides a computer device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the artificial intelligence-based text recognition method according to the first aspect when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the artificial intelligence-based text recognition method according to the first aspect.

[0020] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0021] The image to be recognized is obtained, the image to be recognized is input into the trained character positioning model, and a character bounding box to which each recognized character in the image to be recognized belongs is output. N preset template images are obtained. For any template image, at least one reference bounding box of the template image is obtained. The reference bounding box closest to each character bounding box is determined according to the distance between the character bounding box and the reference bounding box. The intersection over union of each character bounding box and the reference bounding box closest to the character bounding box is calculated. The mean of all intersection over unions is taken as a matching parameter of the corresponding template image. If the largest matching parameter is greater than a preset threshold, the template image corresponding to the largest matching parameter is taken as a reference image. The image to be recognized is corrected according to the rotation parameter of the reference image relative to the preset standard image, and a correction result is determined as a target image. The target image is input into the trained text recognition model, and at least one recognized text and a recognized text box thereof are output. According to the correlation between the standard text boxes in the obtained standard image, the recognized texts in the recognized text boxes corresponding to the standard text boxes are semantically associated. According to the semantic association results of the recognized texts, the recognized texts are corrected to obtain corrected recognized texts. The template image corresponding to the image to be recognized is screened out in the form of bounding box matching, so that the image to be recognized can be corrected according to the rotation parameters of the template image and the standard image. The semantic association information of the recognized texts is mined by applying the correlation information of the standard image, the recognized texts are structurally represented, the recognized texts are corrected in combination with the semantic association results, the situation of recognition error is effectively reduced, the accuracy of text recognition is improved, and the processing accuracy of the financial service platform for financial text data is improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0023] Figure 1 is an application environment schematic diagram of a text recognition method based on artificial intelligence provided by the embodiment one of the present application;

[0024] Figure 2 is a flow schematic diagram of a text recognition method based on artificial intelligence provided by the embodiment one of the present application;

[0025] Figure 3 is a structure schematic diagram of a text recognition device based on artificial intelligence provided by the embodiment two of the present application;

[0026] Figure 4 is a structure schematic diagram of a computer device provided by the embodiment three of the present application. DETAILED DESCRIPTION

[0027] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0028] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It should also be understood that the term "and / or" as used herein refers to any combination of associated listed items, and all possible combinations, and includes these combinations.

[0030] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]," depending on the context.

[0031] In addition, in the description of the application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0032] In the description of the application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0033] Embodiments of the application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0034] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0035] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0036] In order to illustrate the technical solutions of the application, the following will be described by specific embodiments. In order to illustrate the technical solutions of the application, the following will be described by specific embodiments.

[0037] The application embodiment one provides a text recognition method based on artificial intelligence, which can be applied to the application environment as shown in Figure 1 The client and the server communicate with each other. The client includes but is not limited to a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (PDA), and the like. The server can be a stand-alone server or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and the like.

[0038] The client and the server can be deployed in a financial service platform, which can be an insurance system, a bank system, a transaction system, an order system, and the like. The financial service platform can support shopping, socializing, interactive gaming, resource transfer, and the like, and can also have functions of applying for a loan, a credit card, or purchasing insurance, financial products, and the like. The server can provide text recognition services to provide a basis for functions of archiving financial texts, querying financial texts, recommending texts, and inputting identity information in the financial service platform, and improve the accuracy of processing text data in the financial service platform.

[0039] Referring to Figure 2 , it is a flowchart of the text recognition method based on artificial intelligence provided by the application embodiment one. The text recognition method can be applied to the server in Figure 1 The server corresponds to a computer device connected to the client to obtain a to-be-recognized image input by a user from the client. The server corresponds to a computer device deployed with a trained character positioning model and a trained text recognition model. The trained character positioning model can be used to determine the position information of characters in the input image, and the text recognition model can be used to recognize the character information contained in the input image. As shown in Figure 2 The text recognition method can include the following steps:

[0040] Step S201: obtaining a to-be-recognized image, inputting the to-be-recognized image into the trained character positioning model, and outputting a character bounding box to which each recognized character in the to-be-recognized image belongs.

[0041] The to-be-recognized image can be an image that needs to be subjected to text recognition. In this embodiment, the to-be-recognized image can be a contract image, an insurance policy image, a transaction record image, an identity card photo, or the like that contains financial text information. The trained character positioning model can be used to determine position information of each recognized character contained in the to-be-recognized image. The to-be-recognized image can include at least one recognized character. The character bounding box can be used to represent the position information of the corresponding recognized character. The character bounding box can be represented by a set of coordinate pairs, including the coordinates of the top-left corner of the character bounding box and the coordinates of the bottom-right corner of the character bounding box.

[0042] Specifically, the input of the trained character positioning model is the to-be-recognized image. The trained character recognition model can include an encoder and a fully connected layer. The encoder can be used to extract image features of the to-be-recognized image. The fully connected layer can be used to map the extracted image features to an output space. In this embodiment, the output space can be a character bounding box vector space of a recognized character.

[0043] It should be noted that the recognized character in this embodiment refers to a character region, that is, the recognized character region can include at least one character. Correspondingly, the character bounding box can refer to a bounding box of the corresponding recognized character region.

[0044] Optionally, the to-be-recognized image is obtained, and the to-be-recognized image is input into the trained character positioning model to output a character bounding box to which each recognized character in the to-be-recognized image belongs.

[0045] The to-be-recognized image is input into the trained classification model to output a classification result.

[0046] When the classification result meets a preset category, the to-be-recognized image is input into the trained character positioning model to output a character bounding box to which each recognized character in the to-be-recognized image belongs.

[0047] The trained classification model can be used to predict the category of the input to-be-recognized image. In this embodiment, the text recognition task can be a birth certificate recognition task. Correspondingly, the classification result can be a binary classification. One classification result can be a birth certificate category, and the other classification result can be another category. The preset category can be the birth certificate category.

[0048] Specifically, images of a plurality of birth certificates can be collected as positive samples, images of a plurality of other texts can be collected as negative samples, the label corresponding to the positive sample can be the birth certificate category, and the label corresponding to the negative sample can be other categories. In this embodiment, it is suggested that the number of positive samples and the number of negative samples are similar to avoid overfitting of the classification model during training. The classification model is trained according to the positive samples and the labels corresponding thereto, the negative samples and the labels corresponding thereto, and a loss function during training, to obtain a trained classification model. The loss function during training can use a binary cross-entropy loss function.

[0049] In this embodiment, the classification model is used to predict the category of the to-be-recognized image in advance, so as to eliminate irrelevant to-be-recognized images and improve the pertinence of the subsequent text recognition process, that is, only for to-be-recognized images of a specific category, thereby improving the accuracy of text recognition.

[0050] The above step of obtaining the to-be-recognized image, inputting the to-be-recognized image into the trained character positioning model, and outputting the character bounding box to which each recognized character in the to-be-recognized image belongs, extracts the character bounding box, provides a reference for subsequent bounding box matching and text semantic association, and thus improves the accuracy of text recognition.

[0051] In step S202, N preset template images are obtained. For any template image, at least one reference bounding box of the template image is obtained. The reference bounding box closest to each character bounding box is determined according to the distance between the character bounding box and the reference bounding box. The intersection over union of each character bounding box and the reference bounding box closest thereto is calculated. The average of all intersection over unions is used as a matching parameter of the corresponding template image.

[0052] In this embodiment, the birth certificate text recognition is taken as an example. The template image can be a birth certificate text template. N is an integer greater than zero. The N template images can be template images of different image angles. The reference bounding box can be a bounding box of a character region in the corresponding template image. The distance between the character bounding box and the reference bounding box can represent the position difference between the character bounding box and the reference bounding box. The intersection over union can be used to represent the degree of coincidence between the character bounding box and the reference bounding box. The matching parameter can be used to measure the matching degree between the character bounding box of the to-be-recognized image and the reference bounding box of the template image.

[0053] Specifically, during the collection of the to-be-recognized image, the text original is not necessarily placed strictly according to the preset orientation. If the to-be-recognized image is directly compared with the template image in the preset orientation, an error comparison may occur.

[0054] Therefore, during the template image acquisition, the template text original is placed in different orientations to acquire N template images in different image angles, i.e., the template images in different orientations.

[0055] The calculation of the intersection-over-union ratio is specifically as follows: the intersection of the character bounding box and the reference bounding box is calculated, the number of pixel points contained in the intersection is counted, the union of the character bounding box and the reference bounding box is calculated, the number of pixel points contained in the union is counted, and the ratio of the number of pixel points contained in the intersection to the number of pixel points contained in the union is calculated. The ratio is the intersection-over-union ratio of the character bounding box and the reference bounding box.

[0056] Since the value range of the intersection-over-union ratio is [0, 1], the mean value range of all intersection-over-union ratios is also [0, 1], i.e., the value range of the matching parameter is [0, 1]. The closer the matching parameter is to 1, the greater the matching degree of the bounding boxes of the to-be-recognized image and the template image. The closer the matching parameter is to 0, the smaller the matching degree of the bounding boxes of the to-be-recognized image and the template image.

[0057] Optionally, the reference bounding box closest to each character bounding box is determined according to the distance between the character bounding box and the reference bounding box, and the method comprises the following steps.

[0058] The center point of each character bounding box and the center point of each reference bounding box are obtained.

[0059] The reference bounding box closest to each character bounding box is determined according to the distance between the center point of the character bounding box and the center point of the reference bounding box.

[0060] The center point can be used to represent the position information of the bounding box, and the center point can be calculated according to the coordinates of the top-left corner point and the bottom-right corner point of the bounding box.

[0061] Specifically, the horizontal coordinates of the center point are obtained by mean calculation according to the horizontal coordinates of the top-left corner point and the horizontal coordinates of the bottom-right corner point, and the vertical coordinates of the center point are obtained by mean calculation according to the vertical coordinates of the top-left corner point and the vertical coordinates of the bottom-right corner point.

[0062] For any character bounding box, the center point coordinates of the character bounding box are respectively subjected to Euclidean distance calculation with the center point coordinates of each reference bounding box to obtain the distance of the corresponding reference bounding box. The maximum value in all distances is taken, and the reference bounding box corresponding to the maximum value is the reference bounding box closest to the character bounding box.

[0063] The embodiment simplifies the representation form of the position of the bounding box through the center point, and makes the distance between the bounding boxes easy to measure, so that the distance difference between the bounding boxes can be efficiently and quickly measured, the calculation efficiency is improved, and the efficiency of the entire text recognition process is improved.

[0064] Optionally, determining, for each character bounding box, a reference bounding box closest to the character bounding box according to distances between the character bounding box and the reference bounding boxes comprises:

[0065] determining all the reference bounding boxes as candidate bounding boxes, calculating distances between the character bounding box and all the candidate bounding boxes respectively, and determining a candidate bounding box corresponding to a maximum distance as the reference bounding box closest to the character bounding box;

[0066] adjusting the candidate bounding box corresponding to the maximum distance as a non-candidate bounding box;

[0067] iterating through all the character bounding boxes to obtain the reference bounding box closest to each character bounding box, wherein the reference bounding boxes corresponding to any two character bounding boxes are different.

[0068] The candidate bounding box can refer to a reference bounding box that can be used for matching, and the non-candidate bounding box can refer to a reference bounding box that already has a matching object.

[0069] Specifically, when a candidate bounding box is determined to be closest to a character bounding box, the candidate bounding box will be changed to a non-candidate bounding box, that is, the candidate bounding box will no longer participate in the closest reference bounding box screening of subsequent character bounding boxes, so that each reference bounding box has at most one corresponding character bounding box.

[0070] In the embodiment, by setting the candidate bounding box and the non-candidate bounding box, the reference bounding boxes corresponding to any two character bounding boxes are different, which avoids the establishment of incorrect corresponding relationship, improves the accuracy of the matching parameter calculation, and further improves the accuracy of the entire text recognition process.

[0071] The step of obtaining N preset template images, obtaining at least one reference bounding box of each template image, determining, for each character bounding box, a reference bounding box closest to the character bounding box according to distances between the character bounding box and the reference bounding boxes, calculating an intersection-over-union of each character bounding box and the reference bounding box closest to the character bounding box, and taking an average of all the intersection-overs-union as a matching parameter of the corresponding template image, and quantifying the matching degree between the template image and the image to be recognized by the matching parameter, thereby providing a reliable template image as a reference for subsequent image-to-be-recognized correction, and further improving the accuracy of text recognition.

[0072] In step S203, if the maximum matching parameter is greater than a preset threshold, a template image corresponding to the maximum matching parameter is taken as a reference image, the image to be recognized is corrected according to a rotation parameter of the reference image relative to a preset standard image, and a correction result is determined as a target image.

[0073] The preset threshold value can be used to determine whether the to-be-identified image and the template image closest to the to-be-identified image are matched enough, and further determine whether they belong to the same image angle. The reference image can be used to determine the parameter of the to-be-identified image correction, that is, the rotation parameter. The rotation parameter can be represented by a matrix. The standard image can refer to the template image conforming to the preset orientation. The target image can refer to the to-be-identified image after correction.

[0074] Specifically, the parameter of the to-be-identified image correction can be represented by a transformation matrix, which includes a rotation matrix and a translation matrix. In this embodiment, since only the image angle changes when the default image is collected, only the rotation matrix is used as the rotation parameter for image correction.

[0075] Optionally, the correction of the to-be-identified image according to the rotation parameter of the reference image relative to the preset standard image includes:

[0076] Multiplying the inverse of the reference image and the standard image to obtain the rotation parameter.

[0077] Multiplying the to-be-identified image and the rotation parameter to obtain the correction result.

[0078] The reference image can be regarded as a matrix composed of pixel points, and then the inverse calculation can be performed. The calculation result is the inverse of the reference image.

[0079] Specifically, the inverse calculation is performed on the reference image, the inverse of the reference image is multiplied by the standard image to obtain the transformation matrix. It is assumed that the transformation matrix only includes the rotation parameter. The to-be-identified image is multiplied by the transformation matrix to obtain the rotated to-be-identified image, that is, the correction result.

[0080] If the maximum matching parameter is greater than the preset threshold value, the template image corresponding to the maximum matching parameter is used as the reference image. The to-be-identified image is corrected according to the rotation parameter of the reference image relative to the preset standard image, and the correction result is determined as the target image. The to-be-identified image is corrected to obtain the target image, so as to avoid errors in subsequent text recognition caused by different image directions, and improve the accuracy of text recognition.

[0081] In step S204, the target image is input into the trained text recognition model, and at least one recognized text and its recognized text box are output. According to the association relationship between the standard text boxes in the standard image, the recognized texts in the recognized text boxes corresponding to the standard text boxes are associated in semantics.

[0082] The trained text recognition model can be used to extract text information in the target image and mark the position thereof, the recognized text can refer to the text information extracted from the target image, the recognized text box can represent the position of the text information, and the association relationship can refer to the preset association rule.

[0083] Specifically, taking a birth certificate as an example, in general, the parent surname and the child surname have an association relationship, that is, the child surname is usually the same as one of the parent surnames, and the association relationship can be constructed as a preset association rule. The birth certificate number, the child's birthplace, the child's birth date, and the like have an association relationship, and the association relationship can also be constructed as a preset association rule.

[0084] Optionally, according to the association relationship between the standard text boxes in the standard image, the recognized text in the recognized text box corresponding to each standard text box is semantically associated.

[0085] For any recognized text box, the standard text box corresponding to the recognized text box in the standard image is determined, and the association relationship between the standard text box and other standard text boxes is obtained.

[0086] According to the association relationship between the standard text box and other standard text boxes and the correspondence between the standard text box and the recognized text box, the recognized text in the recognized text box is semantically associated with the recognized text in other recognized text boxes.

[0087] The standard text box can refer to a bounding box of text in the standard image, and the association relationship between the standard text box and other standard text boxes is the preset association rule.

[0088] Specifically, after the to-be-recognized image is corrected, it can be considered that each recognized text box in the to-be-recognized image has a standard text box with the same position in the standard image. The semantic association can refer to establishing a semantic association relationship, structuring the recognized text, and each recognized text can have other recognized text having a semantic association relationship therewith.

[0089] In this embodiment, the to-be-recognized image and the standard image are associated, and the standard image has a preset association rule, so that the recognized text is semantically associated and structured, which facilitates subsequent correction of the recognized text according to the semantic association result and improves the accuracy of text recognition.

[0090] The step of inputting the target image into the trained text recognition model to output at least one recognized text and a recognized text box thereof, and performing semantic association on the recognized text in each corresponding standard text box according to the association relationship between the standard text boxes in the standard image, can improve the accuracy of text recognition by mining semantic association rules of specific text and structuring the recognized text, thereby facilitating subsequent correction of the recognized text.

[0091] In step S205, each recognized text is corrected according to the semantic association result of each recognized text to obtain a corrected recognized text.

[0092] The semantic association result can refer to all other recognized texts having a semantic association relationship with any recognized text, and the corrected recognized text can be used as the final output of the text recognition task.

[0093] Optionally, the step of correcting each recognized text according to the semantic association result of each recognized text includes:

[0094] For any recognized text, other recognized texts having a semantic association with the recognized text and the recognized text are spliced, and the spliced result is input into the trained evaluation model to output a confidence of the recognized text.

[0095] If the confidence is less than a preset confidence threshold, key words are extracted from each other recognized text having a semantic association with the recognized text to obtain at least one key word.

[0096] According to the position information of each key word in the other recognized text to which the key word belongs, the position information of each key word in the recognized text is obtained from a preset mapping table to obtain at least one set of key words and characters in the recognized text having a corresponding relationship.

[0097] Each key word is used to replace the corresponding character in the recognized text to determine the replaced recognized text as the modified recognized text.

[0098] The spliced result can be represented in the form of a character vector, and the trained evaluation model can be used to evaluate the semantic association degree of the input vector, which is represented by the confidence.

[0099] The preset confidence threshold can be used to determine whether the semantic association is valid, the key word can refer to a character in the recognized text related to the association rule, the position information of the key word in the recognized text can be represented by a character serial number, and the preset mapping table can include a mapping relationship between the position information of the key word and the character position in the recognized text.

[0100] Specifically, if the confidence is less than a preset confidence threshold, it indicates that the identified text and other identified texts associated with the semantic thereof cannot meet the preset semantic association rule, and thus the identified text needs to be corrected according to information of the other identified texts. In this embodiment, the preset confidence threshold can be set to 0.8, and the implementer can adjust the preset confidence threshold according to actual conditions.

[0101] During keyword extraction, the keywords can be extracted according to a preset extraction rule. For example, for a name, the first character or the first two characters are usually taken as a surname keyword. The character sequence number of the keyword of the parent's surname can be 1, and the character position corresponding to the child's surname is the first character.

[0102] The trained evaluation model can be trained according to the actual semantic association relationship. The characters that meet the semantic association relationship are spliced as training positive samples, and the characters that do not meet the semantic association relationship are spliced as training negative samples. The label is a confidence label. The confidence label of the training positive sample is 1, and the confidence label of the training negative sample is 0. According to the training positive sample, the training negative sample, and the respective confidence labels, the evaluation model is trained using the mean square error loss function. It should be noted that according to different objects of the to-be-recognized text, the implementer can train the corresponding evaluation model according to the semantic association relationship of the corresponding to-be-recognized text object, so as to improve the accuracy of the evaluation.

[0103] The above step of correcting each identified text according to the semantic association result of each identified text to obtain the corrected identified text further corrects the identified text according to the semantic information, thereby improving the accuracy of the semantic of the identified text, that is, improving the accuracy of the text recognition.

[0104] In this embodiment, the template image corresponding to the to-be-recognized image is selected through the bounding box matching form, so that the to-be-recognized image can be corrected according to the rotation parameters of the template image and the standard image, the semantic association information of the identified text is mined by applying the association information of the standard image, the identified text is structuredly represented, the identified text is corrected according to the semantic association result, the situation of recognition error is effectively reduced, the accuracy of the text recognition is improved, and the processing accuracy of the financial service platform for the financial text data is improved.

[0105] The text recognition method based on artificial intelligence according to the above embodiment, Figure 3A structural block diagram of the text recognition device based on artificial intelligence provided by the second embodiment of the present application is shown. The text recognition device is applied to a server. A computer device corresponding to the server is connected to a client to obtain a to-be-recognized image input by a user from the client. The computer device corresponding to the server is deployed with a trained character positioning model and a trained text recognition model. The trained character positioning model can be used to determine position information of characters in the input image. The text recognition model can be used to recognize character information contained in the input image. For the convenience of description, only parts related to the embodiments of the present application are shown.

[0106] Referring to Figure 3 The text recognition device comprises:

[0107] The character positioning module 31 is configured to obtain the to-be-recognized image, input the to-be-recognized image into the trained character positioning model, and output a character bounding box to which each recognized character in the to-be-recognized image belongs.

[0108] The bounding box matching module 32 is configured to obtain N preset template images, obtain at least one reference bounding box of each template image, determine, for each character bounding box, a reference bounding box closest to the character bounding box according to the distance between the character bounding box and the reference bounding box, calculate the intersection over union of each character bounding box and the reference bounding box closest to the character bounding box, and take the average of all intersection over unions as a matching parameter of the corresponding template image.

[0109] The image correction module 33 is configured to, if the largest matching parameter is greater than a preset threshold, take the template image corresponding to the largest matching parameter as a reference image, correct the to-be-recognized image according to the rotation parameter of the reference image relative to a preset standard image, and determine a correction result as a target image.

[0110] The semantic association module 34 is configured to input the target image into the trained text recognition model, output at least one recognized text and a recognized text box thereof, and perform semantic association on the recognized text in each corresponding standard text box according to the association relationship between the standard text boxes in the obtained standard image.

[0111] The text correction module 35 is configured to correct each recognized text according to the semantic association result of each recognized text to obtain a corrected recognized text.

[0112] Optionally, the character positioning module 31 comprises:

[0113] The image classification unit is configured to input the to-be-recognized image into the trained classification model and output a classification result.

[0114] The category determining unit is configured to input the to-be-recognized image into the trained character positioning model when the classification result meets a preset category, and output a character bounding box to which each recognized character in the to-be-recognized image belongs.

[0115] Optionally, the bounding box matching module 32 includes:

[0116] The center point obtaining unit is configured to obtain a center point of each character bounding box and a center point of each reference bounding box.

[0117] The distance measuring unit is configured to determine, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the center point of the character bounding box and the center point of the reference bounding box.

[0118] Optionally, the bounding box matching module 32 includes:

[0119] The distance calculating unit is configured to determine all the reference bounding boxes as candidate bounding boxes, calculate, for any character bounding box, distances between the character bounding box and all the candidate bounding boxes, and determine, as the reference bounding box closest to the character bounding box, a candidate bounding box corresponding to a maximum distance.

[0120] The attribute adjusting unit is configured to adjust the candidate bounding box corresponding to the maximum distance as a non-candidate bounding box.

[0121] The bounding box traversing unit is configured to traverse all the character bounding boxes to obtain the reference bounding box closest to each character bounding box, wherein the reference bounding boxes corresponding to any two character bounding boxes are different.

[0122] Optionally, the image correction module 33 includes:

[0123] The parameter calculating unit is configured to multiply the inverse of the reference image by the standard image to obtain a rotation parameter.

[0124] The parameter correcting unit is configured to multiply the to-be-recognized image by the rotation parameter to obtain a correction result.

[0125] Optionally, the semantic association module 34 includes:

[0126] The association obtaining unit is configured to, for any recognized text box, determine a standard text box corresponding to the recognized text box in the standard image, and obtain an association relationship between the standard text box and other standard text boxes.

[0127] The semantic unit is configured to, according to the association relationship between the standard text box and other standard text boxes and a correspondence relationship between the standard text box and the recognized text box, perform semantic association between recognized texts in the recognized text box and recognized texts in other recognized text boxes.

[0128] Optionally, the text correction module 35 comprises:

[0129] a text evaluation unit configured to, for any one of the recognized texts, splice the other recognized texts having semantic association with the recognized text and the recognized text, input the spliced result into the trained evaluation model, and output a confidence of the recognized text;

[0130] a keyword extraction unit configured to, if the confidence is less than a preset confidence threshold, extract keywords from the other recognized texts having semantic association with the recognized text respectively, and obtain at least one keyword;

[0131] a corresponding relationship determination unit configured to, according to position information of each keyword in the other recognized text to which the keyword belongs, obtain character positions in the recognized text corresponding to the position information of each keyword from a preset mapping table, and obtain at least one group of keywords and characters in the recognized text having a corresponding relationship;

[0132] a character replacement unit configured to replace the corresponding character in the recognized text with each keyword, and determine the replaced recognized text as the modified recognized text.

[0133] It should be noted that the information interaction and execution process between the above modules and units are based on the same concept as the method embodiments of the present application, and the specific functions and technical effects brought about can be referred to the method embodiments part, which will not be repeated here.

[0134] Figure 4 A structural schematic diagram of a computer device is provided for the third embodiment of the present application. As shown in the figure, Figure 4 the computer device of this embodiment comprises at least one processor (only one is shown in the figure), a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor executes the computer program to implement the steps in any one of the above text recognition method embodiments. Figure 4 The computer device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand,

[0135] that the above is only an example of the computer device and does not constitute a limitation on the computer device. The computer device can include more or fewer components than those shown, or combine certain components, or different components, for example, it can also include a network interface, a display screen, and an input device, etc. Figure 4

[0136] ​The processor can be a CPU, and can also be other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0137] The memory includes a readable storage medium, an internal memory, etc., where the internal memory can be a memory of the computer device, and the internal memory provides an environment for running the operating system and the computer-readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the computer device, and in other embodiments, can also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory can include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above device can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program to instruct related hardware. The computer program can be stored in a computer readable storage medium, and when the processor executes the computer program, the steps of the above-mentioned method embodiment can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium can at least include any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0139] The present application realizes all or part of the processes in the above-mentioned embodiment methods, which can also be completed by a computer program product. When the computer program product runs on the computer device, it makes the computer device execute the steps that can realize the above-mentioned method embodiments.

[0140] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0141] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other manners. For example, the described apparatus / computer device embodiments are merely schematic. For example, the division of the modules or units can be different, and each can include a plurality of sub-units. Some or all of the modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e. may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0144] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An artificial intelligence-based text recognition method, characterized by, The text recognition method comprises: acquiring a to-be-recognized image, inputting the to-be-recognized image into a trained character positioning model, and outputting a character bounding box to which each recognized character in the to-be-recognized image belongs; acquiring N preset template images, for any template image, acquiring at least one reference bounding box of the template image, determining, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the character bounding box and the reference bounding box, calculating an intersection-over-union of each character bounding box and the reference bounding box closest to the character bounding box, and taking an average of all intersection-overs-unions as a matching parameter of the corresponding template image; if a maximum matching parameter is greater than a preset threshold, taking the template image corresponding to the maximum matching parameter as a reference image, correcting the to-be-recognized image according to a rotation parameter of the reference image relative to a preset standard image, and determining a correction result as a target image; inputting the target image into a trained text recognition model, outputting at least one recognized text and a recognized text box thereof, and performing semantic association on recognized texts in recognized text boxes corresponding to each standard text box according to an association relationship between the standard text boxes in the standard image; correcting each recognized text according to a semantic association result of each recognized text to obtain a corrected recognized text.

2. The text recognition method of claim 1, wherein, The acquiring a to-be-recognized image, inputting the to-be-recognized image into a trained character positioning model, and outputting a character bounding box to which each recognized character in the to-be-recognized image belongs comprises: inputting the to-be-recognized image into a trained classification model and outputting a classification result; when the classification result meets a preset category, inputting the to-be-recognized image into the trained character positioning model and outputting a character bounding box to which each recognized character in the to-be-recognized image belongs.

3. The text recognition method of claim 1, wherein, The determining, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the character bounding box and the reference bounding box comprises: acquiring a center point of each character bounding box and a center point of each reference bounding box; determining, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the center point of the character bounding box and the center point of the reference bounding box.

4. The text recognition method of claim 1, wherein, The determining, for each character bounding box, a reference bounding box closest to the character bounding box according to a distance between the character bounding box and the reference bounding box comprises: determining all reference bounding boxes as to-be-selected bounding boxes, for any character bounding box, calculating distances between the character bounding box and all to-be-selected bounding boxes, and determining a to-be-selected bounding box corresponding to a maximum distance as a reference bounding box closest to the character bounding box; adjusting the to-be-selected bounding box corresponding to the maximum distance as a non-to-be-selected bounding box; iterating through all character bounding boxes to obtain a reference bounding box closest to each character bounding box, wherein reference bounding boxes corresponding to any two character bounding boxes are different.

5. The text recognition method of claim 1, wherein, The correcting the to-be-recognized image according to a rotation parameter of the reference image relative to a preset standard image comprises: multiplying the inverse of the reference image by the standard image to obtain the rotation parameter; multiplying the to-be-recognized image by the rotation parameter to obtain the correction result.

6. The text recognition method of claim 1, wherein, The semantic association of the recognition texts in the recognition text boxes according to the semantic association results of the recognition texts comprises: For any recognition text, other recognition texts having semantic association with the recognition text are spliced with the recognition text, and a spliced result is input into a trained evaluation model to output a confidence degree of the recognition text; If the confidence degree is less than a preset confidence threshold, key words are extracted from other recognition texts having semantic association with the recognition text respectively to obtain at least one key word; 7. The text recognition method of any one of claims 1 to 6, characterized in that, According to position information of each key word in the other recognition texts, position information of each key word in the recognition text is obtained from a preset mapping table to obtain at least one group of key words and characters in the recognition text having a corresponding relationship; Each key word is used to replace a corresponding character in the recognition text to determine a replaced recognition text as the modified recognition text. The text recognition device comprises: A character positioning module is configured to obtain a to-be-recognized image, input the to-be-recognized image into a trained character positioning model, and output a character bounding box to which each recognition character in the to-be-recognized image belongs; A bounding box matching module is configured to obtain N preset template images, obtain at least one reference bounding box of any template image, determine a reference bounding box closest to each character bounding box according to a distance between the character bounding box and the reference bounding box, calculate an intersection-over-union of each character bounding box and the reference bounding box closest to the character bounding box, and take an average of all intersection-overs-union as a matching parameter of the corresponding template image; 8. An artificial intelligence-based text recognition apparatus, characterized by comprising: An image correction module is configured to take a template image corresponding to the largest matching parameter as a reference image if the largest matching parameter is greater than a preset threshold, correct the to-be-recognized image according to a rotation parameter of the reference image relative to a preset standard image, and determine a correction result as a target image; A semantic association module is configured to input the target image into a trained text recognition model, output at least one recognition text and a recognition text box thereof, and perform semantic association of recognition texts in recognition text boxes corresponding to each standard text box according to an association relationship between each standard text box in the standard image and other standard text boxes; A text correction module is configured to correct each recognition text according to a semantic association result of the recognition text to obtain a modified recognition text. ​ ​ ​ 9. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the processor implements the text recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the text recognition method according to any one of claims 1 to 7.

Images