Optical character recognition method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an optical character recognition method and device, electronic equipment and a readable storage medium, and the method comprises the steps: inputting a target image comprising a to-be-recognized character and a target language type list into an optical character recognition model, and recognizing the to-be-recognized character, a first recognition result output by the optical character recognition model is obtained, and the target language type list corresponds to the target image; based on context information corresponding to a target position in the first recognition result, a target prediction word corresponding to the target position is determined, the target position is the position of a target word in the first recognition result, and the target word is any word in the first recognition result; and filling the target position with the target prediction word to obtain a second recognition result.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an optical character recognition method, device, electronic device and readable storage medium. Background Art

[0002] Optical Character Recognition (OCR) is a process that converts printed or handwritten text into computer-editable text through electronic devices. Optical character recognition uses computer vision and deep learning technology to identify characters in images and convert them into text format. In the digital age, OCR technology has become an essential tool in the fields of information extraction, document management, and data mining.

[0003] In the related art, in the text information obtained through OCR recognition, there are often problems of content recognition errors, splicing errors and other recognition errors, and the recognition accuracy is low. Summary of the invention

[0004] The present application discloses an optical character recognition method, device, electronic device and readable storage medium, which can improve the recognition accuracy.

[0005] In order to solve the above problems, this application adopts the following technical solutions: In a first aspect, an embodiment of the present application discloses an optical character recognition method, comprising: inputting a target image including characters to be recognized and a target language category list into an optical character recognition model, recognizing the characters to be recognized, and obtaining a first recognition result output by the optical character recognition model, wherein the target language category list corresponds to the target image; determining a target prediction word corresponding to the target position based on context information corresponding to a target position in the first recognition result, wherein the target position is the position of a target word in the first recognition result, and the target word is any word in the first recognition result; and obtaining a second recognition result by filling the target prediction word into the target position.

[0006] In a second aspect, an embodiment of the present application discloses an optical character recognition device, comprising: an obtaining module, used to recognize the characters to be recognized by inputting a target image including the characters to be recognized and a target language type list into an optical character recognition model, and obtain a first recognition result output by the optical character recognition model, wherein the target language type list corresponds to the target image; a determination module, used to determine a target prediction word corresponding to the target position based on context information corresponding to the target position in the first recognition result, wherein the target position is the position of the target word in the first recognition result, and the target word is any word in the first recognition result; the obtaining module is also used to obtain a second recognition result by filling the target prediction word into the target position.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, which, when executed by a computer, cause the computer to execute: the steps of the method described in the first aspect.

[0010] The technical solution adopted in this application can achieve the following beneficial effects: An embodiment of the present application provides an optical character recognition method, which inputs a target image including characters to be recognized and a target language category list corresponding to the target image into an optical character recognition model, recognizes the characters to be recognized in the target image, and obtains a first recognition result output by the optical character recognition model. Based on context information corresponding to the target position in the first recognition result, a target prediction word corresponding to the target position is determined, where the target position is the position where the target word is located in the first recognition result, and the target word is any word in the first recognition result. Then, a second recognition result is obtained by filling the target prediction word into the target position, thereby correcting the first recognition result output by the optical character recognition model, and improving the accuracy of optical character recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A schematic diagram of a flow chart of an optical character recognition method disclosed in an embodiment of the present application; Figure 2 A flowchart of an optical character recognition method disclosed in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an optical character recognition device disclosed in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0013] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the electrically connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0014] The optical character recognition method, device, electronic device and readable storage medium disclosed in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0015] The present application discloses an optical character recognition method. Figure 1 Schematic diagram of a flow chart of an optical character recognition method disclosed in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps: S120, by inputting a target image including characters to be recognized and a target language category list into an optical character recognition model, recognizing the characters to be recognized, and obtaining a first recognition result output by the optical character recognition model, wherein the target language category list corresponds to the target image.

[0016] It should be noted that the optical character recognition model in the present application is a pre-trained model. Exemplarily, the target language list may include Chinese, English, Japanese, etc.

[0017] In this application, the feature vector I corresponding to the target image can be determined v , the feature vector and the target language type list are used as inputs of the optical character recognition model to obtain a first recognition result corresponding to the target image output by the optical character recognition model. Exemplarily, the first recognition result , The recognized words or characters are arranged in sequence.

[0018] S140. Determine a target predicted word corresponding to the target position based on context information corresponding to the target position in the first recognition result, wherein the target position is the position of the target word in the first recognition result, and the target word is any word in the first recognition result.

[0019] It should be noted that the target words and predicted words in this application can be words or characters, and this application does not make specific restrictions on this. In addition, the solution of this application traverses each word in the first recognition result, and for each word position in the first recognition result, determines its corresponding target predicted word.

[0020] For example, the first recognition result To traverse, for , according to the first recognition result The context information corresponding to the location determines the In the case of the target prediction word corresponding to the position, construct , where N is an empty code, containing only By Input the prediction model and get The target predicted word corresponding to the position.

[0021] S160: Obtain a second recognition result by filling the target prediction word into the target position.

[0022] The present application obtains a second recognition result corresponding to the target image by filling the target prediction word into the target position in the first recognition result, thereby correcting the first recognition result output by the optical character recognition model, and can improve the accuracy of optical character recognition.

[0023] It should be noted that, in the present application, filling the target predicted word into the target position means replacing the target word with the target predicted word.

[0024] An embodiment of the present application provides an optical character recognition method, which inputs a target image including characters to be recognized and a target language category list corresponding to the target image into an optical character recognition model, recognizes the characters to be recognized in the target image, and obtains a first recognition result output by the optical character recognition model. Based on context information corresponding to the target position in the first recognition result, a target prediction word corresponding to the target position is determined, where the target position is the position where the target word is located in the first recognition result, and the target word is any word in the first recognition result. Then, a second recognition result is obtained by filling the target prediction word into the target position, thereby correcting the first recognition result output by the optical character recognition model, and improving the accuracy of optical character recognition.

[0025] In one implementation, the determining the target prediction word corresponding to the target position based on the context information corresponding to the target position in the first recognition result may include: determining the target prediction word corresponding to the target position based on the context information corresponding to the target position in the first recognition result and the associated domain set corresponding to the target image, so as to improve the accuracy of the determined target prediction word. and the associated domain set are input into the prediction model, and the The target predicted word corresponding to the position.

[0026] In one implementation, the determining the target prediction word corresponding to the target position based on the context information corresponding to the target position in the first recognition result may include: determining the target prediction word corresponding to the target position based on the context information corresponding to the target position in the first recognition result, the associated domain set corresponding to the target image, and the target language category list, so as to improve the accuracy of the determined target prediction word. , the associated domain set and the target language type list are input into the prediction model, and the The target predicted word corresponding to the position.

[0027] In an embodiment of the present application, the determining of the target prediction word corresponding to the target position based on the context information corresponding to the target position in the first recognition result may include: determining the prediction word corresponding to the target position and the probability information corresponding to each of the prediction words based on the context information corresponding to the target position in the first recognition result; determining the target prediction word based on the probability information corresponding to each of the prediction words, wherein the target prediction word is the prediction word with the largest probability information among the prediction words. That is to say, the present application may first determine multiple prediction words corresponding to the target position and the probability information corresponding to each of the prediction words based on the context information corresponding to the target position in the first recognition result, and then use the prediction word with the largest probability information as the target prediction word.

[0028] In an embodiment of the present application, determining the predicted word corresponding to the target position and the probability information corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result may include: determining the predicted word corresponding to the target position and the occurrence probability corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result; respectively determining the similarity probability between the target word and each of the predicted words; and determining the probability information corresponding to each of the predicted words based on the occurrence probability corresponding to each of the predicted words and the similarity probability corresponding to each of the predicted words.

[0029] In this application, it can be done by Input the prediction model and get The predicted word corresponding to the position and the occurrence probability corresponding to each predicted word.

[0030] In the present application, the probability information corresponding to each predicted word can be obtained by multiplying the occurrence probability corresponding to each predicted word with the similarity probability corresponding to each predicted word. For example, ,in, Representation and prediction words The corresponding probability information is , 1≤i≤s, Representation and prediction words The corresponding probability of occurrence is Representation and prediction words The corresponding similarity probability is, for example, In the maximum case, Corresponding As the target prediction word, the second recognition result is obtained .

[0031] It should be noted that the similarity probability between the target word and the predicted word indicates the probability that the target word is the predicted word.

[0032] In one implementation, the step of determining the predicted word corresponding to the target position and the probability of occurrence of each predicted word based on the context information corresponding to the target position in the first recognition result may include: determining the predicted word corresponding to the target position and the probability of occurrence of each predicted word based on the context information corresponding to the target position in the first recognition result and the associated domain set corresponding to the target image, so as to improve the accuracy of the determined predicted word and its probability of occurrence. and the associated domain set are input into the prediction model, and the The predicted word corresponding to the position and the occurrence probability corresponding to each predicted word.

[0033] In one implementation, determining the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result may include: determining the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result, the associated field set corresponding to the target image, and the target language type list, so as to improve the accuracy of the determined predicted words and their occurrence probabilities.

[0034] For example, it can be achieved by , the associated domain set and the target language type list are input into the NLP (Natural Language Processing) model to obtain the predicted output prediction sequence ,in, Represents the predicted word The probability of occurrence is , 1≤i≤s, For The predicted word corresponding to the position.

[0035] In one implementation, the separately determining the similarity probability between the target word and each of the predicted words may include: based on the target word, each of the predicted words, a set of associated fields corresponding to the target image, and the target language category list, separately determining the similarity probability between the target word and each of the predicted words.

[0036] For example, each predicted word is traversed, and the target word, predicted word, associated domain set, and target language category list are input into the text recognition model to obtain the similarity probability between the target word and the predicted word output by the text recognition model, and the traversal results are summarized to obtain ,in, Indicates that the target word is the predicted word The probability of , 1≤i≤s.

[0037] In an embodiment of the present application, before the target image including the characters to be recognized and the target language category list are input into the optical character recognition model to recognize the characters to be recognized and obtain the first recognition result output by the optical character recognition model, the method may also include: determining the field corresponding to the target image based on the source file name of the target image and the source media type of the target image; determining the field in which the user is engaged based on user portrait information; determining the associated field set corresponding to the target image based on the field corresponding to the target image and the field in which the user is engaged; obtaining a set first language category list and a second language category list corresponding to the associated field set; and determining a target language category list based on the first language category list and the second language category list.

[0038] Exemplarily, the target image may be selected or captured from the target data.

[0039] In the present application, the source file name of the target image and the source media type of the target image can be determined based on the metadata of the target image, and then the source file name of the target image and the source media type of the target image are input into the domain judgment model to obtain the domain corresponding to the target image output by the domain judgment model. The electronic device can obtain the user portrait information of the user of the target image (i.e., the user corresponding to the electronic device), and then determine the user's field of work based on the user portrait information. Then the domain corresponding to the target image and the domain in which the user is engaged are merged to form a related domain set .

[0040] When obtaining the first language list, the language in the user's current language setting can be used as the language type for the text type prompt before recognition. If the user does not operate during the prompt period, the default text main language type is Of course, the user can also select other languages ​​as the main language. When the user selects other languages ​​at the same time, { } as the secondary language type, the first language type list is obtained as If the user does not select another language as the secondary language, read the language list configured by the system { }, in the exclusion After, and The first language list obtained by splicing is .

[0041] After obtaining the list of second language types associated with all fields in the associated field set Finally, the first language category list and the second language category list are concatenated to obtain the target language category list. .

[0042] In an embodiment of the present application, before the target image including the characters to be recognized and the target language type list are input into the optical character recognition model to recognize the characters to be recognized and obtain the first recognition result output by the optical character recognition model, the method may also include: determining the target image based on the target data, wherein the target data is one of an online preview resource, a static image file, a dynamic image file and a non-editable document.

[0043] The optical character recognition method of the present application can be applied to cloud disks. When users browse non-text images on a page, the cloud disk supports OCR text recognition of the selected area through area selection. It also supports users to select files that need to be extracted text, and the system converts them into text. The corresponding relationship between the target data and the extraction scheme is as follows:

[0044] The selected area selection extraction scheme is used for online preview resources (files that cannot directly extract text but support online preview). By selecting the area, the selected area is obtained by obtaining the recognition range. The recognition tool takes a screenshot of the selected area in the form of a picture to obtain the target image, writes it into the application, and uploads it to the cloud disk background. After receiving the uploaded image, the cloud disk background saves it in the form of an image.

[0045] The image area selection extraction solution is used for static image files. According to the area selected by the user on the static image, the specified area is intercepted in the source file of the static image to obtain the target image.

[0046] The position area selection extraction scheme is used for dynamic image files. According to the time t selected by the user corresponding to the dynamic image, the target frame image of the dynamic image file at time t is extracted from the source file of the dynamic image as a new static image. Then, according to the area selected by the user, the static image is intercepted and saved to obtain the target image.

[0047] The file resource extraction solution is used for non-editable documents. The non-editable document is parsed through a supported file parser, and the print function of the file parser is called. The page margin parameters are set to 0, no paging, and the size is the actual file size. It is printed as a PDF format file, and the PDF is read page by page as pictures. The read pictures are stitched in sequence in the reading order to form a complete picture to obtain the target image.

[0048] In an embodiment of the present application, determining a target image based on target data may include: determining a first grayscale image to be identified based on the target data; obtaining a first image by performing background stripping on the first grayscale image; determining a target convolution kernel based on the position of each pixel point in the first image, filtering parameters, and image type adjustment factors; convolving the first image based on the target convolution kernel to obtain a target image pixel matrix; and determining a target image based on the target image pixel matrix.

[0049] In the present application, after obtaining an image to be identified based on target data, the image to be identified is converted into a corresponding first grayscale image, where the first grayscale image is a single-channel grayscale image. For example, the first grayscale image , where R, G, and B represent the grayscale values ​​of the red, green, and blue channels in the image to be identified, respectively.

[0050] After the first grayscale image is determined, the first image is obtained by performing background stripping on the first grayscale image.

[0051] In the present application, when the target data is a dynamic image file and the first grayscale image is a grayscale image corresponding to a target frame image in the dynamic image file, the first image is obtained by performing background stripping on the first grayscale image, which may include: acquiring a plurality of second grayscale images, wherein the plurality of second grayscale images are grayscale images corresponding to a preset number of frame images before and after the target frame image in the dynamic image file; adjusting the pixel value of each pixel in the first grayscale image based on the pixel value of the first pixel, the pixel values ​​corresponding to the first pixel in each of the second grayscale images, and a dynamic resource stability threshold, wherein the first pixel is any pixel in the first grayscale image; and determining the first image based on the adjusted pixel values ​​of each pixel in the first grayscale image.

[0052] For example, the target frame image in the dynamic image file can be obtained. The grayscale images corresponding to the previous and next 10 frames of images are used as multiple second grayscale images , for the first pixel (x, y), use the algorithm: , , adjust the pixel value of the first pixel, where, represents the pixel value of the first pixel, represents the pixel value corresponding to the first pixel in the second grayscale image, Indicates the configured dynamic resource stability threshold, The pixel value of the first pixel is represented by the pixel value after being adjusted again, and then the first image obtained by background stripping is determined based on the pixel values ​​of each pixel in the first grayscale image after being adjusted.

[0053] In the present application, when the target data is a non-editable document and the first grayscale image includes multiple document paging images, the first image is obtained by performing background stripping on the first grayscale image, which may include: adjusting the pixel value of the second pixel for each pixel in each of the document paging images based on the pixel value of the second pixel, the pixel value corresponding to the second pixel in each of the first document paging images, and a static resource stability threshold, wherein the second pixel is any pixel in the second document paging image, the second document paging image is any document paging image among the multiple document paging images, and the first document paging image is any document paging image among the multiple document paging images except the second document paging image; and determining the first image based on the adjusted pixel values ​​of each pixel in each of the document paging images.

[0054] For example, the first grayscale image can be segmented according to the original document paging to obtain grayscale images corresponding to multiple document paging images. , for the second document paging image The second pixel (x, y) in the image is calculated using the algorithm , , adjust the pixel value of the second pixel, where, represents the pixel value of the second pixel, represents the pixel value corresponding to the second pixel point in the first document paging image, Indicates the configured static resource stability threshold. The pixel value of the second pixel is represented by the pixel value after being adjusted again, and then the first image obtained by background stripping is determined based on the pixel value of each pixel in each document paging image after being adjusted.

[0055] It should be noted that, when the target data is an online preview resource or a static image file, the background stripping solution in the prior art can be used to obtain the first image.

[0056] After obtaining the first image, a target convolution kernel is determined based on the position of each pixel point in the first image, filtering parameters and image type adjustment factors, and then the first image is convolved based on the target convolution kernel to reduce image noise, obtain a target image pixel matrix, and determine the target image based on the target image pixel matrix.

[0057] For example, the formula , generate a 5×5 target convolution kernel c. The specific logic is that for any position (x, y) of the target convolution kernel, take its value ,in, Indicates the configured filtering parameters, Indicates the image type adjustment factor. When the first image is a file resource extraction, the image type adjustment factor is 1.5. When the first image is of other types, the image type adjustment factor is 3.

[0058] In an embodiment of the present application, determining the target image based on the target image pixel matrix may include: determining a second image corresponding to the target image pixel matrix; determining a transverse gradient matrix corresponding to the second image based on a first convolution window and the second image, wherein the first convolution window is used to perform a convolution operation on the second image from left to right; determining a longitudinal gradient matrix corresponding to the second image based on a second convolution window and the second image, wherein the second convolution window is used to perform a convolution operation on the second image from top to bottom; determining a first pixel gradient intensity and a gradient direction corresponding to a third pixel point based on a first transverse gradient value and a first longitudinal gradient value, wherein the first transverse gradient value is the transverse gradient value. The first longitudinal gradient value is the longitudinal gradient value corresponding to the third pixel in the longitudinal gradient matrix, and the third pixel is any pixel in the second image; based on the gradient direction corresponding to the third pixel, the transverse gradient matrix and the longitudinal gradient matrix, the pixel gradient intensity of the adjacent pixels of the third pixel in the second image is determined; based on the first pixel gradient intensity and the pixel gradient intensity of the adjacent pixels, the pixel gradient intensity of the third pixel is adjusted to obtain the second pixel gradient intensity corresponding to the third pixel; based on the second pixel gradient intensity corresponding to each pixel in the second image, the target image is determined.

[0059] Exemplarily, after determining the second image I corresponding to the pixel matrix of the target image, determining the lateral gradient matrix corresponding to the second image , determine the longitudinal gradient matrix corresponding to the second image ,in, is the first convolution window, The first convolution window and the second convolution window are in a transposed relationship. The first convolution window and the second convolution window start from the upper left corner of the target image pixel matrix, and perform autocorrelation operations in order from left to right and from top to bottom to obtain the first pixel gradient intensity corresponding to the third pixel point. , get the gradient direction corresponding to the third pixel , where i, j represent the position coordinates of the positive direction of the x-axis (horizontally from left to right) and the positive direction of the y-axis (vertically from top to bottom) on the second image I, with the upper left corner as point (0,0). represents the first transverse gradient value, represents the first longitudinal gradient value, After determining the gradient direction corresponding to the third pixel, the pixel gradient strengths of two adjacent pixel points of the third pixel in the second image are determined based on the gradient direction corresponding to the third pixel, the horizontal gradient matrix and the vertical gradient matrix. ,exist When established, set the gradient intensity of the second pixel corresponding to the third pixel Otherwise, set the second pixel gradient intensity corresponding to the third pixel , that is, in When the two sides are equal, the first pixel gradient intensity corresponding to the third pixel point is directly used as its second pixel gradient intensity. When the two sides are not equal, the second pixel gradient intensity corresponding to the third pixel is set to 0. After obtaining the second pixel gradient intensity corresponding to each pixel in the second image, the result of gradient intensity suppression and hysteresis processing is obtained. , can be based on Determining a target image and performing optical character recognition based on the target image of the present application can improve recognition accuracy.

[0060] It should be noted that the two adjacent pixel points of the third pixel point are the pixel points adjacent to the third pixel point in its gradient direction (including the reverse direction), and the specific calculation method of the pixel gradient strength of the two adjacent pixel points of the third pixel point is similar to the first pixel gradient strength of the third pixel point, which will not be repeated in this application.

[0061] In an embodiment of the present application, determining the target image based on the second pixel gradient intensity corresponding to each pixel in the second image may include: adjusting the pixel gradient intensity of each pixel in the second image based on the second pixel gradient intensity corresponding to each pixel in the second image and a minimum threshold configured by the system to obtain a third pixel gradient intensity corresponding to each pixel in the second image; and determining the target image based on the third pixel gradient intensity corresponding to each pixel in the second image.

[0062] When the second pixel gradient intensity of a pixel point in the second image is greater than or equal to the minimum threshold configured by the system, the second pixel gradient intensity corresponding to the pixel point is used as its third pixel gradient intensity. When the second pixel gradient intensity of a pixel point in the second image is less than the minimum threshold configured by the system, the third pixel gradient intensity of the pixel point is set to 0, thereby achieving non-edge texture cleaning of the second image.

[0063] For example, for ,when When the third pixel gradient strength is set ,in, Indicates the minimum threshold configured for the system.

[0064] After determining the third pixel gradient intensity corresponding to each pixel point in the second image, the target image can be determined, and optical character recognition is performed based on the target image of the present application, which can improve the recognition accuracy.

[0065] In an embodiment of the present application, determining the target image based on the third pixel gradient intensity corresponding to each pixel in the second image may include: when the third pixel gradient intensity corresponding to the third pixel is less than or equal to a maximum threshold configured by the system, adjusting the pixel gradient intensity of the third pixel based on the third pixel gradient intensity corresponding to the surrounding pixels of the third pixel in the second image and the maximum threshold to obtain a fourth pixel gradient intensity corresponding to the third pixel; determining the target image based on the fourth pixel gradient intensity corresponding to each pixel in the second image.

[0066] When the third pixel gradient intensity corresponding to the third pixel point is greater than the maximum threshold configured by the system, the third pixel gradient intensity of the third pixel point is used as its fourth pixel gradient intensity. When the third pixel gradient intensity of the third pixel point is less than or equal to the maximum threshold configured by the system, the pixel gradient intensity of the third pixel point is adjusted based on the third pixel gradient intensity corresponding to the surrounding pixels of the third pixel point in the second image and the maximum threshold configured by the system to obtain the fourth pixel gradient intensity corresponding to the third pixel point, thereby achieving edge width compression of the second image.

[0067] For example, for , when the third pixel gradient intensity of the third pixel When , get the maximum third pixel gradient intensity among the 8 pixels around it ,when When the fourth pixel gradient strength of the third pixel is set ,when hour, The third pixel gradient intensity of the third pixel point is taken as its fourth pixel gradient intensity, wherein, Indicates the maximum threshold configured in the system.

[0068] After determining the fourth pixel gradient intensity corresponding to each pixel point in the second image, the target image can be determined, and optical character recognition is performed based on the target image of the present application, which can improve the recognition accuracy.

[0069] In the embodiment of the present application, after obtaining the second recognition result, the second recognition result is returned to the user, and the final recognition result modified by the user is received. .

[0070] After executing the optical character recognition process of this application multiple times, it can be determined that the optimal training data set ,in, , represents the vectorized representation of the grayscale image input to the optical character recognition model for the i-th time, represents the list of target languages ​​corresponding to the i-th recognition, It represents the final recognition result modified by the user for the i-th time, and then takes the grayscale image vectorized representation in the optimized training data set and the target language type list as input and the final recognition result modified by the user as output, and performs transfer learning on the optical character recognition model to improve the recognition accuracy of the optical character recognition model.

[0071] In the present application, the optical character recognition model can be transferred and updated according to a preset time period to continuously improve the performance of the optical character recognition model. Exemplarily, the preset time period can be a month.

[0072] It should be noted that when the present application obtains the first grayscale image and performs image processing such as background stripping on the first grayscale image, the first grayscale image can be processed as a whole, or the first grayscale image can be divided into multiple small images and then processed. The present application does not impose any specific restrictions on this.

[0073] The execution process of the optical character recognition method in this application is as follows: Figure 2As shown, based on the received OCR recognition instruction and target data, an image to be recognized is generated, and the image to be recognized can be determined based on the selection of the online preview resource area, or based on the selection of the static image area, or based on the target frame image in the dynamic image, or based on the non-editable document. Then, a first grayscale image corresponding to the image to be recognized is generated, and the background of the first grayscale image is stripped to obtain a first image. Then, the image contour is enhanced (including reducing image noise, determining gradient intensity, cleaning non-edge textures, and compressing edge widths as described above) on the first image to obtain a target image. Based on the set first language category list and the second language category list corresponding to the associated domain set corresponding to the target image, a target language category list is determined, and the target image and the target language category list are input into the OCR recognition model for OCR recognition and a first recognition result is output. Then, the first recognition result is corrected using the NLP model to obtain a second recognition result, and returned to the user. In addition, the final recognition result modified by the user can also be received, and the OCR recognition model is transferred and updated according to a preset time period to continuously improve the recognition accuracy of the OCR recognition model.

[0074] The optical character recognition method provided in the embodiment of the present application can be executed by an optical character recognition device. In the embodiment of the present application, an optical character recognition device executing the optical character recognition method is taken as an example to illustrate the optical character recognition device provided in the embodiment of the present application.

[0075] Figure 3 FIG. 1 is a schematic diagram of the structure of an optical character recognition device disclosed in an embodiment of the present application. Figure 3 As shown, the optical character recognition device 300 includes: an obtaining module 310 and a determining module 320 .

[0076] In the present application, an obtaining module 310 is used to input a target image including characters to be recognized and a target language category list into an optical character recognition model to recognize the characters to be recognized, thereby obtaining a first recognition result output by the optical character recognition model, wherein the target language category list corresponds to the target image; a determining module 320 is used to determine a target prediction word corresponding to the target position based on context information corresponding to the target position in the first recognition result, wherein the target position is the position of the target word in the first recognition result, and the target word is any word in the first recognition result; the obtaining module 310 is also used to obtain a second recognition result by filling the target prediction word into the target position.

[0077] In one implementation, the determination module 320 determines the target predicted word corresponding to the target position based on the context information corresponding to the target position in the first recognition result, including: determining the predicted word corresponding to the target position and the probability information corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result; determining the target predicted word based on the probability information corresponding to each of the predicted words, wherein the target predicted word is the predicted word with the largest probability information among the predicted words.

[0078] In one implementation, the determination module 320 determines the predicted word corresponding to the target position and the probability information corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result, including: determining the predicted word corresponding to the target position and the occurrence probability corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result; respectively determining the similarity probability between the target word and each of the predicted words; determining the probability information corresponding to each of the predicted words based on the occurrence probability corresponding to each of the predicted words and the similarity probability corresponding to each of the predicted words.

[0079] In one implementation, the determination module 320 determines the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result, including: determining the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result and the associated field set corresponding to the target image.

[0080] In one implementation, the determination module 320 determines the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result, including: determining the predicted word corresponding to the target position and the probability of occurrence corresponding to each of the predicted words based on the context information corresponding to the target position in the first recognition result, the associated field set corresponding to the target image, and the target language type list.

[0081] In one implementation, the determination module 320 determines the similarity probability between the target word and each of the predicted words, including: based on the target word, each of the predicted words, a set of associated fields corresponding to the target image, and the target language category list, determining the similarity probability between the target word and each of the predicted words.

[0082] In one implementation, the above-mentioned device also includes: the determination module 320 is also used to determine the field corresponding to the target image based on the source file name of the target image and the source media type of the target image before the target image including the characters to be recognized and the target language category list are input into the optical character recognition model to recognize the characters to be recognized and obtain the first recognition result output by the optical character recognition model; the determination module 320 is also used to determine the field in which the user is engaged based on the user portrait information; the determination module 320 is also used to determine the associated field set corresponding to the target image based on the field corresponding to the target image and the field in which the user is engaged; the acquisition module is used to acquire the set first language category list and the second language category list corresponding to the associated field set; the determination module 320 is also used to determine the target language category list based on the first language category list and the second language category list.

[0083] In one implementation, the determination module 320 is further used to determine a target image based on target data before inputting a target image including characters to be recognized and a target language type list into an optical character recognition model to recognize the characters to be recognized and obtaining a first recognition result output by the optical character recognition model, wherein the target data is one of an online preview resource, a static image file, a dynamic image file, and a non-editable document.

[0084] In one implementation, the determination module 320 determines the target image based on the target data, including: determining a first grayscale image to be identified based on the target data; obtaining a first image by performing background stripping on the first grayscale image; determining a target convolution kernel based on the position of each pixel point in the first image, filtering parameters, and image type adjustment factors; convolving the first image based on the target convolution kernel to obtain a target image pixel matrix; and determining the target image based on the target image pixel matrix.

[0085] In one implementation, when the target data is a dynamic image file and the first grayscale image is a grayscale image corresponding to a target frame image in the dynamic image file, the determination module 320 obtains a first image by performing background stripping on the first grayscale image, including: acquiring a plurality of second grayscale images, wherein the plurality of second grayscale images are grayscale images corresponding to a preset number of frame images before and after the target frame image in the dynamic image file; adjusting the pixel value of each pixel in the first grayscale image based on the pixel value of the first pixel, the pixel values ​​corresponding to the first pixel in each of the second grayscale images, and a dynamic resource stability threshold, wherein the first pixel is any pixel in the first grayscale image; and determining the first image based on the adjusted pixel values ​​of each pixel in the first grayscale image.

[0086] In one implementation, when the target data is a non-editable document and the first grayscale image includes multiple document paging images, the determination module 320 obtains the first image by performing background stripping on the first grayscale image, including: adjusting the pixel value of the second pixel for each pixel in each of the document paging images based on the pixel value of the second pixel, the pixel value corresponding to the second pixel in each of the first document paging images, and a static resource stability threshold, wherein the second pixel is any pixel in the second document paging image, the second document paging image is any document paging image among the multiple document paging images, and the first document paging image is any document paging image among the multiple document paging images except the second document paging image; and determining the first image based on the adjusted pixel values ​​of each pixel in each of the document paging images.

[0087] In one implementation, the determination module 320 determines the target image based on the target image pixel matrix, including: determining a second image corresponding to the target image pixel matrix; determining a transverse gradient matrix corresponding to the second image based on a first convolution window and the second image, wherein the first convolution window is used to perform a convolution operation on the second image from left to right; determining a longitudinal gradient matrix corresponding to the second image based on a second convolution window and the second image, wherein the second convolution window is used to perform a convolution operation on the second image from top to bottom; determining a first pixel gradient intensity and a gradient direction corresponding to a third pixel point based on a first transverse gradient value and a first longitudinal gradient value, wherein the first transverse gradient value is the transverse gradient value of the first pixel point. the first longitudinal gradient value is the longitudinal gradient value corresponding to the third pixel in the longitudinal gradient matrix, the third pixel is any pixel in the second image; based on the gradient direction corresponding to the third pixel, the transverse gradient matrix and the longitudinal gradient matrix, the pixel gradient intensity of the adjacent pixels of the third pixel in the second image is determined; based on the first pixel gradient intensity and the pixel gradient intensity of the adjacent pixels, the pixel gradient intensity of the third pixel is adjusted to obtain the second pixel gradient intensity corresponding to the third pixel; based on the second pixel gradient intensity corresponding to each pixel in the second image, the target image is determined.

[0088] In one implementation, the determination module 320 determines the target image based on the second pixel gradient intensity corresponding to each pixel in the second image, including: adjusting the pixel gradient intensity of each pixel in the second image based on the second pixel gradient intensity corresponding to each pixel in the second image and a minimum threshold configured by the system to obtain a third pixel gradient intensity corresponding to each pixel in the second image; and determining the target image based on the third pixel gradient intensity corresponding to each pixel in the second image.

[0089] In one implementation, the determination module 320 determines the target image based on the third pixel gradient intensity corresponding to each pixel in the second image, including: when the third pixel gradient intensity corresponding to the third pixel is less than or equal to the maximum threshold configured by the system, based on the third pixel gradient intensity corresponding to the surrounding pixels of the third pixel in the second image and the maximum threshold, adjusting the pixel gradient intensity of the third pixel to obtain a fourth pixel gradient intensity corresponding to the third pixel; and determining the target image based on the fourth pixel gradient intensity corresponding to each pixel in the second image.

[0090] The optical character recognition device provided in the embodiment of the present application can implement each process implemented in the embodiment of the optical character recognition method, and will not be described again here to avoid repetition.

[0091] Alternatively, if Figure 4 As shown, an embodiment of the present application further provides an electronic device 400, including a processor 401 and a memory 402, wherein the memory 402 stores programs or instructions that can be executed on the processor 401, and when the program or instructions are executed by the processor 401, the various steps of the above-mentioned optical character recognition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0092] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

[0093] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned optical character recognition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0094] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0095] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes: the steps of the above-mentioned optical character recognition method.

[0096] The above embodiments of the present application focus on describing the differences between the various embodiments. As long as the different optimization features between the various embodiments are not contradictory, they can be combined to form a better embodiment. Considering the simplicity of the text, they will not be repeated here.

[0097] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An optical character recognition method, characterized in that: include: By inputting a target image including characters to be recognized and a target language category list into an optical character recognition model, the characters to be recognized are recognized to obtain a first recognition result output by the optical character recognition model, wherein the target language category list corresponds to the target image; Determine, based on context information corresponding to a target position in the first recognition result, a target prediction word corresponding to the target position, wherein the target position is a position where a target word is located in the first recognition result, and the target word is any word in the first recognition result; A second recognition result is obtained by filling the target prediction word into the target position.

2. The method according to claim 1, characterized in that The determining, based on the context information corresponding to the target position in the first recognition result, a target prediction word corresponding to the target position, comprises: Determining, based on the context information corresponding to the target position in the first recognition result, a predicted word corresponding to the target position and probability information corresponding to each of the predicted words; Based on the probability information corresponding to each of the predicted words, a target predicted word is determined, wherein the target predicted word is a predicted word having the largest probability information among the predicted words.

3. The method according to claim 2, characterized in that The determining, based on the context information corresponding to the target position in the first recognition result, a predicted word corresponding to the target position and probability information corresponding to each of the predicted words, includes: Determining, based on the context information corresponding to the target position in the first recognition result, a predicted word corresponding to the target position and an occurrence probability corresponding to each of the predicted words; Determining respectively the similarity probability between the target word and each of the predicted words; Based on the appearance probability corresponding to each of the predicted words and the similarity probability corresponding to each of the predicted words, probability information corresponding to each of the predicted words is determined.

4. The method according to claim 3, characterized in that: The determining, based on the context information corresponding to the target position in the first recognition result, the predicted word corresponding to the target position and the occurrence probability corresponding to each of the predicted words includes: Based on the context information corresponding to the target position in the first recognition result and the associated domain set corresponding to the target image, a predicted word corresponding to the target position and an appearance probability corresponding to each of the predicted words are determined.

5. The method according to claim 3, characterized in that: The determining, based on the context information corresponding to the target position in the first recognition result, the predicted word corresponding to the target position and the occurrence probability corresponding to each of the predicted words includes: Based on the context information corresponding to the target position in the first recognition result, the associated field set corresponding to the target image, and the target language category list, a predicted word corresponding to the target position and an appearance probability corresponding to each of the predicted words are determined.

6. The method according to claim 3, characterized in that The respectively determining the similarity probability between the target word and each of the predicted words comprises: Based on the target word, each of the predicted words, a set of related fields corresponding to the target image, and the target language category list, similarity probabilities between the target word and each of the predicted words are determined respectively.

7. The method according to claim 1, characterized in that Before inputting the target image including the characters to be recognized and the target language category list into the optical character recognition model to recognize the characters to be recognized and obtaining the first recognition result output by the optical character recognition model, the method further includes: Determine the domain corresponding to the target image based on the source file name of the target image and the source media type of the target image; Determine the user's field of work based on user portrait information; Determine a set of associated fields corresponding to the target image based on the field to which the target image belongs and the field in which the user is engaged; Acquire a set first language category list and a second language category list corresponding to the associated domain set; A target language category list is determined based on the first language category list and the second language category list.

8. The method according to claim 1, characterized in that Before inputting the target image including the characters to be recognized and the target language category list into the optical character recognition model to recognize the characters to be recognized and obtaining the first recognition result output by the optical character recognition model, the method further includes: Based on the target data, a target image is determined, wherein the target data is one of an online preview resource, a static image file, a dynamic image file, and a non-editable document.

9. The method according to claim 8, characterized in that The step of determining the target image based on the target data includes: Based on the target data, determining a first grayscale image to be identified; Obtaining a first image by performing background stripping on the first grayscale image; Determining a target convolution kernel based on the position of each pixel in the first image, a filtering parameter, and an image type adjustment factor; Convolving the first image based on the target convolution kernel to obtain a target image pixel matrix; Based on the target image pixel matrix, a target image is determined.

10. The method according to claim 9, characterized in that In a case where the target data is a dynamic image file, and the first grayscale image is a grayscale image corresponding to a target frame image in the dynamic image file, the step of performing background stripping on the first grayscale image to obtain the first image includes: Acquire a plurality of second grayscale images, wherein the plurality of second grayscale images are grayscale images corresponding to a preset number of frame images before and after the target frame image in the dynamic image file; For each pixel in the first grayscale image, adjusting the pixel value of the first pixel based on the pixel value of the first pixel, the pixel values ​​corresponding to the first pixel in each of the second grayscale images, and a dynamic resource stability threshold, wherein the first pixel is any pixel in the first grayscale image; A first image is determined based on the adjusted pixel values ​​of each pixel in the first grayscale image.

11. The method according to claim 9, characterized in that In the case where the target data is a non-editable document and the first grayscale image includes a plurality of document page images, the first image is obtained by performing background stripping on the first grayscale image, including: For each pixel point in each of the document paging images, based on the pixel value of the second pixel point, the pixel value corresponding to the second pixel point in each of the first document paging images, and the static resource stability threshold, the pixel value of the second pixel point is adjusted, wherein the second pixel point is any pixel point in the second document paging image, the second document paging image is any document paging image of the multiple document paging images, and the first document paging image is any document paging image of the multiple document paging images except the second document paging image; A first image is determined based on the adjusted pixel values ​​of each pixel point in each of the document paging images.

12. The method according to claim 9, characterized in that The step of determining the target image based on the target image pixel matrix comprises: Determining a second image corresponding to the target image pixel matrix; Determine a transverse gradient matrix corresponding to the second image based on a first convolution window and the second image, wherein the first convolution window is used to perform a convolution operation on the second image from left to right; Determining a longitudinal gradient matrix corresponding to the second image based on a second convolution window and the second image, wherein the second convolution window is used to perform a convolution operation on the second image from top to bottom; Determine a first pixel gradient intensity and a gradient direction corresponding to a third pixel point based on a first transverse gradient value and a first longitudinal gradient value, wherein the first transverse gradient value is a transverse gradient value corresponding to the third pixel point in the transverse gradient matrix, the first longitudinal gradient value is a longitudinal gradient value corresponding to the third pixel point in the longitudinal gradient matrix, and the third pixel point is any pixel point in the second image; Determine pixel gradient intensities of neighboring pixels of the third pixel in the second image based on the gradient direction corresponding to the third pixel, the transverse gradient matrix, and the longitudinal gradient matrix; Based on the first pixel gradient intensity and the pixel gradient intensity of the adjacent pixel point, the pixel gradient intensity of the third pixel point is adjusted to obtain a second pixel gradient intensity corresponding to the third pixel point; A target image is determined based on second pixel gradient strengths corresponding to each pixel point in the second image.

13. The method according to claim 12, characterized in that The determining the target image based on the second pixel gradient intensity corresponding to each pixel point in the second image includes: Based on the second pixel gradient intensity corresponding to each pixel point in the second image and a minimum threshold configured by the system, the pixel gradient intensity of each pixel point in the second image is adjusted to obtain a third pixel gradient intensity corresponding to each pixel point in the second image; A target image is determined based on third pixel gradient strengths corresponding to each pixel point in the second image.

14. The method according to claim 13, characterized in that The determining the target image based on the third pixel gradient intensity corresponding to each pixel point in the second image includes: When the third pixel gradient intensity corresponding to the third pixel is less than or equal to a maximum threshold configured by the system, adjusting the pixel gradient intensity of the third pixel based on the third pixel gradient intensity corresponding to the surrounding pixels of the third pixel in the second image and the maximum threshold, so as to obtain a fourth pixel gradient intensity corresponding to the third pixel; A target image is determined based on fourth pixel gradient strengths corresponding to each pixel point in the second image.

15. An optical character recognition device, characterized in that: include: an obtaining module, configured to recognize the characters to be recognized by inputting a target image including the characters to be recognized and a target language category list into an optical character recognition model, and obtain a first recognition result output by the optical character recognition model, wherein the target language category list corresponds to the target image; a determination module, configured to determine a target prediction word corresponding to the target position based on context information corresponding to the target position in the first recognition result, wherein the target position is a position where the target word is located in the first recognition result, and the target word is any word in the first recognition result; The obtaining module is further used to obtain a second recognition result by filling the target prediction word into the target position.

16. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the optical character recognition method according to any one of claims 1 to 14 are implemented.

17. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the optical character recognition method according to any one of claims 1 to 14 are implemented.

18. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to execute: the steps of the optical character recognition method according to any one of claims 1 to 14.

Images