Method and apparatus for recognizing characters of a seal

Abstract

The application relates to a seal text recognition method and device. The method comprises the following steps: performing text detection and text classification processing on a seal picture to be processed to obtain initial text position information, the initial text comprising initial curved text; acquiring the maximum curved text in the initial curved text, and performing principal component analysis according to the position information of the maximum curved text; calculating a seal rotation angle according to the principal component analysis result; correcting the position information of the initial text based on the seal rotation angle, and performing straightening processing on the curved text after the correction processing to obtain a corrected text slice corresponding to each initial text; and performing text recognition on the corrected text slice to obtain a text recognition result corresponding to the seal picture. The seal text recognition method can accurately recognize all the texts in the seal.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for recognizing characters on a seal. Background Technology

[0002] Seals are stationery used to mark documents for authentication or signature, and are widely used in business scenarios such as contract comparison, risk assessment, and document authentication. With the development of artificial intelligence and image technology, seal recognition has become increasingly intelligent.

[0003] Traditional methods of seal recognition employ seal recognition models. Current models typically use text segmentation to divide the text within the seal into individual characters and then recognize them. This approach is generally effective for rectangular seals, but when the seal contains curved text, computational errors can occur, leading to low recognition rates for the overall text and an inability to accurately identify all the text within the seal. Summary of the Invention

[0004] Therefore, it is necessary to provide a character recognition method and device for seals that can accurately identify the text in the seal, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for recognizing characters on a seal, the method comprising:

[0006] Text detection and text classification are performed on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text;

[0007] Obtain the largest curved text in the initial curved text, and perform principal component analysis based on the position information of the largest curved text;

[0008] The seal rotation angle was calculated based on the principal component analysis results.

[0009] The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text.

[0010] The corrected text slice is subjected to character recognition to obtain the character recognition result corresponding to the seal image.

[0011] In one embodiment, the initial text further includes initial horizontal text, and the text detection and text classification processing of the seal image to be processed to obtain the position information of the initial text includes:

[0012] Text detection is performed on the seal image to be processed to obtain text slices in the seal image;

[0013] Determine the minimum bounding rectangle for each text slice and calculate the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle.

[0014] Text slices with a ratio less than a preset threshold are classified as curved text slices, and text slices with a ratio greater than or equal to the preset threshold are classified as horizontal text slices.

[0015] The position information corresponding to the curved text slice is used as the position information of the initial curved text, and the position information corresponding to the horizontal text slice is used as the position information of the initial horizontal text.

[0016] In one embodiment, the principal component analysis based on the location information of the most curved text includes:

[0017] Based on the location information of the most curved text, a mask image of the most curved text is obtained;

[0018] Based on the stamp image and the mask image, obtain the segmented image of the text with the maximum curvature;

[0019] The segmented image is binarized to obtain a binarized image of the most curved text, and the coordinates of the target pixel representing the curved text in the binarized image are determined.

[0020] Principal component analysis is performed based on the coordinates of the target pixel to obtain the principal component analysis results, which include multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue.

[0021] In one embodiment, the step of performing principal component analysis based on the coordinates of the target pixel to obtain the principal component analysis result includes:

[0022] Determine the x-coordinate and y-coordinate values ​​of the target pixel;

[0023] For each target pixel, the covariance matrix is ​​determined based on the x-coordinate and y-coordinate values ​​of the corresponding target pixel.

[0024] Principal component analysis is performed based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results.

[0025] In one embodiment, the principal component analysis result includes multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue, and the calculation of the seal rotation angle based on the principal component analysis result includes:

[0026] Sort the multiple feature values ​​by size, and obtain the first target feature value with the largest feature value and the second target feature value with the second largest feature value;

[0027] Determine the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis, and determine the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis;

[0028] When the difference between the first included angle and the second included angle is greater than a preset difference, the opposite of the first included angle is taken as the rotation angle of the seal.

[0029] When the difference between the first included angle and the second included angle is less than or equal to a preset difference, the slope of the feature vector of the first target feature value is compared with the slope of the feature vector of the second target feature value, and the included angle corresponding to the feature vector with the smaller slope is taken as the seal rotation angle.

[0030] In one embodiment, the initial text further includes initial horizontal text, and the corrected text slice includes corrected horizontal text slices and corrected curved text slices;

[0031] The process of correcting the position information of the initial text based on the rotation angle of the seal, and straightening the corrected curved text to obtain corrected text slices corresponding to each initial text, includes:

[0032] The seal rotation matrix is ​​calculated based on the seal rotation angle, and the position information of each initial horizontal text is corrected based on the seal rotation matrix to obtain a corrected horizontal text slice.

[0033] The position information of the most curved text is corrected according to the rotation angle of the seal, and the corrected most curved text is straightened to obtain a slice of the corrected curved text.

[0034] Identify the other curved texts in the initial curved text besides the largest curved text, correct the position information of the other curved texts based on the stamp rotation matrix, and straighten the other curved texts after correction to obtain the corrected curved text slices of each other curved text.

[0035] In one embodiment, the step of straightening the corrected text with the largest curvature to obtain a corrected curvature text slice of the largest curvature text includes:

[0036] Create a background image;

[0037] Based on the polar coordinate information of the pixels in the background image, the coordinates of the pixels of the most curved text after correction are converted into corresponding polar coordinates.

[0038] The maximum curvature text, converted to polar coordinates, is mapped onto the background image to obtain a mapped image;

[0039] The mapped image is cropped based on the most curved text in the mapped image to obtain a cropped image;

[0040] The pixels belonging to the background image in the cropped image are replaced with the background pixels of the maximum curved text to obtain the corrected curved text slice of the maximum curved text.

[0041] In one embodiment, the step of correcting the position information of each initial horizontal text based on the seal rotation matrix to obtain corrected horizontal text slices includes:

[0042] Based on the seal rotation matrix, the position information of each initial horizontal text is corrected to obtain a first-corrected slice.

[0043] Based on the coordinates of the four vertices of the first-correction slice, a second-correction process is performed on the first-correction slice to obtain a corrected horizontal text slice.

[0044] In one embodiment, the step of performing character recognition on the corrected text slice to obtain a character recognition result corresponding to the seal image includes:

[0045] The corrected text slice is mirrored to obtain a mirrored text slice;

[0046] The corrected text slice is subjected to character recognition to obtain a first recognition result and the confidence level of the first recognition result;

[0047] The mirrored text slice is subjected to text recognition to obtain a second recognition result and the confidence level of the second recognition result;

[0048] The recognition result with higher confidence among the first and second recognition results is taken as the text recognition result corresponding to the seal image.

[0049] Secondly, this application also provides a character recognition device for a seal, the device comprising:

[0050] The detection and classification module is used to perform text detection and text classification processing on the seal image to be processed, and to obtain the position information of the initial text, which includes the initial curved text.

[0051] The analysis module is used to obtain the maximum curved text in the initial curved text and perform principal component analysis based on the position information of the maximum curved text;

[0052] The calculation module is used to calculate the seal rotation angle based on the principal component analysis results;

[0053] The correction module is used to correct the position information of the initial text based on the rotation angle of the seal, and to straighten the curved text after correction to obtain the corrected text slices corresponding to each initial text.

[0054] The recognition module is used to perform character recognition on the corrected text slices to obtain the character recognition result corresponding to the seal image.

[0055] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0056] Text detection and text classification are performed on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text;

[0057] Obtain the largest curved text in the initial curved text, and perform principal component analysis based on the position information of the largest curved text;

[0058] The seal rotation angle was calculated based on the principal component analysis results.

[0059] The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text.

[0060] The corrected text slice is subjected to character recognition to obtain the character recognition result corresponding to the seal image.

[0061] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0062] Text detection and text classification are performed on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text;

[0063] Obtain the largest curved text in the initial curved text, and perform principal component analysis based on the position information of the largest curved text;

[0064] The seal rotation angle was calculated based on the principal component analysis results.

[0065] The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text.

[0066] The corrected text slice is subjected to character recognition to obtain the character recognition result corresponding to the seal image.

[0067] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0068] Text detection and text classification are performed on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text;

[0069] Obtain the largest curved text in the initial curved text, and perform principal component analysis based on the position information of the largest curved text;

[0070] The seal rotation angle was calculated based on the principal component analysis results.

[0071] The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text.

[0072] The corrected text slice is subjected to character recognition to obtain the character recognition result corresponding to the seal image.

[0073] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for recognizing characters on seals obtain the initial text position information by performing text detection and classification on the seal image to be processed. It then acquires the most curved text from the initial curved text and performs principal component analysis based on the position information of the most curved text. The seal rotation angle can be quickly and accurately obtained based on the principal component analysis results. The initial text position information is then corrected based on the seal rotation angle to obtain a relatively accurate correction result. Furthermore, the corrected curved text is straightened to obtain corrected text slices corresponding to each initial text. Character recognition can then be performed on these corrected text slices to accurately obtain the character recognition result corresponding to the seal image. Attached Figure Description

[0074] Figure 1 This is a flowchart illustrating a method for recognizing characters on a seal in one embodiment;

[0075] Figure 2 This is a schematic diagram of the stamp image to be processed in one embodiment;

[0076] Figure 3 This is a schematic diagram showing the area of ​​the slice of the most curved text and the area of ​​the corresponding minimum bounding rectangle in one embodiment.

[0077] Figure 4 This is a schematic diagram of the mask pattern in one embodiment;

[0078] Figure 5 This is a schematic diagram of the segmentation graph of the most curved text in one embodiment;

[0079] Figure 6 This is a schematic diagram of a binarized image in one embodiment;

[0080] Figure 7 This is a schematic diagram illustrating the correction of horizontal text slices in one embodiment;

[0081] Figure 8 This is a schematic diagram of the maximum curvature text after correction in one embodiment;

[0082] Figure 9 This is a schematic diagram of a background image in one embodiment;

[0083] Figure 10 This is a schematic diagram of a mapped image in one embodiment;

[0084] Figure 11 This is a schematic diagram of image cropping in one embodiment;

[0085] Figure 12 This is a schematic diagram of a corrected curved text slice of the most curved text in one embodiment;

[0086] Figure 13 This is a structural block diagram of a character recognition device for a seal in one embodiment;

[0087] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0089] In one embodiment, such as Figure 1 As shown, a method for recognizing characters on a seal is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0090] Step 101: Perform text detection and text classification on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text.

[0091] Text detection involves detecting the text in the stamp image to be processed, text classification distinguishes between curved and horizontal text, and positional information refers to the coordinate information of each pixel of each text. In one embodiment, the terminal can establish a Cartesian coordinate system with the upper left corner of the stamp image to be processed as the origin. This Cartesian coordinate system has two coordinate axes, X and Y. The coordinate information of each text pixel is obtained based on this Cartesian coordinate system. Of course, the terminal can also establish a Cartesian coordinate system with other points in the stamp image (such as the upper right corner, lower left corner, or lower right corner) as the origin. This application embodiment does not limit this.

[0092] In some embodiments, the terminal can perform text detection on the stamp image to be processed to identify the positional information of each initial text. The terminal can also classify the identified initial text into curved and straight lines to distinguish between curved and horizontal text. Curved text is text displayed in a curved form, while horizontal text is text displayed in a straight line. The text detection can be implemented using OCR (optical character recognition) or neural network algorithms such as CTPN (connection text proposal network).

[0093] In some embodiments, the terminal can perform text detection on the seal image to be processed using the DBNet (differentiable Binarization net) algorithm model. After detection, all text slices and their position information can be obtained. The DBNet algorithm model can detect text in any direction, and due to its excellent post-processing performance, it can efficiently detect text in circular seals.

[0094] In some embodiments, the initial text includes curved text; in other embodiments, the initial text includes both curved and horizontal text.

[0095] In one embodiment, before performing text detection and text classification on the seal image to be processed, the method further includes: obtaining the original document image and performing circular seal detection on the original document image; cropping the original document image containing the circular seal to obtain the seal image to be processed.

[0096] The terminal can use the RetinaNet object detection algorithm to acquire the original ticket image and then perform detection and cropping on the original ticket image to obtain the seal image to be processed. The RetinaNet algorithm is a deep learning-based object detection algorithm with fast inference speed and good detection performance.

[0097] Step 102: Obtain the maximum curved text in the initial curved text, and perform principal component analysis based on the position information of the maximum curved text.

[0098] Among them, the most curved text refers to the curved text with the largest area among all curved texts, and the location information of the most curved text refers to the coordinate information of each pixel in the text slice corresponding to the most curved text. The terminal can perform principal component analysis based on the location information of the most curved text to make the analysis results of the stamp image more accurate.

[0099] Principal component analysis is a multivariate statistical method for examining the correlation between multiple variables. It studies how to reveal the internal structure between multiple variables through a few principal components, that is, to derive a few principal components from the original variables so that they retain as much information as possible about the original variables and are uncorrelated with each other.

[0100] In some embodiments, the terminal can perform principal component analysis based on the coordinate information of each pixel in the text slice corresponding to the most curved text, using the cov(Covariance) function of the numpy (mathematical processing) module in the Python library.

[0101] Step 103: Calculate the rotation angle based on the principal component analysis results.

[0102] The principal component analysis results include multiple eigenvalues ​​and their corresponding eigenvectors. The rotation angle of the most curved text is exactly the same as the rotation angle of the seal in the image to be processed. Therefore, the rotation angle calculated by the terminal based on the principal component analysis results is the same as the rotation angle of the seal in the image to be processed.

[0103] In some embodiments, the terminal can determine the stamp rotation angle based on the angle between the eigenvector of the first target eigenvalue with the largest eigenvalue and the coordinate axis. It is understood that the eigenvalues ​​obtained through principal component analysis can be used to describe the amount of information contained in the direction of the corresponding eigenvector. Accordingly, the eigenvector of the first target eigenvalue is highly likely to be considered as the eigenvector representing the most curved text. Therefore, based on the angle between the eigenvector corresponding to the first target eigenvalue and the coordinate axis, the stamp rotation angle can be obtained quickly and accurately.

[0104] In some embodiments, the terminal can sort multiple feature values ​​according to their magnitude to obtain a first target feature value with the largest feature value and a second target feature value with the second largest feature value. A first angle between the feature vector of the first target feature value and a first coordinate axis is determined, and a second angle between the feature vector of the second target feature value and a second coordinate axis is determined. The seal rotation angle is then obtained based on the first and second angles.

[0105] Step 104: Correct the position information of the initial text based on the stamp rotation angle, and straighten the curved text after correction to obtain the corrected text slices corresponding to each initial text.

[0106] Specifically, the terminal can correct the position information of each initial text based on the stamp rotation angle. If the initial text includes initial horizontal text, the corrected text slice corresponding to the initial horizontal text can be obtained after correction. If the initial text includes initial curved text, the curved text after correction will be straightened to obtain the corrected text slice corresponding to the initial curved text.

[0107] In some embodiments, the correction of the positional information of the initial text can specifically be based on the rotation angle of the seal and the rotation center of the seal image to correct the coordinate information of each pixel of the initial text.

[0108] Step 105: Perform character recognition on the corrected text slices to obtain the character recognition results corresponding to the seal image.

[0109] In some embodiments, the corrected text slice includes a corrected curved text slice. In other embodiments, the corrected text slice includes a corrected horizontal text slice and a corrected curved text slice. The text recognition of the corrected text slice can refer to the text recognition of the corrected curved text slice, or it can refer to the text recognition of all corrected horizontal text slices and all corrected curved text slices respectively.

[0110] The terminal can use OCR (optical character recognition), SegLink (segmented linking), TextBoxes (text boxes), or CRNN-CTC (convolutional recurrent neural network-based temporal classification) to recognize characters in each text slice. The character recognition result refers to the output text information after recognition.

[0111] The seal text recognition method in the above embodiments obtains the initial text position information by performing text detection and text classification on the seal image to be processed. By obtaining the maximum curved text in the initial curved text, and performing principal component analysis based on the position information of the maximum curved text, the seal rotation angle can be obtained quickly and accurately based on the principal component analysis results. Then, the position information of the initial text is corrected based on the seal rotation angle to obtain a relatively accurate correction result. Furthermore, the curved text after correction is straightened to obtain corrected text slices corresponding to each initial text. Then, text recognition can be performed on the corrected text slices to accurately obtain the text recognition result corresponding to the seal image.

[0112] In one embodiment, the initial text further includes initial horizontal text. Step 101 involves performing text detection and text classification on the seal image to be processed to obtain the position information of the initial text, which may include:

[0113] Step 1011: Perform text detection on the seal image to be processed to obtain text slices in the seal image.

[0114] In some embodiments, the terminal can perform text detection on the seal image to be processed using the DBNet (differentiable Binarization net) algorithm model, and after detection, it can obtain text slices and the position information of the text slices.

[0115] It should be noted that, in some embodiments, the text slices include horizontal text slices and curved text slices. After the terminal performs text detection on the stamp image to be processed, it obtains the position information of the curved text slices, horizontal text slices, and all text slices in the stamp image.

[0116] Step 1012: Determine the minimum bounding rectangle corresponding to each text slice, and calculate the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle. Text slices with a ratio less than a preset threshold are designated as curved text slices, and text slices with a ratio greater than or equal to the preset threshold are designated as horizontal text slices.

[0117] See Figure 2 Taking a round seal as an example, Figure 2 In the example, the text corresponding to "XX Seal Chain YY Company" is curved text, while the text corresponding to "Contract Seal" is horizontal text. Clearly, the area of ​​the curved text slice is significantly smaller than the area of ​​its smallest bounding rectangle, while the area of ​​the horizontal text slice is close to the area of ​​its smallest bounding rectangle, with a ratio close to 1. Please continue reading. Figure 3 ,by Figure 2Taking the most curved text in the text as an example, Figure 3 In the diagram, A represents the slice shape of the most curved text, and B represents the minimum bounding rectangle corresponding to the most curved text. The ratio between the area of ​​the text slice of the most curved text and the area of ​​the corresponding minimum bounding rectangle is the ratio between the area of ​​A and the area of ​​B.

[0118] Based on this, the terminal can determine the minimum bounding rectangle corresponding to each text slice and calculate the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle. Text slices with a ratio less than a preset threshold are designated as curved text slices, while text slices with a ratio greater than or equal to the preset threshold are designated as horizontal text slices. The preset threshold can be a pre-defined value, such as 0.9.

[0119] Step 1013: Use the position information corresponding to the curved text slice as the position information of the initial curved text, and use the position information corresponding to the horizontal text slice as the position information of the initial horizontal text.

[0120] Specifically, the terminal can use the coordinate information of all pixels in the curved text slice as the initial position information of the curved text, and use the coordinate information of all pixels in the horizontal text slice as the initial position information of the horizontal text.

[0121] In the above embodiments, by comparing the area of ​​the text slice with the area of ​​the minimum bounding rectangle corresponding to the text slice, curved text and horizontal text can be quickly distinguished, so that subsequent adaptive processing of curved text and horizontal text can be performed separately to obtain more accurate processing results.

[0122] In some embodiments, performing principal component analysis based on the location information of the most curved text may include:

[0123] Step 1021: Based on the position information of the most curved text, obtain the mask image of the most curved text.

[0124] Among them, you can refer to Figure 2 For example, the curved text corresponding to "XX Seal Chain YY Company" is... Figure 2 For the most curved text in the image, the terminal can obtain a mask image of the most curved text based on the coordinate information of each pixel. This mask image can be found in [reference needed]. Figure 4 .

[0125] In some embodiments, the terminal can set the pixel value of the pixel with the most curved text in the stamp image to 0, and set the pixel value of the pixel in the area other than the most curved text in the stamp image to 255, thereby obtaining a mask image.

[0126] Step 1022: Based on the stamp image and mask image, obtain the segmentation image of the text with the maximum curvature.

[0127] Specifically, the terminal can Figure 4 mask pattern and Figure 2 The segmentation graphic is obtained by overlaying the images of the seals. For the segmentation graphic of the most curved text, please refer to [reference needed]. Figure 5 The segmentation graph of the maximum curvature text can be understood as the graph that segments the maximum curvature text from the stamp image to be processed, so that the maximum curvature text is independent of other text in the stamp.

[0128] Step 1023: Binarize the segmented image to obtain the binarized image of the most curved text, and determine the coordinates of the target pixel representing the curved text in the binarized image.

[0129] In some embodiments, the terminal can set the grayscale value of the pixel containing the largest curved text in the segmented image to 255, and set the grayscale value of the pixels in the image other than the largest curved text to 0, to obtain a binarized image. The binarized image exhibits a clear black and white effect, which can be referenced in [reference needed]. Figure 6 .

[0130] Furthermore, the terminal can determine the coordinates of the target pixel representing the curved text in the binarized image for use in subsequent principal component analysis.

[0131] Step 1024: Perform principal component analysis based on the coordinates of the target pixel to obtain the principal component analysis results. The principal component analysis results include multiple eigenvalues ​​and the eigenvectors corresponding to each eigenvalue.

[0132] In the above embodiments, a mask image of the most curved text is obtained based on the location information of the most curved text. Then, a segmentation image of the most curved text is obtained based on the stamp image and the mask image to extract the most curved text separately from the stamp image. Binarization processing is performed on the segmentation image of the most curved text to distinguish the pixels of the most curved text from the pixels of the background, making it easier to confirm the coordinates of the pixels of the most curved text and the pixels of the background. The obtained coordinate values ​​are very accurate. Principal component analysis is performed using the coordinates of the pixels of the most curved text as the coordinates of the target pixels, making the results obtained after principal component analysis more accurate.

[0133] In one embodiment, step 1024, which involves performing principal component analysis based on the coordinates of the target pixel to obtain the principal component analysis result, may include:

[0134] Step 10241: Determine the x-coordinate and y-coordinate values ​​of the target pixel.

[0135] Specifically, the coordinates of the i-th target pixel can be denoted as (xi, yi), with the horizontal coordinate value being xi and the vertical coordinate value being yi.

[0136] Step 10242: For each target pixel, determine the covariance matrix based on the corresponding x-coordinate and y-coordinate values ​​of the target pixel.

[0137] The terminal can use the `cov` function to calculate the covariance of the x-coordinate itself, the covariance between the x-coordinate and y-coordinate, the covariance of the y-coordinate itself, and the covariance between the y-coordinate and x-coordinate for each target pixel, thus obtaining a covariance matrix. The covariance helps to understand how each target pixel changes relative to the average value.

[0138] In one embodiment, the covariance cov(x,y) between the x and y coordinates of the i-th target pixel is obtained using the following formula:

[0139]

[0140] Where xi is the x-coordinate of the i-th target pixel, yi is the y-coordinate of the i-th target pixel, and n is the number of target pixels. The average of the x-coordinates of all target pixels. This is the average of the ordinates of all target pixels;

[0141] Similarly, the covariance cov(x,x) of the x-coordinate of the i-th target pixel is obtained using the following formula:

[0142]

[0143] The covariance cov(y,x) between the ordinate and abscissa of the i-th target pixel is obtained using the following formula:

[0144]

[0145] The covariance cov(y,y) of the ordinate of the i-th target pixel is obtained using the following formula:

[0146]

[0147] The covariance matrix C is obtained from the covariances cov(x,y), cov(x,x), cov(y,x), and cov(y,y). The covariance matrix C satisfies the following formula:

[0148]

[0149] Step 10243: Perform principal component analysis based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results.

[0150] The principal component analysis (PCA) results include multiple eigenvalues ​​and their corresponding eigenvectors. The terminal performs eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors of the covariance matrix, thus obtaining the PCA results.

[0151] In the above embodiments, the terminal obtains the covariance data required for principal component analysis based on the actual horizontal and vertical coordinates of each target pixel. The obtained covariance data is very accurate data calculated in real time. The covariance matrix is ​​obtained based on the covariance, and then principal component analysis is performed on the covariance matrix to obtain the principal component analysis result, making the principal component analysis result very accurate.

[0152] In one embodiment, step 103, calculating the rotation angle based on the principal component analysis results, may include:

[0153] Step 1031: Sort the multiple feature values ​​according to their size, and obtain the first target feature value with the largest feature value and the second target feature value with the second largest feature value.

[0154] Among them, the first target feature value and the second target feature value are the two feature values ​​with the highest correlation. The first target feature value can be denoted as k1 and the second target feature value can be denoted as k2.

[0155] Step 1032: Determine the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis, and determine the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis.

[0156] Specifically, the first coordinate axis can be the X-axis, and the second coordinate axis can be the Y-axis; the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis refers to the angle between the projection of the eigenvector of the first target eigenvalue onto the rectangular coordinate system and the X-axis, and the first angle can be denoted as θ1; the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis refers to the angle between the projection of the eigenvector of the second target eigenvalue onto the rectangular coordinate system and the Y-axis, and the second angle can be denoted as θ2.

[0157] Step 1033: When the difference between the first included angle and the second included angle is greater than a preset difference, the opposite number of the first included angle is taken as the stamp rotation angle.

[0158] Specifically, when the first included angle θ1 and the second included angle θ2 are greater than a preset difference, -θ1 can be used as the stamp rotation angle.

[0159] Step 1034: When the difference between the first included angle and the second included angle is less than or equal to a preset difference, compare the slope of the feature vector of the first target feature value with the slope of the feature vector of the second target feature value, and take the included angle corresponding to the feature vector with the smaller slope as the stamp rotation angle.

[0160] Specifically, the terminal obtains the slope of the feature vector of the first target feature value and the slope of the feature vector of the second target feature value. The slope of the feature vector of the first target feature value is denoted as t1, and the slope of the feature vector of the second target feature value is denoted as t2. When the first angle θ1 and the second angle θ2 are less than or equal to a preset difference, t1 and t2 are compared. If t1 is less than t2, then θ1 is taken as the stamp rotation angle; if t2 is less than t1, then θ2 is taken as the stamp rotation angle.

[0161] In the above embodiments, as mentioned earlier, the results of principal component analysis using the coordinates of the target pixels are very accurate. The rotation angle calculated based on the principal component analysis results, and the eigenvectors of the first and second target eigenvalues ​​obtained through principal component analysis, are the two eigenvectors most correlated with the most curved text. Therefore, based on the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis, and the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis, the stamp rotation angle representing the rotation angle of the most curved text can be obtained, and the obtained stamp rotation angle is also very accurate. Furthermore, by confirming the difference between the first and second angles, when the difference is greater than a preset difference, the opposite of the first angle is taken as the stamp rotation angle; when the difference is less than or equal to the preset difference, the slopes of the eigenvectors of the first and second target eigenvalues ​​are compared, and the angle corresponding to the eigenvector with the smaller slope is taken as the stamp rotation angle. The final obtained stamp rotation angle is also the rotation angle with the highest correlation.

[0162] In some embodiments, the initial text further includes initial horizontal text, and the corrected text slices include corrected horizontal text slices and corrected curved text slices. The positional information of the initial text is corrected based on the stamp rotation angle, and the corrected curved text is straightened to obtain corrected text slices corresponding to each initial text, including:

[0163] Step 1041: Calculate the seal rotation matrix based on the seal rotation angle, and correct the position information of each initial horizontal text based on the seal rotation matrix to obtain corrected horizontal text slices.

[0164] In some embodiments, the terminal can obtain the seal rotation matrix using the getRotationMatrix2D (two-dimensional rotation matrix) function in OpenCV (Open Computer Vision) based on the seal rotation angle and the rotation center of the seal image to be processed.

[0165] In some embodiments, the positional information of each initial horizontal text is corrected based on the stamp rotation matrix to obtain corrected horizontal text slices, including:

[0166] Step 10411: Correct the position information of each initial horizontal text based on the seal rotation matrix to obtain a first-corrected slice.

[0167] The terminal uses the matrix multiplication function in NumPy matrix multiplication to multiply the stamp rotation matrix with the coordinates of each pixel of the initial horizontal text to obtain the corrected coordinates of each pixel. Based on the corrected coordinates, the position information of each initial horizontal text is corrected to obtain a first-stage corrected slice.

[0168] Step 10412: Based on the coordinates of the four vertices of the first-correction slice, perform a second-correction process on the first-correction slice to obtain the corrected horizontal text slice.

[0169] The four vertices of the first-corrected slice are the top-left, bottom-left, top-right, and bottom-right vertices. Specifically, the terminal can perform secondary correction processing on the first-corrected slice based on the coordinates of the four vertices, which can reduce the impact of image quality, noise, and other interference on the accuracy of correction, thereby obtaining a corrected horizontal text slice.

[0170] Please refer to Figure 2 , Figure 2 In the example, the text corresponding to "Contract Seal" is horizontal text. For example, the corresponding corrected horizontal text slice is as follows: Figure 7 As shown.

[0171] Step 1042: Correct the position information of the most curved text according to the stamp rotation angle, straighten the corrected most curved text, and obtain the corrected curved text slice of the most curved text.

[0172] In one embodiment, the terminal performs angular transformation on the coordinate information of each pixel in the most curved text based on the seal rotation angle and the rotation center of the seal in the seal image, obtaining the transformed coordinates of each pixel. Based on these transformed coordinates, the most curved text is corrected to obtain the corrected most curved text. For example, the corrected most curved text is as follows: Figure 8 As shown.

[0173] In one embodiment, straightening the corrected text with the maximum curvature to obtain a corrected curvature text slice of the maximum curvature text may include:

[0174] Step 10421: Create a background image.

[0175] The background image is composed of pixels with all grayscale values ​​of 0. A pure black image can be used as the background image. (See reference for background images.) Figure 9 .

[0176] Step 10422: Based on the polar coordinate information of the pixels in the background image, convert the coordinates of the pixels of the most curved text after correction to the corresponding polar coordinates.

[0177] The conversion formulas between polar coordinates and rectangular coordinates are x = r * cos(θ) and y = r * sin(θ), where x and y represent the x-coordinate and y-coordinate of a pixel in the rectangular coordinate system, respectively, and (r, θ) are the polar coordinates of the corresponding pixel.

[0178] Specifically, the terminal can convert the coordinates of the pixels of the most curved text after correction to the corresponding polar coordinates according to the conversion formula between polar coordinates and rectangular coordinates, so as to obtain the most curved text in polar coordinates.

[0179] Step 10423: Map the maximum curvature text, converted to polar coordinates, onto the background image to obtain the mapped image.

[0180] The resulting mapped image can be found in [reference]. Figure 10 .

[0181] Step 10424: Crop the mapped image based on the text with the largest curvature in the mapped image to obtain a cropped image.

[0182] The resulting cropped image can be found in [reference]. Figure 11 Specifically, the terminal can crop the mapped image based on the region where the most curved text is located, thus obtaining a cropped image.

[0183] Step 10425: Replace the pixels belonging to the background image in the cropped image with the background pixels of the maximum curved text to obtain the corrected curved text slice of the maximum curved text.

[0184] For details, please refer to the corrected curved text slices of the most curved text. Figure 12 Specifically, the terminal can replace all pixels with a grayscale value of 0 that belong to the background image in the cropped image with the background pixels of the maximum curvature text, so that the background pixels of the corrected curvature text slice of the maximum curvature text are unified with the background pixels of the maximum curvature text.

[0185] Step 1043: Determine the other curved texts in the initial curved text except for the largest curved text, correct the position information of the other curved texts based on the stamp rotation matrix, and straighten the other curved texts after correction to obtain the corrected curved text slices of each other curved text.

[0186] In some embodiments, the terminal uses the matrix multiplication function in NumPy matrix multiplication to multiply the coordinates of each pixel of the seal rotation matrix with the coordinates of each pixel of the other curved text to obtain the corrected coordinates of each pixel. Based on the corrected coordinates, the position information of each other curved text is corrected to obtain the other curved text after correction.

[0187] The method for straightening other curved texts after correction to obtain corrected curved text slices can refer to the method for straightening the largest curved text after correction in steps 10421 to 10425.

[0188] In the above embodiments, a first-corrected slice is obtained by correcting the horizontal text according to the stamp rotation matrix, and a corrected horizontal text slice is obtained by processing the first-corrected slice based on four-point coordinates; the position information of the most curved text is corrected according to the stamp rotation angle, and the corrected most curved text is straightened to obtain a corrected curved text slice of the most curved text; other curved texts besides the most curved text are corrected according to the stamp rotation matrix, and the other curved texts are straightened to obtain corrected curved text slices corresponding to each other curved text; that is, this application does not use a single text processing procedure to process all different texts, but rather processes different texts separately to ensure that different texts have a processing method that is very suitable for them, so as to obtain very accurate processing results.

[0189] In one embodiment, step 105, performing character recognition on the corrected text slice to obtain the character recognition result corresponding to the seal image, may include:

[0190] Step 1051: Mirror the corrected text slice to obtain a mirrored text slice.

[0191] The mirroring of the corrected text slices can refer to mirroring the curved text slices to obtain mirrored curved text slices; or it can refer to mirroring all horizontal text slices and all curved text slices separately to obtain mirrored horizontal text slices and mirrored curved text slices.

[0192] Specifically, mirroring refers to the terminal rotating each corrected text slice 180° clockwise or 180° counterclockwise to obtain a mirrored text slice corresponding to each corrected text slice.

[0193] Step 1052: Perform character recognition on the corrected text slice to obtain the first recognition result and the confidence level of the first recognition result.

[0194] In some embodiments, the terminal uses a CRNN-CTC recognition model to perform text recognition on the corrected curved text slices. Alternatively, the terminal can use the CRNN-CTC recognition model to perform text recognition on all corrected horizontal text slices and all corrected curved text slices separately. After recognition, the text recognition result for each corrected text slice is obtained, i.e., the first recognition result. Confidence level is the reliability of the text recognition result; the higher the confidence level, the more reliable the first recognition result.

[0195] Step 1053: Perform text recognition on the mirrored text slice to obtain a second recognition result and the confidence level of the second recognition result.

[0196] Specifically, text recognition on mirrored text slices refers to the terminal using a CRNN-CTC recognition model to recognize text on mirrored curved text slices. Alternatively, the terminal can use the CRNN-CTC recognition model to recognize text on all mirrored horizontal text slices and all mirrored curved text slices separately. After recognition, the text recognition result for each mirrored text slice is obtained, i.e., the second recognition result. The higher the confidence level, the more reliable the second recognition result.

[0197] Step 1054: The recognition result with higher confidence among the first recognition result and the second recognition result is taken as the text recognition result corresponding to the seal image.

[0198] Among them, the recognition result with higher confidence between the first recognition result and the second recognition result is used as the text recognition result corresponding to the seal image, which can obtain a more accurate text recognition result; each text slice can determine the text recognition result corresponding to that text slice, and the sum of the text recognition results of all text slices is the text recognition result corresponding to the seal image.

[0199] In the above embodiments, the recognition results of the obtained corrected text slices are not directly used as the final text recognition results of the seal image. Instead, the recognition results of the corrected text slices and the mirrored text slices are obtained separately. The recognition results with higher confidence are used as the text recognition results corresponding to the seal image. This is to avoid the text of the corrected text slices after correction being displayed in reverse rather than in the forward direction. Therefore, the recognition results of the mirrored slices are combined for comprehensive judgment. By comparing the confidence levels, the recognition results with the highest reliability are selected as the final text recognition results of the seal image to make the recognition results more accurate.

[0200] In one embodiment, the character recognition method for a seal may include the following:

[0201] Obtain the original invoice image and perform circular seal detection on it. Crop the original invoice image containing the circular seal to obtain the seal image to be processed.

[0202] Text detection is performed on the stamp image to be processed, resulting in text slices within the stamp image. The minimum bounding rectangle corresponding to each text slice is determined, and the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle is calculated. Text slices with a ratio less than a preset threshold are designated as curved text slices, while those with a ratio greater than or equal to the preset threshold are designated as horizontal text slices. The positional information corresponding to the curved text slices is used as the initial positional information for the curved text, and the positional information corresponding to the horizontal text slices is used as the initial positional information for the horizontal text.

[0203] Based on the location information of the most curved text, a mask image of the most curved text is obtained. Based on the stamp image and the mask image, a segmentation image of the most curved text is obtained. The segmentation image is binarized to obtain a binary image of the most curved text, and the coordinates of the target pixels representing the curved text in the binary image are determined. The x-coordinate and y-coordinate values ​​of the target pixels are determined. For each target pixel, the covariance matrix is ​​determined based on its x-coordinate and y-coordinate values. Principal component analysis is performed based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results, which include multiple eigenvalues ​​and the eigenvectors corresponding to each eigenvalue.

[0204] Sort the multiple feature values ​​by size, and obtain the first target feature value with the largest feature value and the second target feature value with the second largest feature value. Determine the first angle between the feature vector of the first target feature value and the first coordinate axis, and determine the second angle between the feature vector of the second target feature value and the second coordinate axis. When the difference between the first angle and the second angle is greater than a preset difference, use the opposite of the first angle as the stamp rotation angle.

[0205] The seal rotation matrix is ​​calculated based on the seal rotation angle. The positional information of each initial horizontal text is then corrected based on this matrix to obtain a first-stage corrected slice. A second-stage correction is performed on the first-stage corrected slice based on the coordinates of its four vertices to obtain a corrected horizontal text slice.

[0206] The positional information of the most curved text is corrected based on the stamp rotation angle to obtain the corrected text. A background image is created, and the coordinates of the pixels of the corrected text are converted to corresponding polar coordinates based on the polar coordinate information of the background image. The converted text is then mapped onto the background image to obtain a mapped image. The mapped image is cropped based on the text of the most curved text, resulting in a cropped image. Pixels belonging to the background image in the cropped image are replaced with background pixels of the most curved text to obtain a slice of the corrected text.

[0207] Identify the other curved texts in the initial curved text except for the largest curved text, correct the position information of the other curved texts based on the stamp rotation matrix, and straighten the other curved texts after correction to obtain the corrected curved text slices of each other curved text.

[0208] The corrected text slice is mirrored to obtain a mirrored text slice. Character recognition is performed on the corrected text slice to obtain a first recognition result and its confidence level. Character recognition is then performed on the mirrored text slice to obtain a second recognition result and its confidence level. The recognition result with the higher confidence level between the first and second recognition results is taken as the character recognition result corresponding to the seal image.

[0209] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0210] Based on the same inventive concept, this application also provides a seal character recognition device for implementing the seal character recognition method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more seal character recognition device embodiments provided below can be found in the limitations of the seal character recognition method described above, and will not be repeated here.

[0211] In one embodiment, such as Figure 13 As shown, a character recognition device for a seal is provided, the device comprising:

[0212] The detection and classification module 1301 is used to perform text detection and text classification processing on the seal image to be processed, and to obtain the position information of the initial text, which includes the initial curved text.

[0213] Analysis module 1302 is used to obtain the maximum curved text in the initial curved text and perform principal component analysis based on the position information of the maximum curved text;

[0214] Calculation module 1303 is used to calculate the seal rotation angle based on the principal component analysis results;

[0215] The correction module 1304 is used to correct the position information of the initial text based on the rotation angle of the seal, and to straighten the curved text after correction to obtain the corrected text slices corresponding to each initial text.

[0216] The recognition module 1305 is used to perform text recognition on the corrected text slices to obtain the text recognition result corresponding to the seal image.

[0217] The seal text recognition device in the above embodiments obtains the initial text position information by performing text detection and text classification processing on the seal image to be processed. The analysis module obtains the maximum curved text in the initial curved text and performs principal component analysis based on the position information of the maximum curved text. The calculation module can quickly and accurately obtain the seal rotation angle of the seal image at any rotation angle. The correction module corrects the position information of the initial text based on the seal rotation angle to obtain a relatively accurate correction result. The corrected curved text is then straightened to obtain corrected text slices corresponding to each initial text. The recognition module performs text recognition on the corrected text slices to accurately obtain the text recognition result corresponding to the seal image.

[0218] In one embodiment, the initial text further includes initial horizontal text. The detection and classification module 1301 is further configured to: perform text detection on the stamp image to be processed to obtain text slices in the stamp image; determine the minimum bounding rectangle corresponding to each text slice, and calculate the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle; classify text slices with a ratio less than a preset threshold as curved text slices, and classify text slices with a ratio greater than or equal to the preset threshold as horizontal text slices; use the position information corresponding to the curved text slices as the position information of the initial curved text, and use the position information corresponding to the horizontal text slices as the position information of the initial horizontal text.

[0219] In the above embodiments, the detection and classification module 1301 can quickly distinguish curved text from horizontal text by comparing the area of ​​the text slice with the area of ​​the minimum bounding rectangle corresponding to the text, instead of treating all text indiscriminately. This allows for more accurate processing results when adaptive processing is performed on curved text and horizontal text separately in the future.

[0220] In one embodiment, the analysis module 1302 is further configured to: obtain a mask image of the maximum curved text based on the location information of the maximum curved text; obtain a segmentation image of the maximum curved text based on the stamp image and the mask image; perform binarization processing on the segmentation image to obtain a binarized image of the maximum curved text, and determine the coordinates of the target pixel point representing the curved text in the binarized image; perform principal component analysis based on the coordinates of the target pixel point to obtain the principal component analysis result, the principal component analysis result including multiple eigenvalues ​​and the eigenvectors corresponding to each eigenvalue.

[0221] In the above embodiments, the analysis module 1302 obtains the mask image of the most curved text based on the location information of the most curved text, and then obtains the segmentation image of the most curved text based on the stamp image and the mask image, so as to extract the most curved text separately from the stamp image. The segmentation image of the most curved text is binarized, which can distinguish the pixels of the most curved text from the pixels of the background, making it easier to confirm the coordinates of the pixels of the most curved text and the coordinates of the pixels of the background. The obtained coordinate values ​​are very accurate. The coordinates of the pixels of the most curved text are used as the coordinates of the target pixels for principal component analysis, making the results obtained after principal component analysis more accurate.

[0222] In one embodiment, the analysis module 1302 is further configured to: determine the abscissa and ordinate values ​​of the target pixel; for each target pixel, determine the covariance matrix based on the abscissa and ordinate values ​​of the corresponding target pixel; and perform principal component analysis based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results.

[0223] In the above embodiments, the analysis module 1302 obtains the covariance data required for principal component analysis based on the actual horizontal and vertical coordinate information of each target pixel. The obtained covariance data is very accurate data calculated in real time. The covariance matrix is ​​obtained based on the covariance, and principal component analysis is performed on the covariance matrix to obtain the principal component analysis result, making the principal component analysis result very accurate.

[0224] In one embodiment, the principal component analysis result includes multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; the calculation module 1303 is further configured to: sort the multiple eigenvalues ​​according to their magnitude, obtain the first target eigenvalue with the largest eigenvalue and the second target eigenvalue with the second largest eigenvalue; determine the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis, and determine the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis; when the difference between the first angle and the second angle is greater than a preset difference, take the opposite of the first angle as the stamp rotation angle; when the difference between the first angle and the second angle is less than or equal to the preset difference, compare the slope of the eigenvector of the first target eigenvalue with the slope of the eigenvector of the second target eigenvalue, and take the angle corresponding to the eigenvector with the smaller slope as the stamp rotation angle.

[0225] In the above embodiment, the calculation module 1303 calculates the rotation angle based on the principal component analysis results. As mentioned above, the results of principal component analysis using the coordinates of the target pixel are very accurate. The rotation angle calculated based on the principal component analysis results, the feature vectors of the first target feature value and the feature vector of the second target feature value obtained through principal component analysis are the two feature vectors with the highest correlation to the maximum curved text. Therefore, based on the first angle between the feature vector of the first target feature value and the first coordinate axis and the second angle between the feature vector of the second target feature value and the second coordinate axis, the stamp rotation angle that can represent the maximum curved text rotation angle can be obtained. The stamp rotation angle obtained in this way is also very accurate. Furthermore, by confirming the difference between the first included angle and the second included angle, when the difference between the first included angle and the second included angle is greater than a preset difference, the opposite of the first included angle is taken as the stamp rotation angle. When the difference between the first included angle and the second included angle is less than or equal to the preset difference, the slope of the feature vector of the first target feature value is compared with the slope of the feature vector of the second target feature value, and the included angle corresponding to the feature vector with the smaller slope is taken as the stamp rotation angle. The stamp rotation angle obtained in the end is also the rotation angle with the highest correlation.

[0226] In one embodiment, the initial text further includes initial horizontal text, and the corrected text slice includes a corrected horizontal text slice and a corrected curved text slice; the correction module 1304 is further configured to: calculate a seal rotation matrix based on the seal rotation angle, and correct the position information of each initial horizontal text based on the seal rotation matrix to obtain a corrected horizontal text slice; correct the position information of the most curved text based on the seal rotation angle, and straighten the corrected most curved text to obtain a corrected curved text slice of the most curved text; determine other curved texts in the initial curved text other than the most curved text, correct the position information of the other curved texts based on the seal rotation matrix, and straighten the corrected other curved texts to obtain corrected curved text slices of each other curved text.

[0227] In one embodiment, the correction module 1304 is further configured to: create a background image; convert the coordinates of the pixels of the corrected maximum curvature text to corresponding polar coordinates based on the polar coordinate information of the pixels in the background image; map the maximum curvature text converted to polar coordinates onto the background image to obtain a mapped image; crop the mapped image according to the maximum curvature text in the mapped image to obtain a cropped image; and replace the pixels belonging to the background image in the cropped image with the background pixels of the maximum curvature text to obtain a corrected curvature text slice of the maximum curvature text.

[0228] In one embodiment, the correction module 1304 is further configured to: perform correction processing on the position information of each initial horizontal text based on the stamp rotation matrix to obtain a first-correction slice; and perform a second-correction processing on the first-correction slice based on the coordinates of the four vertices of the first-correction slice to obtain a corrected horizontal text slice.

[0229] In the embodiment of the above-mentioned correction module 1304, the correction module 1304 performs correction processing on the horizontal text according to the stamp rotation matrix to obtain a first-correction slice, and processes the first-correction slice based on four-point coordinates to obtain a corrected horizontal text slice; the correction module 1304 corrects the position information of the most curved text according to the stamp rotation angle, and straightens the corrected most curved text to obtain a corrected curved text slice of the most curved text; the correction module 1304 performs correction processing on other curved texts other than the most curved text according to the stamp rotation matrix, and straightens the corrected other curved texts to obtain corrected curved text slices corresponding to each other curved text; that is, the correction module 1304 does not use a single text processing procedure to process all different texts, but processes different texts separately to ensure that different texts have a processing method that is very suitable for them, so as to obtain a very accurate processing result.

[0230] In one embodiment, the recognition module 1305 is further configured to: perform mirroring processing on the corrected text slice to obtain a mirrored text slice; perform character recognition on the corrected text slice to obtain a first recognition result and a confidence level of the first recognition result; perform character recognition on the mirrored text slice to obtain a second recognition result and a confidence level of the second recognition result; and take the recognition result with higher confidence level between the first recognition result and the second recognition result as the character recognition result corresponding to the seal image.

[0231] In the above embodiment, the recognition module 1305 does not directly use the recognition result of the obtained corrected text slice as the final text recognition result of the seal image. Instead, it obtains the recognition results of the corrected text slice and the mirrored text slice separately, and uses the recognition result with higher confidence as the text recognition result corresponding to the seal image. This is to avoid the text of the corrected text slice obtained after correction not being displayed in the forward direction, but possibly in the reverse direction. Therefore, it combines the recognition result of the mirrored slice to make a comprehensive judgment, and selects the recognition result with the highest reliability as the final text recognition result of the seal image by comparing the confidence levels, so as to make the recognition result more accurate.

[0232] The various modules in the aforementioned character recognition device for the seal can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0233] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for recognizing characters on a seal. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0234] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0235] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0236] The text detection and text classification processes are performed on the seal image to be processed to obtain the position information of the initial text, which includes the initial curved text.

[0237] Obtain the maximum curved text in the initial curved text, and perform principal component analysis based on the location information of the maximum curved text;

[0238] The seal rotation angle was calculated based on the principal component analysis results.

[0239] The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text.

[0240] The corrected text slices are subjected to character recognition to obtain the character recognition results corresponding to the seal image.

[0241] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0242] Text detection is performed on the stamp image to be processed to obtain text slices in the stamp image; the minimum bounding rectangle corresponding to each text slice is determined, and the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle is calculated. Text slices with a ratio less than a preset threshold are identified as curved text slices, and text slices with a ratio greater than or equal to the preset threshold are identified as horizontal text slices; the position information corresponding to the curved text slices is used as the initial position information of the curved text, and the position information corresponding to the horizontal text slices is used as the initial position information of the horizontal text.

[0243] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0244] Based on the location information of the most curved text, a mask image of the most curved text is obtained; based on the seal image and the mask image, a segmentation image of the most curved text is obtained; the segmentation image is binarized to obtain a binarized image of the most curved text, and the coordinates of the target pixel representing the curved text in the binarized image are determined; principal component analysis is performed based on the coordinates of the target pixel to obtain the principal component analysis results, which include multiple eigenvalues ​​and the eigenvectors corresponding to each eigenvalue.

[0245] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0246] Determine the x-coordinate and y-coordinate values ​​of the target pixels; for each target pixel, determine the covariance matrix based on its x-coordinate and y-coordinate values; perform principal component analysis based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results.

[0247] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0248] Sort the multiple feature values ​​according to their magnitude, obtain the first target feature value with the largest feature value and the second target feature value with the second largest feature value; determine the first angle between the feature vector of the first target feature value and the first coordinate axis, and determine the second angle between the feature vector of the second target feature value and the second coordinate axis; when the difference between the first angle and the second angle is greater than a preset difference, take the opposite of the first angle as the seal rotation angle; when the difference between the first angle and the second angle is less than or equal to the preset difference, compare the slope of the feature vector of the first target feature value with the slope of the feature vector of the second target feature value, and take the angle corresponding to the feature vector with the smaller slope as the seal rotation angle.

[0249] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0250] The seal rotation matrix is ​​calculated based on the seal rotation angle, and the position information of each initial horizontal text is corrected based on the seal rotation matrix to obtain a corrected horizontal text slice. The position information of the most curved text is corrected based on the seal rotation angle, and the corrected most curved text is straightened to obtain a corrected curved text slice of the most curved text. Other curved texts other than the most curved text are identified in the initial curved text, and the position information of the other curved texts is corrected based on the seal rotation matrix. The corrected other curved texts are straightened to obtain corrected curved text slices of each other curved text.

[0251] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0252] Create a background image; based on the polar coordinate information of the pixels in the background image, convert the coordinates of the pixels of the corrected maximum curvature text to the corresponding polar coordinates; map the converted maximum curvature text to the background image to obtain a mapped image; crop the mapped image according to the maximum curvature text in the mapped image to obtain a cropped image; replace the pixels in the cropped image that belong to the background image with the background pixels of the maximum curvature text to obtain a slice of the corrected curvature text of the maximum curvature text.

[0253] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0254] The positional information of each initial horizontal text is corrected based on the stamp rotation matrix to obtain a first-corrected slice; based on the coordinates of the four vertices of the first-corrected slice, a second-corrected slice is performed to obtain a corrected horizontal text slice.

[0255] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0256] The corrected text slice is mirrored to obtain a mirrored text slice; the corrected text slice is subjected to character recognition to obtain a first recognition result and a confidence level of the first recognition result; the mirrored text slice is subjected to character recognition to obtain a second recognition result and a confidence level of the second recognition result; the recognition result with higher confidence level between the first recognition result and the second recognition result is taken as the character recognition result corresponding to the seal image.

[0257] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above embodiments of the character recognition method for each seal.

[0258] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described embodiments of the character recognition method for each seal.

[0259] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0260] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0261] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for recognizing characters on a seal, characterized in that, The method includes: Text detection and text classification are performed on the seal image to be processed to obtain the initial text position information, which includes initial curved text; the position information is the coordinate information of each pixel point of each text. Obtain the largest curved text in the initial curved text, and based on the position information of the largest curved text, obtain the mask pattern of the largest curved text; Based on the stamp image and the mask image, obtain the segmented image of the text with the maximum curvature; The segmented image is binarized to obtain a binarized image of the most curved text, and the coordinates of the target pixel representing the curved text in the binarized image are determined. Principal component analysis is performed based on the coordinates of the target pixel to obtain the principal component analysis results, which include multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue. The seal rotation angle was calculated based on the principal component analysis results. The position information of the initial text is corrected based on the rotation angle of the seal, and the curved text after correction is straightened to obtain the corrected text slices corresponding to each initial text. The corrected text slice is subjected to character recognition to obtain the character recognition result corresponding to the seal image.

2. The method according to claim 1, characterized in that, The initial text also includes initial horizontal text. The text detection and classification processing of the seal image to be processed to obtain the positional information of the initial text includes: Text detection is performed on the seal image to be processed to obtain text slices in the seal image; Determine the minimum bounding rectangle corresponding to each text slice, and calculate the ratio between the area of ​​each text slice and the area of ​​its corresponding minimum bounding rectangle. Text slices with a ratio less than a preset threshold are designated as curved text slices, and text slices with a ratio greater than or equal to the preset threshold are designated as horizontal text slices. The position information corresponding to the curved text slice is used as the position information of the initial curved text, and the position information corresponding to the horizontal text slice is used as the position information of the initial horizontal text.

3. The method according to claim 1, characterized in that, The principal component analysis based on the coordinates of the target pixel is performed to obtain the principal component analysis results, including: Determine the x-coordinate and y-coordinate values ​​of the target pixel; For each target pixel, the covariance matrix is ​​determined based on the x-coordinate and y-coordinate values ​​of the corresponding target pixel. Principal component analysis is performed based on the covariance matrix corresponding to each target pixel to obtain the principal component analysis results.

4. The method according to claim 1, characterized in that, The principal component analysis results include multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue. The calculation of the seal rotation angle based on the principal component analysis results includes: Sort the multiple feature values ​​by size, and obtain the first target feature value with the largest feature value and the second target feature value with the second largest feature value; Determine the first angle between the eigenvector of the first target eigenvalue and the first coordinate axis, and determine the second angle between the eigenvector of the second target eigenvalue and the second coordinate axis; When the difference between the first included angle and the second included angle is greater than a preset difference, the opposite of the first included angle is taken as the rotation angle of the seal. When the difference between the first included angle and the second included angle is less than or equal to a preset difference, the slope of the feature vector of the first target feature value is compared with the slope of the feature vector of the second target feature value, and the included angle corresponding to the feature vector with the smaller slope is taken as the seal rotation angle.

5. The method according to claim 1, characterized in that, The initial text also includes initial horizontal text, and the corrected text slices include corrected horizontal text slices and corrected curved text slices; The process of correcting the position information of the initial text based on the rotation angle of the seal, and straightening the corrected curved text to obtain corrected text slices corresponding to each initial text, includes: The seal rotation matrix is ​​calculated based on the seal rotation angle, and the position information of each initial horizontal text is corrected based on the seal rotation matrix to obtain a corrected horizontal text slice. The position information of the most curved text is corrected according to the rotation angle of the seal, and the corrected most curved text is straightened to obtain a slice of the corrected curved text. Identify the other curved texts in the initial curved text besides the largest curved text, correct the position information of the other curved texts based on the stamp rotation matrix, and straighten the other curved texts after correction to obtain the corrected curved text slices of each other curved text.

6. The method according to claim 5, characterized in that, The process of straightening the most curved text after correction to obtain a corrected curved text slice of the most curved text includes: Create a background image; Based on the polar coordinate information of the pixels in the background image, the coordinates of the pixels of the most curved text after correction are converted into corresponding polar coordinates. The maximum curvature text, converted to polar coordinates, is mapped onto the background image to obtain a mapped image; The mapped image is cropped based on the most curved text in the mapped image to obtain a cropped image; The pixels belonging to the background image in the cropped image are replaced with the background pixels of the maximum curved text to obtain the corrected curved text slice of the maximum curved text.

7. The method according to claim 5, characterized in that, The step of correcting the position information of each initial horizontal text based on the seal rotation matrix to obtain corrected horizontal text slices includes: Based on the seal rotation matrix, the position information of each initial horizontal text is corrected to obtain a first-corrected slice. Based on the coordinates of the four vertices of the first-correction slice, a second-correction process is performed on the first-correction slice to obtain a corrected horizontal text slice.

8. The method according to any one of claims 1 to 7, characterized in that, The step of performing character recognition on the corrected text slice to obtain the character recognition result corresponding to the seal image includes: The corrected text slice is mirrored to obtain a mirrored text slice; The corrected text slice is subjected to character recognition to obtain a first recognition result and the confidence level of the first recognition result; The mirrored text slice is subjected to text recognition to obtain a second recognition result and the confidence level of the second recognition result; The recognition result with higher confidence among the first and second recognition results is taken as the text recognition result corresponding to the seal image.

9. A character recognition device for a seal, characterized in that, The device includes: The detection and classification module is used to perform text detection and text classification processing on the seal image to be processed, and to obtain the position information of the initial text, which includes the initial curved text; the position information is the coordinate information of each pixel point of each text. The analysis module is used to obtain the maximum curved text in the initial curved text, and obtain a mask image of the maximum curved text based on the position information of the maximum curved text; obtain a segmentation image of the maximum curved text based on the stamp image and the mask image; perform binarization processing on the segmentation image to obtain a binarized image of the maximum curved text, and determine the coordinates of the target pixel points representing the curved text in the binarized image; perform principal component analysis based on the coordinates of the target pixel points to obtain principal component analysis results, the principal component analysis results including multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; The calculation module is used to calculate the seal rotation angle based on the principal component analysis results; The correction module is used to correct the position information of the initial text based on the rotation angle of the seal, and to straighten the curved text after correction to obtain the corrected text slices corresponding to each initial text. The recognition module is used to perform character recognition on the corrected text slices to obtain the character recognition result corresponding to the seal image.

Images