Contract seal identification and automatic completion method based on OCR technology

Abstract

The present application relates to the technical field of image processing, in particular to a contract seal identification and automatic completion method based on OCR technology. The seal image is obtained, the seal pixel points are screened by analyzing the color channel value difference of each pixel point due to the difference between the seal color and the paper and text color; the seal sub-region is divided based on the non-seal pixel point position distribution, the seal sub-region is aligned by the seal center and then rotated and spliced, the rotation coefficient is determined by combining the seal pixel point position relationship and the OCR recognition result, the seal effective index is calculated based on the rotation coefficient and the seal pixel point quantity characteristics, the geometric deformation and physical coverage are verified, and finally the sub-region integrity is quantitatively evaluated by the index to improve the recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a contract stitching seal recognition and automatic completion method based on OCR technology. BACKGROUND

[0002] In the contract signing and archiving process, the stitching seal (i.e. the complete seal pattern formed by the continuous seal on the edges of multiple pages of the contract) is a key anti-counterfeiting means for verifying the integrity and authenticity of the contract, which is used to ensure that the pages with the stitching seal are integrated, and can prevent the contract from being arbitrarily split, missing pages or misplacing pages, and avoid the contract content being arbitrarily changed or tampered with; therefore, it is very important to identify and complete the contract stitching seal.

[0003] In the prior art, when the integrity of the stitching seal on the contract is identified, OCR is usually used in isolation to verify the continuity of the text in the stitching seal, so as to determine the integrity of the seal; however, in actual scenarios, since the stitching seal covers multiple pages, after the seal is completed, the contract pages may be displaced, deflected, etc., which may cause the stitching seal in the collected seal image to be unable to be aligned in a standard manner, and at this time, the use of the OCR recognition result in isolation may easily lead to misjudgment, resulting in a large error in the integrity identification. SUMMARY

[0004] In order to solve the technical problem that in actual scenarios, since the stitching seal covers multiple pages, after the seal is completed, the contract pages may be displaced, deflected, etc., which may cause the stitching seal in the collected seal image to be unable to be aligned in a standard manner, and at this time, the use of the OCR recognition result in isolation may easily lead to misjudgment, resulting in a large error in the integrity identification, the purpose of the present application is to provide a contract stitching seal recognition and automatic completion method based on OCR technology, and the technical solution adopted is as follows:

[0005] Obtain a seal image of a seal area on a contract, the seal image being an RGB image;

[0006] Based on the difference characteristics of the channel values of the pixel points in the seal image under the color channels, seal pixel points are screened out from all the pixel points; based on the position distribution of the non-seal pixel points in the seal image, the seal image is regionally divided to obtain all seal sub-regions;

[0007] Analyze the shape characteristics of each seal sub-region to determine the seal center corresponding to each seal sub-region; after aligning adjacent seal sub-regions according to the seal center, perform rotation splicing in different directions, so as to analyze the positional relationship between the seal pixel points, and combine with the OCR recognition result after splicing, to determine the rotation coefficient of each seal sub-region under different rotation splicing modes;

[0008] Based on the numerical characteristics of the rotation coefficients of each seal sub-region under the rotation splicing in different directions, and in combination with the number characteristics of the seal pixel points of each seal sub-region, a seal effective index of each seal sub-region is calculated for performing the effectiveness verification and completion judgment of the seal.

[0009] Further, the seal pixel point acquisition method comprises:

[0010] In the seal image, for any one pixel point, the difference value between the channel value of the pixel point in the R channel and the channel value in each remaining color channel is taken as a channel difference value, and the normalized value of the sum of the two channel difference values corresponding to the pixel point is taken as the R channel highlight value of the pixel point.

[0011] The product of the R channel highlight value of the pixel point and the R channel value of the pixel point is normalized to obtain the seal evaluation index of the pixel point.

[0012] In the seal image, the pixel point with a seal evaluation index greater than a preset seal evaluation threshold is taken as a seal pixel point.

[0013] Further, the seal sub-region acquisition method comprises:

[0014] All non-seal pixel points in the seal image are analyzed by using a Hough straight line transformation, so as to extract all straight lines as test straight lines.

[0015] The non-seal pixel points in the seal image are subjected to edge detection to obtain edge lines, and the edge lines coinciding with the test straight lines are taken as target edge lines, and the target edge lines with a length greater than or equal to the short side length of the seal image are taken as contract page edge lines.

[0016] In the seal image, the region surrounded by each adjacent two contract page edge lines and the seal image boundary is taken as a seal sub-region.

[0017] Further, the seal center acquisition method comprises:

[0018] In each seal sub-region, all seal pixel points are subjected to edge analysis and edge lines are obtained by using a Canny operator, and the region corresponding to each edge line is taken as a connected domain based on connected domain analysis.

[0019] When there is only one connected domain in a certain seal sub-region, in the connected domain of the seal sub-region, three seal pixel points are selected to obtain two line segments, and the intersection point of the perpendicular bisectors of the two line segments is taken as the seal center of the seal sub-region.

[0020] When there are at least two connected components in a certain stamp sub-region, obtain the centroid of each connected component. Select one of the two connected components with the greatest Euclidean distance between their centroids as the target region. In the target region, select three stamp pixels to obtain two line segments. Take the intersection of the perpendicular bisectors of these two line segments as the stamp center of the stamp sub-region.

[0021] Furthermore, the method for obtaining the rotation coefficient includes:

[0022] Sort all the seal sub-regions according to the page sorting direction of the contract to obtain the seal sub-region sequence;

[0023] Choose any one of the seal sub-regions as the region to be analyzed, and use the previous seal sub-region adjacent to the region to be analyzed as the comparison region of the region to be analyzed.

[0024] Rotation splicing methods include counterclockwise rotation splicing and clockwise rotation splicing;

[0025] In each rotation stitching method, the center of the stamp in the area to be analyzed is aligned with the center of the stamp in the comparison sub-area, and then the area to be analyzed is rotated. The rotation stops when the stamp pixels in the area to be analyzed are connected to the stamp pixels in the comparison area, and the stitching result is obtained.

[0026] In the stitching results of each rotation stitching method, the positional relationship between the stamp pixels in the area to be analyzed and the comparison area is analyzed to determine the stitching integrity factor of the area to be analyzed in the stitching results of each rotation stitching method.

[0027] OCR character recognition is performed on the splicing results under each rotation splicing method, thereby outputting the confidence level of the character recognition results corresponding to the region to be analyzed;

[0028] The normalized value of the product of the stitching integrity factor and the confidence level of the stitching results of the region under each rotation stitching method is used as the stitching integrity index of the region under each rotation stitching method.

[0029] Analyze the numerical characteristics of the splicing integrity index between adjacent seal sub-regions under different rotation splicing methods, and determine the rotation coefficient of each seal sub-region under each rotation splicing method.

[0030] Furthermore, the method for obtaining the complete splicing factor includes:

[0031] In the stitching results of each rotation stitching method, the stamp pixels in the area to be analyzed are taken as the points to be analyzed, and the stamp pixels in the comparison area are taken as the comparison points. Several concentric circles are drawn with the center of the stamp as the center, so that each stamp pixel in the area to be analyzed exists on the concentric circles.

[0032] On any concentric circle, the line connecting each point to be analyzed and each comparison point on that concentric circle is intercepted as a connected curve segment. Among all connected curve segments, the connected curve segment with only one point to be analyzed and one comparison point is taken as the target line segment. The sum of the longest and shortest target line segments is taken as the splicing and fitting deviation factor of the region to be analyzed on that concentric circle. The splicing and fitting deviation factor is multiplied by the sum of the number of points to be analyzed and comparison points on that concentric circle, and the normalized value of the product is taken as the splicing and fitting deviation index of the region to be analyzed on that concentric circle.

[0033] The sum of the splicing deviation indices of the region to be analyzed on all concentric circles is negatively correlated and normalized, and the resulting value is used as the splicing integrity factor of the region to be analyzed in the splicing result under each rotation splicing method.

[0034] Furthermore, the analysis of the numerical characteristics of the splicing integrity index between adjacent seal sub-regions under different rotation splicing methods, and the determination of the rotation coefficient of each seal sub-region under each rotation splicing method, includes:

[0035] Counterclockwise rotation splicing is denoted as reverse splicing, and clockwise rotation splicing is denoted as clockwise splicing. The rotation coefficient includes both the counterclockwise rotation coefficient and the clockwise rotation coefficient.

[0036] For any given seal sub-region, the product of the completeness index of the seal sub-region under reverse splicing and the completeness index of the next adjacent seal sub-region under forward splicing is used as the rotation coefficient of the seal sub-region under reverse splicing, denoted as the reverse rotation coefficient. The product of the completeness index of the seal sub-region under forward splicing and the completeness index of the next adjacent seal sub-region under reverse splicing is used as the rotation coefficient of the seal sub-region under forward splicing, denoted as the forward rotation coefficient.

[0037] Furthermore, the method for obtaining the effective indicators of the seal includes:

[0038] For any seal sub-region, calculate the absolute value of the difference between the clockwise rotation coefficient and the counterclockwise rotation coefficient of the seal sub-region, and the ratio of the sum of the clockwise rotation coefficient and the counterclockwise rotation coefficient. The normalized value of this ratio is used as the completion coefficient of the seal sub-region.

[0039] The percentage of the number of stamp pixels in this stamp sub-region relative to the total number and value of stamp pixels in all stamp sub-regions is used as the stamp validity factor for this stamp sub-region.

[0040] The normalized value of the product of the completion coefficient and the seal validity factor of the seal sub-region is used as the seal validity index of the seal sub-region.

[0041] Further, the effectiveness verification and completion determination of the notarization include:

[0042] When the notarization effectiveness index of a certain notarization sub-region is less than the preset notarization evaluation threshold, the notarization of the contract is considered invalid, and needs to be re-notarized for completion.

[0043] Further, the preset notarization evaluation threshold has a value range of [0.6, 1).

[0044] The present application has the following advantages:

[0045] Since the color of the notarization is usually different from the color of the paper and the text, the notarization image of the notarization area on the contract obtained should be an RGB image. By analyzing the difference characteristics of the channel values of the pixel points in the RGB image in each color channel, the notarization pixel points and the background pixel points (such as text, paper texture, etc.) can be efficiently distinguished. Since the notarization is a complete notarization pattern formed by continuous notarization of the edges of multiple pages of the contract, in the notarization image, the notarization sub-regions are divided based on the position distribution of the non-notarization pixel points, so as to realize the distinction and segmentation of the notarization areas on different contract pages or different paper pages. Since there may be displacement and rotation of the paper when the image is obtained, part of the notarization area may be blocked. Therefore, the notarization sub-regions can be rotated and spliced, so as to analyze the rotation coefficients under different rotation splicing modes, which are used to represent the possible blocking of the notarization. When analyzing the rotation coefficients, since the notarization pattern is usually circular, the notarization sub-regions can be aligned according to the notarization center (i.e. the center of the outer circular contour of the notarization pattern) and then rotated and spliced. On this basis, the position relationship of the notarization pixel points and the OCR recognition result are analyzed to determine the rotation coefficients under each rotation splicing mode. Then, the notarization effectiveness index is calculated based on the numerical characteristics of the rotation coefficients and the number characteristics of the notarization pixel points, realizing the cooperative verification of geometric deformation (rotation consistency) and physical coverage (pixel integrity), effectively solving the misjudgment problem of single OCR verification when the pages are misaligned, and significantly improving the recognition accuracy. Finally, the completeness of each notarization sub-region is quantitatively evaluated by the notarization effectiveness index, the missing or broken notarization parts can be automatically located, and the targeted completion process is triggered. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0047] Figure 1A method flow chart of a contract seal recognition and automatic completion method based on an OCR technology provided by an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a seal image provided by an embodiment of the present application;

[0049] Figure 3 A method flow chart of a rotation coefficient acquisition method provided by an embodiment of the present application;

[0050] Figure 4 A schematic diagram of a splicing result in two rotation splicing modes provided by an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a target line segment in a counterclockwise rotation splicing provided by an embodiment of the present application;

[0052] The figure legend: 1-concentric circle; 2-seal contour; 3-seal center; 4-seal pixel point; 5-target line segment. DETAILED DESCRIPTION

[0053] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined purposes, the following describes the specific implementation, structure, features and effects of a contract seal recognition and automatic completion method based on an OCR technology according to the present application in combination with the preferred embodiments and the drawings. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0055] The following specifically describes the specific scheme of a contract seal recognition and automatic completion method based on an OCR technology provided by the present application in combination with the drawings.

[0056] Please refer to Figure 1 which shows a method flow chart of a contract seal recognition and automatic completion method based on an OCR technology provided by an embodiment of the present application. The method includes the following steps:

[0057] Step S1: acquiring a seal image of a seal area on a contract, the seal image being an RGB image.

[0058] The purpose of the embodiment of the present application is to collect the seal image and identify its validity after the contract is sealed, so first, the seal image of the seal area on the contract is collected, and the specific process is as follows:

[0059] The user operates the seal, the intelligent seal system receives the trigger signal (such as a physical button) of the seal starting, and waits for the seal to be completed; after several seconds (such as two seconds), the seal action is completed and the seal is stationary, a collection preparation instruction is sent to the camera system; the camera system receives the collection preparation instruction, uses a high-definition industrial camera to shoot the seal area of the contract, judges whether the seal area is within the shooting range through color segmentation or a lightweight neural network, and makes the seal area as central as possible on the screen; the obtained image is recorded as a seal image, please refer to Figure 2 , which shows a schematic diagram of the seal image in an embodiment of the application. Since the seal color often has a large difference from the contract page background and the contract page text, the seal image should be an RGB image, which is convenient for further analyzing the seal integrity in the subsequent process.

[0060] Step S2: Based on the difference characteristics of the channel values of the pixel points in the seal image in the color channels, the seal pixel points are screened out from all the pixel points; in the seal image, the seal image is regionally divided based on the position distribution of the non-seal pixel points, and all seal sub-regions are obtained.

[0061] The seal color often has a large difference from the contract page background and the contract page text, for example, in the embodiment of the application, the seal is often red, the page background is white, and the page text is black, so the pixel points corresponding to the seal content can be extracted in the seal image based on the difference of the channel values of the pixel points in each color channel, and they are taken as the seal pixel points.

[0062] Preferably, in an embodiment of the application, the method for obtaining the seal pixel points comprises:

[0063] If the seal corresponding pixel point is red, the channel value in the red channel is large and the channel values in the other color channels are small. Therefore, in the seal image, for any pixel point, the difference between the channel value in the R channel and the channel value in each remaining color channel (G channel and B channel) of the pixel point is calculated as the channel difference value, the larger the channel difference value is, the greater the red prominence of the pixel point is, then the sum value of the two channel difference values of the pixel point after normalization is taken as the R channel prominence value of the pixel point, the larger the R channel prominence value is, the higher the prominence of the channel value in the R channel relative to the other color channels is, and the more likely the pixel point is the seal corresponding pixel point. Since the channel difference value can be positive or negative, the normalization method here can use .

[0064] Then, the product of the R channel highlight value of the pixel point and the normalized value of the R channel value of the pixel point is taken as a seal evaluation index of the pixel point. The seal evaluation index fuses the indexes of two dimensions of color purity and color relative highlight degree. The greater the value is, the more likely the pixel point is a pixel point corresponding to a red seal. The normalization is a technical means familiar to those skilled in the art. The selection of the normalization function can be linear normalization or standard normalization, and the specific normalization method is not limited here.

[0065] Finally, in the seal image, the pixel points with the seal evaluation index greater than a preset seal evaluation threshold are taken as seal pixel points.

[0066] It should be noted that the value range of the preset seal evaluation threshold is [0.6, 1), which is set to 0.65 in this embodiment of the application. The specific value can be adjusted according to the implementation scene, and is not limited here.

[0067] Since the riding seal is pasted on multiple pages of the contract, there is a part of the seal content on each page, and the multiple pages are combined to form a complete riding seal. If the pages are displaced or deflected during image acquisition, the riding seal on the acquired pages may not be standard and aligned, and the fragments still need to be analyzed. Therefore, in the seal image, the seal image can be regionally divided based on the position distribution of the non-seal pixel points, that is, the multiple stacked and staggered contract pages are segmented to obtain all seal sub-regions, so as to facilitate the analysis of whether there is enough content on each page in the subsequent process for judging the effectiveness of the riding seal.

[0068] Preferably, in an embodiment of the application, the method for obtaining the seal sub-region comprises:

[0069] In the seal image, the distribution of the non-seal pixel points contains the straight line features of the page edges, that is, the boundaries of the multiple stacked and staggered contract pages. Therefore, the Hough straight line transformation (a known technology, the process is not described in detail) is used to analyze all the non-seal pixel points in the seal image, so as to extract all the straight lines as the to-be-tested straight lines.

[0070] Since the contract page edges are usually long line segments (such as the left and right boundaries of A4 paper) that run through the entire seal image, and the texts usually only produce short line segments, the edge detection is performed on the non-seal pixel points in the seal image to obtain the edge lines (which can use the Canny operator, which is a known technology, and the process is not described in detail), and the edge lines coinciding with the to-be-tested straight lines are taken as target edge lines. Among all the target edge lines, the target edge lines with a length greater than or equal to the length of the short side of the seal image are taken as the contract page edge lines.

[0071] At this point, all contract page edge lines in the seal image (the boundary lines of A4 paper in the embodiment of the present application) can be extracted, and finally in the seal image, a ray is made horizontally to the right through the midpoint of the left boundary line of the seal image, all contract page edge lines are sorted according to the order of intersection of the ray and the contract page edge lines, so that the region surrounded by each adjacent two contract page edge lines and the seal image boundary is taken as a seal sub-region.

[0072] It should be noted that in the embodiment of the present application, if it is necessary to perform gray scale processing on the seal image, average gray scale processing, maximum value gray scale processing and the like can be adopted, which are well-known technologies and the process will not be described herein.

[0073] Step S3: analyzing the shape features of each seal sub-region to determine the seal center corresponding to each seal sub-region; aligning adjacent seal sub-regions according to the seal center and then rotating and splicing in different directions, so as to analyze the positional relationship between seal pixel points and combine with the OCR recognition result after splicing, for determining the rotation coefficient of each seal sub-region under different rotation splicing modes.

[0074] When collecting the seal image, since the pages are not necessarily completely regular, some pages may be displaced or deflected, resulting in that the seal pattern is covered and shielded by the pages, the pattern is incomplete when collecting the image for recognition, and part of the page content is insufficient, which is misjudged as invalid seal content, so that the recognition result deviates from the actual situation. Therefore, it is necessary to analyze the shielding condition of the page according to the local missing state of the seal in each seal sub-region, so that the subsequent recognition result is more accurate.

[0075] Since the seal contour is circular, the seal centers corresponding to the local seal contour in different seal sub-regions should be consistent, so the shape features of each seal sub-region are analyzed first, so as to determine the seal center corresponding to each seal sub-region.

[0076] Preferably, the method for obtaining the seal center in an embodiment of the present application comprises:

[0077] In each seal sub-region, all seal pixel points are subjected to edge analysis and edge lines are obtained by using a Canny operator, and then the region corresponding to each edge line is taken as a connected domain based on connected domain analysis.

[0078] When there is only one connected domain in a certain seal sub-region, it is considered that only the local content of the outer contour of the seal (for example, the uppermost contract page in the stacked and staggered multiple contract pages) should be included in the connected domain, so in the connected domain of the seal sub-region, three seal pixel points are selected to obtain two line segments (the line segment is also the chord in the process of finding the center of a circular arc), and the intersection point of the perpendicular bisector of the two line segments is taken as the seal center of the seal sub-region.

[0079] When there are at least two connected domains in a certain seal sub-region, the center of mass of each connected domain is obtained, and the two connected domains corresponding to the two farthest centers of mass tend to correspond to the outer circular contour of the seal, so one of the two connected domains with the farthest Euclidean distance between the centers of mass is selected as the target region. In the same way, in the target region, three seal pixels are selected to obtain two line segments, and the intersection point of the perpendicular bisectors of the two line segments is taken as the seal center of the seal sub-region.

[0080] At this point, the corresponding seal center of each seal sub-region can be determined through the above process.

[0081] After determining the corresponding seal center of each seal sub-region, the adjacent seal sub-regions can be aligned according to the seal center. Since there may be a rotational displacement of the page, the result obtained by aligning only the seal center may be incomplete and still cannot determine the specific seal missing state, so after alignment, the seal sub-regions are rotated and spliced in different directions, and then in the splicing result, the positional relationship between the seal pixels is analyzed and combined with the OCR recognition result after splicing to determine the rotation coefficient under each rotation splicing mode, which is used to represent the possible occlusion of the seal.

[0082] Preferably, the method for obtaining the rotation coefficient in an embodiment of the present application comprises:

[0083] Please refer to Figure 3 which shows a method flowchart of the method for obtaining the rotation coefficient in an embodiment of the present application, and the method comprises the following steps:

[0084] Step S301: Sort all seal sub-regions according to the page sorting direction of the contract to obtain a sequence of seal sub-regions.

[0085] The contract page is usually arranged in a fixed order, so the page sorting direction of the contract can be used to sort all seal sub-regions to obtain a sequence of seal sub-regions. For example, in the horizontal right direction, the stacking order of the contract pages is page 1 at the top, and then the seal sub-regions are arranged in the horizontal right direction.

[0086] Step S302: Select one seal sub-region as an analysis region, and select the adjacent previous seal sub-region of the analysis region as a comparison region of the analysis region.

[0087] All seal sub-regions are spliced with the adjacent previous seal sub-region, in order to facilitate subsequent analysis and description, one seal sub-region is selected as an analysis region, and the adjacent previous seal sub-region of the analysis region in the sequence of seal sub-regions is selected as a comparison region of the analysis region.

[0088] Step S303: The rotation splicing mode includes counterclockwise rotation splicing and clockwise rotation splicing, and in each rotation splicing mode, the to-be-analyzed region is rotated to obtain a splicing result.

[0089] In each rotation splicing mode, the to-be-analyzed region is aligned with the seal center of the contrast region, so that the to-be-analyzed region is rotated (clockwise rotation splicing is clockwise rotation, and counterclockwise rotation splicing is counterclockwise rotation) with the seal center as a fixed point, and when the seal pixel points in the to-be-analyzed region and the seal pixel points in the contrast region are connected during rotation, the rotation is stopped, and a splicing result in each rotation splicing mode is obtained. Please refer to Figure 4 which shows the splicing results in two rotation splicing modes in an embodiment of the present application.

[0090] Step S304: In the splicing result in each rotation splicing mode, the positional relationship between the seal pixel points in the to-be-analyzed region and the contrast region is analyzed, and a splicing integrity factor of the to-be-analyzed region in the splicing result in each rotation splicing mode is determined.

[0091] In the splicing result in each rotation splicing mode, the matching condition between patterns can be analyzed to quantify the splicing integrity.

[0092] In the splicing result in each rotation splicing mode, the seal pixel points in the to-be-analyzed region are taken as to-be-analyzed points, the seal pixel points in the contrast region are taken as contrast points, and a plurality of concentric circles are drawn with the seal center as the center, so that each seal pixel point in the to-be-analyzed region is present on a concentric circle.

[0093] When the distance between the seal pixel points in the to-be-analyzed region and the seal pixel points in the contrast region in the splicing result is too far, it can be considered that there is a missing, and the smaller the splicing integrity is, so on any concentric circle, the connecting line between each to-be-analyzed point and each contrast point on the concentric circle is taken as a connected curve segment, and in all connected curve segments, the connected curve segment with only one to-be-analyzed point and one contrast point is taken as a target line segment (only two end points on the target line segment are seal pixel points), please refer to Figure 5 which shows a schematic diagram of a target line segment in counterclockwise rotation splicing in an embodiment of the present application.

[0094] When the length of the target line segment is shorter, it can be considered that the pattern fitting degree between the to-be-analyzed region and the contrast region in the splicing result is better, so the sum of the longest target line segment and the shortest target line segment is taken as the splicing fitting deviation factor of the to-be-analyzed region on the concentric circle; at the same time, if the number of the seal pixel points on the concentric circle is larger, it indicates that the seal pixel points are distributed more densely, and under normal circumstances, the spliced state should be more fitted, but if the seal pixel points are densely distributed, the splicing fitting deviation factor is larger, which can be considered as the fitting degree being worse, so the splicing fitting deviation factor is multiplied by the sum of the number of the to-be-analyzed points and the contrast points on the concentric circle, and the normalized value of the obtained product is taken as the splicing fitting deviation index of the to-be-analyzed region on the concentric circle. The normalization is a technical means familiar to those skilled in the art, and the selection of the normalization function can be linear normalization or standard normalization, and the specific normalization method is not limited here.

[0095] At this point, the splicing fitting deviation index of the to-be-analyzed region on each concentric circle can be obtained, and the larger the value is, the worse the fitting degree of the seal pattern between the to-be-analyzed region and the contrast region on each concentric circle is, and the lower the splicing integrity is, so the sum of the splicing fitting deviation indexes of the to-be-analyzed region on all concentric circles is negatively correlated and normalized to correct the logical relationship, and the splicing integrity factor of the to-be-analyzed region in the splicing result under each rotation splicing mode is obtained, and the larger the splicing integrity factor is, the higher the rationality of the seal content of the splicing result of the to-be-analyzed region and the contrast region under the rotation splicing mode is. The negative correlation mapping and normalization processing here can adopt the formula wherein, represents the exponential function with the natural constant e as the base, and x represents the independent variable.

[0096] Step S305: In each rotation splicing mode, the character recognition processing is performed on the splicing result corresponding to the to-be-analyzed region, and the confidence is obtained.

[0097] The OCR confidence can provide the semantic level splicing quality evaluation, and supplement the limitations in the foregoing geometric level, so the OCR character recognition is performed on the splicing result corresponding to the to-be-analyzed region in each rotation splicing mode, and the confidence of the character recognition result corresponding to the to-be-analyzed region is output, and the larger the confidence is, the higher the semantic integrity of the splicing result is, and the higher the rationality of the seal content of the splicing result is.

[0098] Step S306: The confidence corresponding to the splicing result of the to-be-analyzed region in each rotation splicing mode and the splicing integrity factor are combined, and the splicing integrity index of the to-be-analyzed region in each rotation splicing mode is obtained.

[0099] The product of the stitching completeness factor and the confidence of the stitching result of the region to be analyzed under each rotation stitching mode after normalization is taken as the stitching completeness index of the region to be analyzed under each rotation stitching mode. Based on the analysis in steps S304 and S305, the larger the stitching completeness index, the higher the rationality of the combined seal content of the region to be analyzed and the contrast region under the rotation stitching mode, the higher the consistency with the original combined state of the actual seal, and the less the shielding or missing. The normalization is a technique familiar to those skilled in the art, and the normalization function can be linear normalization or standard normalization. The specific normalization method is not limited here.

[0100] Step S307: Analyzing the numerical characteristics of the stitching completeness indexes of the adjacent seal sub-regions under different rotation stitching modes to determine the rotation coefficient of each seal sub-region under each rotation stitching mode.

[0101] According to the above steps, the stitching completeness indexes of all seal sub-regions under different rotation stitching modes can be obtained. When the contract page is rotated or translated to cause shielding, taking the rth seal sub-region as an example, if the rth seal sub-region corresponds to a counterclockwise rotation of the contract page relative to the seal image acquisition time, the upper left corner of the rth seal sub-region will be shielded by the r-1th seal sub-region, and the lower left corner of the r+1th seal sub-region will be shielded and missing by the rth seal sub-region, that is, the stitching completeness degree of the rth seal sub-region under counterclockwise rotation stitching is lower than that of the r+1th seal sub-region under clockwise rotation stitching.

[0102] Therefore, counterclockwise rotation stitching is recorded as reverse stitching, and clockwise rotation stitching is recorded as forward stitching.

[0103] Then, for any seal sub-region, the product of the stitching completeness index of the seal sub-region under reverse stitching and the stitching completeness index of the adjacent next seal sub-region under forward stitching is taken as the rotation coefficient of the seal sub-region under reverse stitching, recorded as reverse rotation coefficient; the product of the stitching completeness index of the seal sub-region under forward stitching and the stitching completeness index of the adjacent next seal sub-region under reverse stitching is taken as the rotation coefficient of the seal sub-region under forward stitching, recorded as forward rotation coefficient. The rotation coefficient comprehensively considers the change rule of the stitching completeness degree of the stitching result of the adjacent seal sub-regions under different rotation stitching modes, and therefore helps to more accurately measure the seal effectiveness of the seal sub-region in the subsequent process.

[0104] Step S4: based on the numerical characteristics of the rotation coefficients of each seal sub-region under rotation splicing in different directions, and in combination with the number characteristics of the seal pixel points of each seal sub-region, the seal effective index of each seal sub-region is calculated for verifying the effectiveness of the seal and making a judgment.

[0105] When the seal pattern is missing due to rotation of the contract page when the seal image is collected, there will be a large difference between the forward rotation coefficient and the reverse rotation coefficient obtained in step S3. When the seal pattern is missing due to the existence of an obstacle on the contract page when the seal image is collected, the forward rotation coefficient and the reverse rotation coefficient will be smaller. In addition, when identifying and verifying the effectiveness of the seal, it is often necessary to analyze whether the seal content on each page of the contract is sufficient. This feature can be represented by the number of seal pixel points in each seal sub-region. Therefore, in this embodiment of the present application, based on the numerical characteristics of the rotation coefficients of each seal sub-region under rotation splicing in different directions, and in combination with the number characteristics of the seal pixel points of each seal sub-region, the seal effective index of each seal sub-region is calculated to measure the effectiveness of the seal content in the seal sub-region.

[0106] Preferably, in an embodiment of the present application, the method for obtaining the seal effective index comprises:

[0107] Based on the foregoing logical analysis, for any one seal sub-region, the absolute value of the difference between the forward rotation coefficient and the reverse rotation coefficient of the seal sub-region, and the ratio of the sum of the forward rotation coefficient and the reverse rotation coefficient, are calculated. The value of the ratio after normalization is taken as the completion coefficient of the seal sub-region. The larger the completion coefficient, the greater the possibility of obstruction when the seal image is obtained. The normalization is a technique well known to those skilled in the art, and the selection of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited herein.

[0108] Then, for any one seal sub-region, the proportion of the number of seal pixel points in the seal sub-region in the sum of the number of seal pixel points of all seal sub-regions is taken as the seal effective factor of the seal sub-region. The larger the seal effective factor, the greater the proportion of the number of seal pixel points in the seal sub-region, and thus the higher the integrity.

[0109] Finally, the product of the completion coefficient of the seal sub-region and the normalized value of the seal effective factor is taken as the seal effective index of the seal sub-region. The larger the seal effective index, the more likely it is that the content is too small due to certain occlusion when collecting the seal image, and the higher the completeness of the seal in the seal sub-region itself should be. The normalization is a technique well known to those skilled in the art, and the selection of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0110] After obtaining the seal effective index of each seal sub-region, the effectiveness of the seal of the contract can be verified and the completion can be determined according to the index.

[0111] Preferably, in an embodiment of the present application, the effectiveness verification and completion determination of the seal of the contract comprises:

[0112] When the seal effective index of a certain seal sub-region is less than a preset seal effective threshold, it is considered that the seal of the contract is invalid and needs to be resealed for completion.

[0113] It should be noted that the preset seal effective threshold is 0.75, and the specific value can be adjusted according to the implementation scene, which is not limited here.

[0114] In summary, since the seal color is usually different from the paper and text color, the obtained seal image of the contract stamping area should be an RGB image. By analyzing the difference characteristics of the channel values of the pixel points in each color channel in the RGB image, the seal pixel points and the background pixel points (such as text, paper texture, etc.) can be efficiently distinguished. Because the riding seal is a complete seal pattern formed by the continuous stamping of the edges of multiple pages of the contract, in the seal image, the seal sub-regions are divided based on the position distribution of the non-seal pixel points, so as to realize the distinction and segmentation of the seal regions on different contract pages or different paper pages. Since there may be displacement and rotation of the paper when the image is obtained, part of the seal region may be blocked. Therefore, the seal sub-regions can be rotated and spliced, so as to analyze the rotation coefficients under different rotation splicing modes, which are used to represent the possible blocking conditions of the seal. In the analysis of the rotation coefficients, because the seal pattern is usually circular, the seal sub-regions can be aligned according to the center of the seal (i.e. the center of the outer circular contour of the seal pattern) and then rotated and spliced. On this basis, the position relationship of the seal pixel points and the OCR recognition result are analyzed, which are used to determine the rotation coefficients under each rotation splicing mode. Then, the seal effective indicators are calculated based on the numerical characteristics of the rotation coefficients and the number characteristics of the seal pixel points, realizing the cooperative verification of geometric deformation (rotation consistency) and physical coverage (pixel integrity), effectively solving the misjudgment problem of single OCR verification when the pages are misaligned, and significantly improving the recognition accuracy. Finally, the integrity of each seal sub-region is quantitatively evaluated by the seal effective indicators, the missing or broken parts of the seal can be automatically located, and the targeted completion process is triggered.

[0115] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0116] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment mainly describes the differences from other embodiments.

Claims

1. A method for recognizing and automatically completing contract seals across the seams based on OCR technology, characterized in that, The method includes: Obtain an image of the seal in the stamping area of ​​the contract; the seal image is an RGB image. Based on the difference in channel values ​​of pixels in the color channel of the seal image, seal pixels are selected from all pixels; in the seal image, the seal image is divided into regions based on the positional distribution of non-seal pixels to obtain all seal sub-regions. The shape features of each stamp sub-region are analyzed to determine the stamp center corresponding to each stamp sub-region; adjacent stamp sub-regions are aligned according to the stamp center and then rotated and spliced ​​in different directions to analyze the positional relationship between stamp pixels, and combined with the spliced ​​OCR recognition results to determine the rotation coefficient of each stamp sub-region under different rotation splicing methods. Based on the numerical characteristics of the rotation coefficient of each seal sub-region under different directions of rotation splicing, and combined with the number of seal pixels in each seal sub-region, the seal validity index of each seal sub-region is calculated for the validity verification and completion judgment of the seal across the seam. The method for obtaining the rotation coefficient includes: Sort all the seal sub-regions according to the page sorting direction of the contract to obtain the seal sub-region sequence; Choose any one of the seal sub-regions as the region to be analyzed, and use the previous seal sub-region adjacent to the region to be analyzed as the comparison region of the region to be analyzed. Rotation splicing methods include counterclockwise rotation splicing and clockwise rotation splicing; In each rotation stitching method, the center of the stamp in the area to be analyzed is aligned with the center of the stamp in the comparison sub-area, and then the area to be analyzed is rotated. The rotation stops when the stamp pixels in the area to be analyzed are connected to the stamp pixels in the comparison area, and the stitching result is obtained. In the stitching results of each rotation stitching method, the positional relationship between the stamp pixels in the area to be analyzed and the comparison area is analyzed to determine the stitching integrity factor of the area to be analyzed in the stitching results of each rotation stitching method. OCR character recognition is performed on the splicing results under each rotation splicing method, thereby outputting the confidence level of the character recognition results corresponding to the region to be analyzed; The normalized value of the product of the stitching integrity factor and the confidence level of the stitching results of the region under each rotation stitching method is used as the stitching integrity index of the region under each rotation stitching method. Analyze the numerical characteristics of the splicing integrity index between adjacent seal sub-regions under different rotation splicing methods, and determine the rotation coefficient of each seal sub-region under each rotation splicing method.

2. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The method for obtaining the stamp pixels includes: In the stamp image, for any pixel, the difference between the channel value of the pixel in the R channel and the channel value of each of the remaining color channels is taken as the channel difference. The sum of the two channel differences corresponding to the pixel is normalized and taken as the R channel highlight value of the pixel. The normalized value of the product of the R channel protrusion value and the R channel value of the pixel is used as the stamp evaluation index of the pixel. In the seal image, pixels whose seal evaluation index is greater than the preset seal evaluation threshold are designated as seal pixels.

3. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The method for obtaining the seal sub-region includes: The Hough line transform is used to analyze all non-stamp pixels in the stamp image, thereby extracting all straight lines as the lines to be measured. Edge detection is performed on non-stamp pixels in the stamp image to obtain edge lines. Edge lines that coincide with the line to be tested are taken as target edge lines, and target edge lines whose length is greater than or equal to the short side length of the stamp image are taken as contract page edge lines. In the seal image, the area enclosed by each pair of adjacent contract page edge lines and the seal image boundary is considered as a seal sub-region.

4. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The method for obtaining the center of the seal includes: In each stamp sub-region, the Canny operator is used to perform edge analysis on all stamp pixels and obtain edge lines. Based on connected component analysis, the region corresponding to each edge line is treated as a connected component. When there is only one connected component in a certain stamp sub-region, three stamp pixels are randomly selected in the connected component of the stamp sub-region to obtain two line segments, and the intersection of the perpendicular bisectors of these two line segments is taken as the stamp center of the stamp sub-region. When there are at least two connected components in a certain stamp sub-region, obtain the centroid of each connected component. Select one of the two connected components with the greatest Euclidean distance between their centroids as the target region. In the target region, select three stamp pixels to obtain two line segments. Take the intersection of the perpendicular bisectors of these two line segments as the stamp center of the stamp sub-region.

5. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The method for obtaining the complete splicing factor includes: In the stitching results of each rotation stitching method, the stamp pixels in the area to be analyzed are taken as the points to be analyzed, and the stamp pixels in the comparison area are taken as the comparison points. Several concentric circles are drawn with the center of the stamp as the center, so that each stamp pixel in the area to be analyzed exists on the concentric circles. On any concentric circle, the line connecting each point to be analyzed and each comparison point on that concentric circle is intercepted as a connected curve segment. Among all connected curve segments, the connected curve segment with only one point to be analyzed and one comparison point is taken as the target line segment. The sum of the longest and shortest target line segments is taken as the splicing and fitting deviation factor of the region to be analyzed on that concentric circle. The splicing and fitting deviation factor is multiplied by the sum of the number of points to be analyzed and comparison points on that concentric circle, and the normalized value of the product is taken as the splicing and fitting deviation index of the region to be analyzed on that concentric circle. The sum of the splicing deviation indices of the region to be analyzed on all concentric circles is negatively correlated and normalized, and the resulting value is used as the splicing integrity factor of the region to be analyzed in the splicing result under each rotation splicing method.

6. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The analysis examines the numerical characteristics of the splicing integrity index between adjacent seal sub-regions under different rotation splicing methods, determining the rotation coefficient of each seal sub-region under each rotation splicing method, including: Counterclockwise rotation splicing is denoted as reverse splicing, and clockwise rotation splicing is denoted as clockwise splicing. The rotation coefficient includes both the counterclockwise rotation coefficient and the clockwise rotation coefficient. For any given seal sub-region, the product of the completeness index of the seal sub-region under reverse splicing and the completeness index of the next adjacent seal sub-region under forward splicing is used as the rotation coefficient of the seal sub-region under reverse splicing, denoted as the reverse rotation coefficient. The product of the completeness index of the seal sub-region under forward splicing and the completeness index of the next adjacent seal sub-region under reverse splicing is used as the rotation coefficient of the seal sub-region under forward splicing, denoted as the forward rotation coefficient.

7. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 6, characterized in that, The method for obtaining the effective indicators of the seal includes: For any seal sub-region, calculate the absolute value of the difference between the clockwise rotation coefficient and the counterclockwise rotation coefficient of the seal sub-region, and the ratio of the sum of the clockwise rotation coefficient and the counterclockwise rotation coefficient. The normalized value of this ratio is used as the completion coefficient of the seal sub-region. The percentage of the number of stamp pixels in this stamp sub-region relative to the total number and value of stamp pixels in all stamp sub-regions is used as the stamp validity factor for this stamp sub-region. The normalized value of the product of the completion coefficient and the seal validity factor of the seal sub-region is used as the seal validity index of the seal sub-region.

8. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 1, characterized in that, The verification and completion judgment of the seal across the seam include: If the seal validity index of a certain seal sub-area is less than the preset seal validity threshold, the seal across the seam of the contract is considered invalid and needs to be re-stamped to make up for it.

9. The method for recognizing and automatically completing contract seals based on OCR technology according to claim 2, characterized in that, The preset stamp evaluation threshold ranges from [0.6, 1].

Images