Method and device for intelligent file investigation and identification of text detector based on multispectral imaging

Abstract

The invention belongs to the technical field of computer vision, and discloses a multispectral imaging-based intelligent file investigation and identification method and device for a text detector, and the method comprises the steps: firstly carrying out the distortion correction of a file, and extracting the corrected file from an image; segmenting the text line into continuous complete characters by using a text line character segmentation method improved based on a projection method; then detecting the geometric center of each character of the text line, and calculating a center curve of the text line; and finally, the row height and the row spacing are calculated, and the abnormal row height and the abnormal row spacing are detected. In the whole process, not only is global distortion correction carried out on the document, but also direction detection is carried out on each character of each text line, so that detection errors caused by file distortion are effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an intelligent document inspection and identification method and device of a document inspection instrument based on multi-spectral imaging. Background Art

[0002] Document inspection is a mature physical evidence technology, so document inspection instruments have become a commonly used device for judicial identification. The physical evidence products inspected by them are the result of interaction with other elements in the case. A comprehensive grasp of relevant evidence can play an auxiliary or even key role in the case, so the role of document inspection instruments in practice is becoming more and more significant. The super document inspection workstation has the advantages of not destroying the inspection material, high sensitivity, intuitiveness, and speed. It collects multispectral images of documents through multispectral imaging technology, and then manually identifies the documents. However, the current intelligence level of document inspection equipment is low. In most cases, document inspection equipment is only used as an auxiliary observation tool, and the document identification task is mainly completed by the user's document inspection experience, which requires a lot of cost to train professionals with document inspection experience. Therefore, using the upper computer software algorithm to empower the document inspection instrument, making the document inspection instrument with complete hardware functions intelligent, can greatly reduce labor costs in specific tasks and improve detection efficiency to a certain extent.

[0003] Through the above analysis, the problems and defects of the prior art are as follows:

[0004] At present, the intelligence level of document inspection equipment is relatively low. In most cases, document inspection equipment is only used as an auxiliary observation tool, and the document identification task is mainly completed relying on the user's document inspection experience. This requires a lot of cost to train professionals with document inspection experience. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides an intelligent document inspection and identification method and device for a document inspection instrument based on multi-spectral imaging.

[0006] The present invention is implemented as follows: an intelligent document inspection and identification method based on multispectral imaging of a document inspection instrument comprises:

[0007] Step 1, extract the document from the image and correct it;

[0008] Step 2, using a text line character segmentation method based on clustering to segment the text line into continuous complete characters;

[0009] Step 3, detecting the geometric center of each character in the text line and calculating the text line center curve;

[0010] Step 4: Calculate row height and row spacing, and detect abnormal row height and row spacing.

[0011] Further, the document is extracted from the image and corrected:

[0012] To solve the problem of document distortion correction, the TPS thin plate spline interpolation method is used. The sampled edge detection points are regarded as control points, and the projection points are regarded as target points after the control points are moved, so as to perform global correction on the document distortion.

[0013] Furthermore, the text line is segmented into continuous complete characters using the clustering-based text line character segmentation method:

[0014] To address the problem of over-segmentation of left-right structured Chinese characters, a clustering algorithm is used to divide characters into complete and incomplete character classes, and adjacent incomplete character classes are continuously spliced ​​together to form complete characters.

[0015] Further, the text line center curve is calculated:

[0016] To address the problem of increased line height calculation errors caused by text line distortion and inclination, the center curve of the text line is recalculated using the center of the minimum circumscribed circle of each character, and the average line height in the normal direction of the curve is used as the line height.

[0017] Furthermore, the abnormal line height and line spacing are detected:

[0018] In order to prevent the document reference line height or line spacing from being affected by the extreme line height, a reference value is selected by mutual voting between line heights or line spacing, and this is used as a reference for anomaly detection to reduce the impact of extreme values ​​on anomaly detection.

[0019] Furthermore, the intelligent document inspection and identification method based on multispectral imaging also includes:

[0020] Seal comparison methods include:

[0021] The seal extraction method based on color channel suppression of background interference aims at the problem of background interference in the seal extraction process. The difference between the seal foreground and background in each RGB channel is used to effectively suppress background interference such as printed text and handwritten text, and extract the foreground seal.

[0022] Seal shape classification and parameter calculation method, for the problem of seal size calculation, the seal shape is obtained by using seal contour extraction, polygon fitting and minimum circumscribed circle fitting algorithm, and then the seal size is calculated according to the seal shape;

[0023] The seal similarity calculation method based on feature point matching aims at the similarity calculation problem of two seals with different scales and directions. The SIFT algorithm with scale and direction invariance is used to extract feature points. The matching point transformation matrix is ​​obtained by combining the RANSAC algorithm. The two seals are transformed into the same scale and direction. Finally, the difference points of the two seals are calculated and the similarity is calculated using the formula.

[0024] Another object of the present invention is to provide a yellow spot tracking method, comprising:

[0025] Method for removing the background of a color laser printed document image and extracting a yellow dot array;

[0026] Spatial feature extraction method of points in yellow dot array;

[0027] Pattern fusion method for points with similar spatial features;

[0028] A fusion method for different dot patterns based on their relative positions in space.

[0029] Another object of the present invention is to provide an intelligent document inspection and identification device based on multispectral imaging, comprising:

[0030] A correction module, used to extract the document from the image and perform correction;

[0031] A segmentation module, for segmenting a text line into continuous complete characters using a text line character segmentation method based on clustering;

[0032] A calculation module, used to detect the geometric center of each character in the text line and calculate the center curve of the text line;

[0033] The detection module is used to calculate the row height and row spacing, and detect abnormal row height and row spacing.

[0034] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the intelligent document inspection and identification method based on multi-spectral imaging.

[0035] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the intelligent document inspection and identification method based on multi-spectral imaging.

[0036] Another object of the present invention is to provide an information data processing terminal, which is used to implement the intelligent document inspection and identification device of the document inspection instrument based on multi-spectral imaging.

[0037] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0038] First, the present invention first corrects the distortion of the document and extracts it from the image; then uses the improved text line character segmentation method based on the projection method to segment the text line into continuous complete characters; then detects the geometric center of each character in the text line and calculates the text line center curve; finally calculates the line height and line spacing, and detects abnormal line height and line spacing. The entire process not only performs global distortion correction on the document, but also performs direction detection on each character in each text line, effectively reducing the detection error caused by file distortion.

[0039] Second, the technical solution of the present invention solves several key problems in the prior art in industrial applications and has achieved significant technical progress, which is specifically reflected in the following aspects:

[0040] 1) Improve the accuracy of document inspection and identification

[0041] In the prior art, the inspection and identification of documents mainly rely on manual or simple image processing methods, which are easily affected by image quality, document tilt and distortion, resulting in low accuracy of the inspection results. The present invention uses multispectral imaging technology to accurately extract and correct documents, effectively solving the problems of image distortion and noise interference, and greatly improving the accuracy of document inspection.

[0042] 2) Improve character segmentation efficiency and accuracy

[0043] The character segmentation algorithms in the prior art often cannot accurately identify continuous characters when processing complex text lines, especially in handwritten documents where character aliasing is prone to occur. The present invention adopts a text line character segmentation method based on clustering, which can effectively identify and segment continuous complete characters, greatly improving the efficiency and accuracy of character segmentation, and is particularly suitable for text processing in complex scenarios.

[0044] 3) Provide intelligent geometric feature analysis

[0045] The existing technology lacks intelligent analysis tools for character geometric features, especially the analysis of text line arrangement, line spacing and character morphology is relatively limited. The present invention can automatically identify abnormal typesetting or forged information in documents by calculating the geometric center and center curve of text lines and combining the detection of line height and line spacing, providing a more intelligent and automated analysis method for document inspection and identification.

[0046] 4) Significantly improve the efficiency of document authentication

[0047] Traditional document authentication methods are time-consuming and highly dependent on manual analysis. This invention significantly shortens the working time of document authentication through a full-process automated processing method, including document extraction, character segmentation, geometric analysis and anomaly detection, while ensuring higher accuracy and consistency, providing an efficient and reliable solution for document authentication and review in industrial applications.

[0048] Through the application of the above technical solutions, the present invention has achieved remarkable technical progress in the field of document inspection and identification, which not only solves the bottleneck of the existing technology, but also provides strong technical support for industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method for intelligent document inspection and identification based on multi-spectral imaging provided by an embodiment of the present invention;

[0050] Figure 2 is a flowchart of A4 document correction and segmentation provided by an embodiment of the present invention;

[0051] Figure 3 is a flow chart of a text line character segmentation method based on clustering provided by an embodiment of the present invention;

[0052] Figure 4 is a flow chart of text line center curve detection provided by an embodiment of the present invention;

[0053] Figure 5 is a flowchart of text line height and line spacing calculation and anomaly detection provided by an embodiment of the present invention;

[0054] Figure 6 is a flow chart of seal comparison provided by an embodiment of the present invention;

[0055] Figure 7 This is a flow chart of seal extraction provided by an embodiment of the present invention

[0056] Figure 8 is a flow chart of seal shape detection provided by an embodiment of the present invention;

[0057] Fig. 9 The present invention provides a seal inner and outer diameter calculation process;

[0058] Fig.10 is a flow chart of seal difference point detection provided by an embodiment of the present invention;

[0059] Fig.11 It is a comparison diagram of the detection results of the HED algorithm and the Canny algorithm provided in an embodiment of the present invention;

[0060] Fig.12 It is a schematic diagram of A4 document correction and segmentation results based on thin plate spline interpolation provided by an embodiment of the present invention;

[0061] Fig.13 is a schematic diagram of segmenting text lines using a horizontal projection method provided by an embodiment of the present invention;

[0062] Fig.14 Schematic diagram of character segmentation using the vertical projection method provided by an embodiment of the present invention;

[0063] Fig.15 is an RGB channel histogram provided by an embodiment of the present invention;

[0064] Fig.16 is a flowchart of obtaining a dot matrix provided by an embodiment of the present invention;

[0065] Fig.17 The present invention provides a block diagram of the structure of an intelligent document inspection and identification device based on multi-spectral imaging provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] like Figure 1 As shown, an intelligent document inspection and identification method based on multispectral imaging provided by an embodiment of the present invention comprises the following steps:

[0068] S101, extracting the document from the image and correcting it;

[0069] S102, segmenting the text line into continuous complete characters using a text line character segmentation method based on clustering;

[0070] S103, detecting the geometric center of each character in the text line and calculating the text line center curve;

[0071] S104, calculating the row height and the row spacing, and detecting abnormal row height and row spacing.

[0072] The specific working principle of the intelligent document inspection and identification method based on multispectral imaging provided by the embodiment of the present invention is as follows:

[0073] 1. Document extraction and correction

[0074] In step S101, the image data of the document is first obtained through multispectral imaging technology. The document is extracted from the background using an image processing algorithm, and the document's geometric distortions such as tilt and twist are corrected to ensure the accuracy of subsequent analysis. This step ensures that the text content in the image maintains the correct arrangement and format by adjusting the geometric form of the document image, avoiding errors in subsequent analysis due to image distortion.

[0075] 2. Clustering-based character segmentation

[0076] In step S102, a clustering algorithm is used to perform character segmentation on the text lines in the document. The text lines are divided into continuous complete characters by identifying the blank areas and the intervals between characters in each line. This segmentation method can effectively separate individual characters by analyzing the pixel density and distribution of the characters, ensuring that there is no overlap or confusion between characters, which is convenient for subsequent character analysis.

[0077] 3. Character geometric center detection and text line center curve calculation

[0078] In step S103, the geometric features of the characters are used to detect the geometric center point of each character and calculate the center curve of each line of text. By calculating the geometric center of the characters, the arrangement position of each character can be accurately located, and the curve shape of each line of text can be further analyzed to detect the anomalies in handwriting or printing. This process can effectively identify whether the character arrangement in the document is regular, and is particularly suitable for the analysis of handwritten documents or complex texts.

[0079] 4. Calculation of row height and row spacing and anomaly detection

[0080] In step S104, the system calculates the height of each line of text (line height) and the distance between two lines (line spacing). Based on the preset standard values, the system can automatically detect abnormal line height and line spacing in the document, which is important for identifying handwriting anomalies, typesetting errors or signs of forgery in the document. By analyzing these parameters, potential problems in the document can be quickly discovered, assisting document inspectors in further identification.

[0081] The embodiment of the present invention provides extracting a document from an image and correcting it:

[0082] To solve the problem of document distortion correction, the TPS thin plate spline interpolation method is used. The sampled edge detection points are regarded as control points, and the projection points are regarded as target points after the control points are moved, so as to perform global correction on the document distortion.

[0083] The text line character segmentation method based on clustering provided in the embodiment of the present invention is used to segment the text line into continuous complete characters:

[0084] To address the problem of over-segmentation of left-right structured Chinese characters, a clustering algorithm is used to divide characters into complete and incomplete character classes, and adjacent incomplete character classes are continuously spliced ​​together to form complete characters.

[0085] The embodiment of the present invention provides a text line center curve calculation method:

[0086] To address the problem of increased line height calculation errors caused by text line distortion and inclination, the center curve of the text line is recalculated using the center of the minimum circumscribed circle of each character, and the average line height in the normal direction of the curve is used as the line height.

[0087] The embodiment of the present invention provides the detection of abnormal line height and line spacing:

[0088] In order to prevent the document reference line height or line spacing from being affected by the extreme line height, a reference value is selected by mutual voting between line heights or line spacing, and this is used as a reference for anomaly detection to reduce the impact of extreme values ​​on anomaly detection.

[0089] The intelligent document inspection and identification method based on multispectral imaging provided by the embodiment of the present invention also includes:

[0090] Seal comparison methods include:

[0091] The seal extraction method based on color channel suppression of background interference aims at the problem of background interference in the seal extraction process. The difference between the seal foreground and background in each RGB channel is used to effectively suppress background interference such as printed text and handwritten text, and extract the foreground seal.

[0092] Seal shape classification and parameter calculation method, for the problem of seal size calculation, the seal shape is obtained by using seal contour extraction, polygon fitting and minimum circumscribed circle fitting algorithm, and then the seal size is calculated according to the seal shape;

[0093] The seal similarity calculation method based on feature point matching aims at the similarity calculation problem of two seals with different scales and directions. The SIFT algorithm with scale and direction invariance is used to extract feature points. The matching point transformation matrix is ​​obtained by combining the RANSAC algorithm. The two seals are transformed into the same scale and direction. Finally, the difference points of the two seals are calculated and the similarity is calculated using the formula.

[0094] A yellow dot tracking method, comprising:

[0095] Method for removing the background of a color laser printed document image and extracting a yellow dot array;

[0096] Spatial feature extraction method of points in yellow dot array;

[0097] Pattern fusion method for points with similar spatial features;

[0098] A fusion method for different dot patterns based on their relative positions in space.

[0099] like Fig.17 As shown, an intelligent document inspection and identification device based on multispectral imaging provided by an embodiment of the present invention includes:

[0100] A correction module, used to extract the document from the image and perform correction;

[0101] A segmentation module, for segmenting a text line into continuous complete characters using a text line character segmentation method based on clustering;

[0102] A calculation module, used to detect the geometric center of each character in the text line and calculate the center curve of the text line;

[0103] The detection module is used to calculate the row height and row spacing, and detect abnormal row height and row spacing.

[0104] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the intelligent document inspection and identification method based on multi-spectral imaging.

[0105] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the intelligent document inspection and identification method based on multi-spectral imaging.

[0106] Another object of the present invention is to provide an information data processing terminal, which is used to implement the intelligent document inspection and identification device of the document inspection instrument based on multi-spectral imaging.

[0107] The present invention is specifically implemented:

[0108] In the judicial field, document authenticity identification is a major problem. Document formation time identification is one of the ways to determine the authenticity of documents. However, due to the numerous and complex factors that affect the verification of document formation time, there is still no systematic identification theory and method for document formation time identification. In view of the above problems, the computer field has found an effective solution from the perspective of images. At present, the deep learning method has made breakthroughs in tasks such as red and black time sequence identification. However, as one of the means of document authenticity identification, document overprint detection currently lacks an effective solution. Document overprint detection is to detect whether the document font size and line spacing are abnormal, which involves text detection methods. As an important step in OCR technology, text detection currently has many mature deep learning-based methods, such as OCR technology provided by Baidu and Tencent. However, for the problem of document overprint detection, the current text detection technology has problems such as insufficient accuracy and high training cost. Therefore, how to improve text detection accuracy and reduce costs in document overprint detection is a problem to be solved.

[0109] To solve the above problems, this application proposes an A4 document anomaly detection method. First, the document is distorted and extracted from the image; then the text line is segmented into continuous complete characters using an improved text line character segmentation method based on the projection method; then the geometric center of each character in the text line is detected, and the text line center curve is calculated; finally, the line height and line spacing are calculated, and abnormal line height and line spacing are detected. The entire process not only performs global distortion correction on the document, but also performs direction detection on each text line and each character, effectively reducing the detection error caused by file distortion.

[0110] Figure 1 The flowchart of the A4 document anomaly detection method specifically includes the following steps:

[0111] Step 1: Dedistort and extract the document, such as Figure 2 A4 document segmentation and correction flow chart is shown.

[0112] First, the HED edge detection algorithm is used to detect the edge of the A4 document;

[0113] Then obtain the inner contour coordinate point set of the edge of the A4 document, perform quadrilateral fitting, and find four corner points CornerPts = {cornerPts1, cornerPts2, cornerPts3, cornerPts4};

[0114] Then, the edge points of each side are sampled at intervals to obtain a sampling point set Samples = {sample1, sample2, ...}, and the projection point Targets = {target1, target2, ...} of the sampling point Samples on the corresponding side of the fitted quadrilateral is calculated;

[0115] Then the TPS algorithm is used to transform the sampling point set to the target point set, that is,

[0116]

[0117] Calculate the TPS transformation matrix W and perform distortion correction on the entire document;

[0118] Then calculate the perimeter of the fitted quadrilateral, and determine the document length, width, and the quadrilateral corner mapping target point CornerDsts according to the A4 aspect ratio of 297:210.

[0119] CornerDsts={cornerDst1, cornerDst2, cornerDst3, cornerDst4}

[0120] in

[0121]

[0122] Use perspective transformation to perform perspective correction on documents;

[0123] Finally, the perspective-corrected document is segmented to obtain the globally corrected A4 document. The complete process diagram is shown in the figure. Fig.12 shown.

[0124] Holistically-nested edge detection (HED) is an end-to-end edge detection system. Holistically means that the result of edge prediction is based on an image-to-image, end-to-end process, while nested emphasizes the process of continuously inheriting and learning to obtain accurate edge prediction maps in the generated output process. HED uses a multi-scale method to learn features. The schematic diagram of the edge detection results of this method at multiple scales is shown in the figure. Fig.13 As shown in the figure, d, e, and f are the edge detection results obtained by the response of convolutional layers 2, 3, and 4 respectively. The final output of the HED method is far better than the Canny operator. Thin Plate Spline (TPS) is a commonly used 2D interpolation method. Given some corresponding control points in two images, the TPS method can perform a specific deformation on one of the images so that its control points can coincide with the control points of the other image.

[0125] In step 1, if the Canny edge detection is used to detect the edge of the A4 document, it cannot guarantee that the continuous edge of the document can be detected. Therefore, the HED edge detection algorithm is used to stably detect the continuous edge of the document. To correct the distortion of the document, the TPS thin plate spline interpolation method is used, and the sampled edge detection points are regarded as control points, and the projection points are regarded as the target points after the control points are moved. To a certain extent, the document distortion can be globally corrected.

[0126] Step 2: Use the clustering-based text line character segmentation method to segment the text line into continuous and complete character sequences, such as Figure 3 The flowchart of the clustering-based text line character segmentation method is shown in the figure.

[0127] First, the A4 document is projected horizontally to obtain a histogram, and the text lines are segmented according to the histogram, such as Fig.11 As shown;

[0128] Then use the vertical projection histogram method to preliminarily segment the text line characters, such as Fig.14 As shown, the width and height of the characters are counted, and characters with a smaller sum of width and height (punctuation marks, noise, etc.) are removed to obtain a text line character sequence Seq = {char1, char2, char3, ...};

[0129] Then, the width is divided into two categories, A and B, using a clustering algorithm. Category A represents complete characters, and category B represents incomplete characters.

[0130]

[0131] And find the minimum width of the A-type characters min ;

[0132] Then set the B character char i Merge two adjacent Class B characters backward to get a new character char i+1 , determine the width of the merged characters Is it smaller than width? min If it is less than width min , then continue to merge the adjacent Class B characters backwards, otherwise the merged characters are regarded as Class A characters. The formula is as follows:

[0133] char i+1 =Concat(char i ,char i+1 ),char i ,char i+1 ∈B

[0134]

[0135] Finally, the text line character segmentation result is obtained.

[0136] In step 2, the traditional vertical projection histogram character segmentation method has defects, that is, there is an over-segmentation problem for left-right structured Chinese characters. The proportion of Chinese character shape structures in IDS data statistics shows that left-right structured Chinese characters account for about 65%, so it can be estimated that the ratio of left-right structured Chinese characters to non-left-right structured Chinese characters in the text line is about 7:3. The traditional vertical projection histogram character segmentation method can find complete characters in the segmentation results. According to the characteristics of Chinese characters, it can be known that the widths of Chinese characters with the same font size in the same text line are basically the same. In view of the problem of over-segmentation of left-right structured Chinese characters, a text line character segmentation method based on clustering is proposed based on the above two points. Among them, after the initial segmentation of the text line characters, all punctuation marks need to be removed as much as possible, so commas and periods with a small sum of width and height are removed. Because the width of complete characters and incomplete characters is significantly different, the initial segmented characters are divided into two categories according to the width, which is a binary classification problem. In addition to using clustering algorithms such as K-means, classification algorithms such as support vector machines SVM can also be used.

[0137] Step 3: Detect the geometric center of each character in the text line and calculate the center curve of the text line, such as Figure 4 The text line center curve detection flow chart is shown in the figure.

[0138] First calculate the center of the character's smallest circumscribed circle and set the center i Considered as the geometric center of the character

[0139] center={center1,center2,center3,…};

[0140] Then use the polynomial fitting function y = a0 + a1x + a2x 2 +…+a i x i +…a n x n Perform curve fitting on the geometric center of text lines.

[0141] The geometric center line of each character in a text line approaches a horizontal straight line, but due to document distortion, some text line center curves are bent or tilted, so the center curve of the text line needs to be recalculated.

[0142] Step 4: Calculate the row height and row spacing, and detect abnormal row height and row spacing, such as Figure 5 The text line height and line spacing calculation and anomaly detection flowchart is shown in the figure.

[0143] First, calculate the normal of the center curve of the text line at the geometric center point, and calculate the character height Height along the normal direction;

[0144] Then calculate the average height of the characters in the text line Represents the height of the text line, and uses the line height to calculate the Y coordinates of the upper and lower edges of the text line up and Y bottom , the formula is as follows,

[0145]

[0146] Line up and Line bottom Indicates the vertical coordinates of the upper and lower boundaries of the text line in the entire document.

[0147] Then use the Y coordinates of the upper and lower edges of each text line to calculate the line spacing;

[0148] Then set the row height and row spacing error thresholds errorH and errorD, and count the height of each row The absolute value of the difference between the height of the other rows and the number of rows within errorH is nums. The row with the largest height of nums is selected as the reference row height stdH. The absolute value of the difference between the height of the reference row and The row height is recorded as abnormal row height SuspiciousH, and the abnormal row spacing detection method is the same as above;

[0149] Finally, convert the row height and row spacing into millimeters based on the actual length and width dimensions of the A4 paper and the number of pixels in the length and width.

[0150] In step 4, the row number needs to be recorded each time the row height and the Y coordinates of the upper and lower edges of the text row are calculated to facilitate the subsequent calculation of the row spacing and the detection of anomalies. The actual error size value in millimeters is converted to the number of pixels in pixels and regarded as the error threshold errorH or errorD.

[0151] In summary, the present application proposes an A4 document anomaly detection method, which first corrects the distortion of the document and extracts it from the image, and can globally correct the document distortion to a certain extent; then uses the clustering-based text line character segmentation method to segment the text line into continuous complete characters, which can improve the problem of excessive segmentation of left-right structured Chinese characters by the traditional vertical projection histogram character segmentation method; then detects the geometric center of each character in the text line, and calculates the text line center curve, reducing the influence of text line inclination and curvature on character height; finally, calculates the line height and line spacing, and detects abnormal line height and line spacing.

[0152] In the judicial field, seal comparison to identify true and false seals is an important task. Seal comparison judges the authenticity of seals based on the shape, size, and detail differences of seals. At present, the technical solutions for seal recognition include end-to-end seal detection and recognition, seal sequence prediction, etc., but there are problems such as slow speed, difficult training, and high maintenance cost. Therefore, how to improve detection efficiency, reduce training difficulty, reduce maintenance costs, and thus improve user experience is a technical problem to be solved.

[0153] To solve the above problems, this application proposes a seal comparison method. First, seal extraction is performed to obtain a standard seal and a seal to be compared; then, seal shape detection is performed to classify the seal into one of four shapes: circular, elliptical, rectangular and square; then, the inner and outer diameters or the length and width of the seal are calculated according to the seal shape; finally, the difference points of the two seals are detected and the seal similarity is calculated. The entire process does not require model training, the maintenance cost is low, the detection process is simple and efficient, the measurement error is within 1 mm, the accuracy is high, and seals of different scales and directions can be compared.

[0154] Figure 6 The flow chart of the seal comparison method is as follows:

[0155] Step 1: Seal extraction, obtain the standard seal and the seal to be compared, such as Figure 7 The seal extraction flow chart is shown in the figure.

[0156] First, separate the RGB channels of the collected standard seal and the seal image to be compared, and calculate the histogram of each channel image respectively, as shown in the following figure: Fig.15 shown.

[0157] Then calculate the gradient of each histogram

[0158]

[0159] Find the brightness b corresponding to the maximum value of the histogram, and traverse the gradient grad from b→0 C , looking for the first satisfying The brightness value threshold is set, and the pixel values ​​less than the threshold are set to 0, and the pixel values ​​greater than or equal to the pixel value are set to 255, and the binary image after each channel processing is obtained. The formula is as follows. The purpose of the above steps is to separate the foreground and background of each channel image.

[0160] Foreground = {pix|pix∈Gray C ,Brightness(pix)=0}

[0161] Background={pix|pix∈Gray C ,Brightness(pix)=255}

[0162]

[0163] Finally, the image with more non-zero values ​​in the R and B channels is used as the background image BackgroundImg, and the image with fewer non-zero values ​​in the other two channels is used as the foreground seal image ForegroundImg. The pixel points with a pixel value of 0 in the background image are assigned a value of 255 in the foreground seal image. The resulting image is the final extracted seal image.

[0164] StampImg=ForegroundImg|~BackgroundImg

[0165] For example, taking the red seal sample as an example, in the above steps, the B channel image has more non-zero values ​​in the R and B channel images, so the B channel image is used as the background image BackgroundImg B , the image with fewer non-zero values ​​in the R and G channels is used as the foreground stamp image, assuming that the R channel image is used as the foreground stamp image ForegroundImg R , then StampImg=ForegroundImg R |~BackgroundImg B , that is, suppress the background interference (such as printed text, handwritten signature, etc.) in the foreground stamp image StampImg, so that the pixels with a value of 0 in the obtained foreground stamp image represent the stamp, and the pixels with a value of 255 represent the background, thereby achieving the purpose of extracting the stamp.

[0166] Step 2, perform seal shape detection and classify the seal into one of the four shapes: circle, oval, rectangle and square. Figure 8 The seal shape detection flow chart is shown in the figure.

[0167] First, the outline of the seal extracted in step 1 is extracted.

[0168] Then calculate the minimum circumscribed rectangle and the aspect ratio of the circumscribed rectangle for the outline. From the aspect ratio of the circumscribed rectangle, it can be preliminarily determined whether the seal is a circular or square seal with an aspect ratio close to 1 or an elliptical or rectangular seal with an aspect ratio far from 1.

[0169] Perform polygon fitting on the seal outline. If the number of sides is 4, it means the seal is a square or rectangle. If the number of sides is greater than 4, it means the seal is a circle or an ellipse.

[0170] The shape of the seal can be obtained by combining the aspect ratio of the circumscribed rectangle calculated above.

[0171] Step 3: Calculate the inner and outer diameters or length and width of the stamp according to its shape, such as Fig. 9 The flow chart for calculating the inner and outer diameters of a seal is shown in the figure.

[0172] First determine the shape of the seal. If it is a circular seal, calculate the inner and outer diameters of the seal. Otherwise, calculate the length and width of the circumscribed rectangle as the output result.

[0173] The calculation methods of the inner and outer diameters of circular seals include:

[0174] The seal contour is extracted, which includes the inner contour and outer contour of the circular seal. The radius and center coordinates of the circular contour are obtained by using the minimum circumscribed circle fitting algorithm.

[0175] The ratio of the contour area to the area of ​​the minimum circumscribed circle of the contour is calculated. Contours with a ratio < 0.8 are eliminated, indicating that the contour is not a circle but the contour of a font or other pattern inside a circular seal.

[0176] After eliminating the non-circular contours, the minimum circumscribed circles of the remaining contours are arranged in descending order of radius {R1, R2, R3, ... | R1 ≥ R2 ≥ R3 ≥ ...}, where R1 is the outer diameter of the seal and R2 is the inner diameter of the seal.

[0177] The length and width of the circumscribed rectangle have been calculated in step 2, seal shape detection.

[0178] Finally, the obtained inner and outer diameters or length and width are converted from pixel units to millimeter units using the conversion coefficient of the image acquisition system.

[0179] Step 4: Detect the difference points between the two seals and calculate the seal similarity, such as Fig.10 The seal difference point detection flow chart is shown in the figure.

[0180] First, the SIFT algorithm is used to extract the standard stamps in step 1. 标准印章 Stamp to be compared 待比对印章 Extract feature points and match them. SIFT algorithm is scale-invariant and direction-invariant, and has high efficiency. It is suitable for targets with simple feature points, uncertain scale and direction, such as seals.

[0181] Then use the RANSAC algorithm to remove the mismatched points and obtain the feature point transformation matrix W.

[0182] After obtaining the transformation matrix W, use affine transformation to transform Stamp 标准印章 and Stamp 待比对印章 The size and direction are transformed to be consistent.

[0183] The final difference Mark the difference points on the standard seal and use the formula

[0184]

[0185] The similarity between the two seals is calculated, where CountNonzero(binaryImg) represents the number of non-zero pixels in the binary image binaryImg.

[0186] In summary, the seal comparison method proposed in this application first performs seal extraction to separate the target seal from the background interference; then detects the seal shape and divides the seal shape into four categories: circular, elliptical, rectangular and square; then calculates the inner and outer diameters or length and width of the seal according to the seal shape; finally, detects the difference points between the standard seal and the seal to be compared, and calculates the similarity. The seal extraction uses the histogram analysis method to effectively remove interference such as background text, and the method is simple and efficient; the method is pixel-level operation for measuring the inner and outer diameters and length and width of the seal, so the error is controllable; since the application uses the SIFT algorithm to match the features of the two seals, the size and direction of the two seals do not need to be consistent, which can improve the user experience.

[0187] Furthermore, the background of the image is removed to obtain the yellow dot array. First, the obtained color laser printed document image is gray-scaled to obtain a grayscale image, and then the grayscale image is Gaussian blurred and then threshold segmented. Then a series of morphological operations are performed on the obtained image to connect the background part of the image to facilitate background removal, and finally the yellow dot array in the image is obtained. The number and coordinates of the points in the dot matrix are obtained through the connected components of the dot matrix. The process is as follows Fig.16 shown.

[0188] The distribution of points around each point is used as the spatial feature of this point to obtain the feature of the point. Each point is taken as a pole in turn, and the relative position between the fixed number of points closest to the center point and the center point is expressed in polar coordinates. The fixed number of polar angles and polar diameters obtained are the features of the point. Assume that the target point is P0, and the coordinates in the rectangular coordinate system are (x0, y0). A point P around k (x k ,y k ), then the polar coordinates of the point relative to the target point can be calculated by the following formula:

[0189] θ k =arctan((y k -y0) / (x k -x0))

[0190] Get a set of coordinates (l k ,θ k), after calculating a fixed number, a set of coordinate lists is obtained as follows

[0191] {(l1,θ1),(l2,θ2)...(l n ,θ n )}

[0192] Finally, the result obtained by sorting the polar angles from small to large is the feature representation of point P0.

[0193] Points with the same or similar features are grouped together. k A group of its own k , then take two groups of G each time from all groups i ,G j , analyze their feature similarity, and the calculation method is as follows:

[0194]

[0195] Among them, λ is a variable parameter that can adjust the weights of the polar diameter and polar angle.

[0196] Select groups with the same or similar features and combine them into the same group. Each group only selects fixed points as feature representatives. The surrounding points of the same group have similar distributions relative to the center point, that is, they have the same pattern. It can be considered that the points in a set are the corresponding points when the same yellow dot pattern appears in different positions. Finally, it is necessary to check whether there is overlap between patterns in different groups. For each pair of patterns, calculate the distance matrix from each point in the pattern to each point in the other pattern. Points with too small distances can be considered as incorrectly classified points and need to be deleted.

[0197] After grouping, the distribution of points around each point in the same group has a strong similarity, so we can perform image fusion based on this feature to obtain a pattern of the center point and the surrounding points, which is part of the complete dot matrix pattern.

[0198] The patterns of different groups are fused to obtain a complete dot matrix pattern. After the fusion of the same group of patterns is completed, the dot matrix sub-images of different mode patterns are gradually fused according to the relative positions in space. In the fusion process of different groups of patterns, complementary points need to be filled.

[0199] The obtained complete periodic dot matrix pattern is compared with the existing dot matrix periodic database to decipher the file-related information contained in the dot matrix.

[0200] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0201] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. An intelligent document inspection and identification method based on multispectral imaging, characterized in that: The following steps are involved: Step 1, extract the document from the image and correct it; Step 2, using a text line character segmentation method based on clustering to segment the text line into continuous complete characters; Step 3, detecting the geometric center of each character in the text line and calculating the text line center curve; Step 4: Calculate row height and row spacing, and detect abnormal row height and row spacing.

2. The intelligent document inspection and identification method based on multispectral imaging of document inspection instrument as claimed in claim 1 is characterized in that: The document is extracted from the image and corrected: To solve the problem of document distortion correction, the TPS thin plate spline interpolation method is used. The sampled edge detection points are regarded as control points, and the projection points are regarded as target points after the control points are moved, so as to perform global correction on the document distortion.

3. The intelligent document inspection and identification method based on multispectral imaging of document inspection instrument as claimed in claim 1 is characterized in that: The text line is segmented into continuous complete characters using the clustering-based text line character segmentation method: To address the problem of over-segmentation of left-right structured Chinese characters, a clustering algorithm is used to divide characters into complete and incomplete character classes, and adjacent incomplete character classes are continuously spliced ​​together to form complete characters.

4. The intelligent document inspection and identification method based on multispectral imaging of document inspection instrument as claimed in claim 1, characterized in that: The calculation of the text line center curve is: To address the problem of increased line height calculation errors caused by text line distortion and inclination, the center curve of the text line is recalculated using the center of the minimum circumscribed circle of each character, and the average line height in the normal direction of the curve is used as the line height.

5. The intelligent document inspection and identification method based on multispectral imaging of document inspection instrument as claimed in claim 1, characterized in that: The detection of abnormal line height and line spacing: In order to prevent the document reference line height or line spacing from being affected by the extreme line height, a reference value is selected by mutual voting between line heights or line spacing, and this is used as a reference for anomaly detection to reduce the impact of extreme values ​​on anomaly detection.

6. The intelligent document inspection and identification method based on multispectral imaging of document inspection instrument as claimed in claim 1, characterized in that: The intelligent document inspection and identification method based on multispectral imaging also includes: Seal comparison methods include: The seal extraction method based on color channel suppression of background interference aims at the problem of background interference in the seal extraction process. The difference between the seal foreground and background in each RGB channel is used to effectively suppress background interference such as printed text and handwritten text, and extract the foreground seal. Seal shape classification and parameter calculation method, for the problem of seal size calculation, the seal shape is obtained by using seal contour extraction, polygon fitting and minimum circumscribed circle fitting algorithm, and then the seal size is calculated according to the seal shape; The seal similarity calculation method based on feature point matching aims at the similarity calculation problem of two seals with different scales and directions. The SIFT algorithm with scale and direction invariance is used to extract feature points. The matching point transformation matrix is ​​obtained by combining the RANSAC algorithm. The two seals are transformed into the same scale and direction. Finally, the difference points of the two seals are calculated and the similarity is calculated using the formula. A yellow dot tracking method, comprising: Method for removing the background of a color laser printed document image and extracting a yellow dot array; Spatial feature extraction method of points in yellow dot array; Pattern fusion method for points with similar spatial features; A fusion method for different dot patterns based on their relative positions in space.

7. An intelligent document inspection and identification device based on multispectral imaging for implementing the intelligent document inspection and identification method based on multispectral imaging for document inspection as claimed in any one of claims 1 to 6, characterized in that: The intelligent document inspection and identification device based on multispectral imaging includes: A correction module, used to extract the document from the image and perform correction; A segmentation module, for segmenting a text line into continuous complete characters using a text line character segmentation method based on clustering; A calculation module, used to detect the geometric center of each character in the text line and calculate the center curve of the text line; The detection module is used to calculate the row height and row spacing, and detect abnormal row height and row spacing.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the intelligent document inspection and identification method based on multi-spectral imaging of a document inspection instrument as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the intelligent document inspection and identification method based on multispectral imaging of a document inspection instrument as described in any one of claims 1 to 6.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the intelligent document inspection and identification device based on multi-spectral imaging as described in claim 7.

Images