Method, apparatus, system, and storage medium for detecting the rotation angle of a paper document to add a physical mark to the paper document

Abstract

The present invention relates to a method, apparatus, system, and storage medium for detecting the rotation angle of a paper document to add a physical mark to the paper document. The method includes: obtaining a reference image and a target image; extracting a set of reference feature points from the reference image and a set of target feature points from the target image; extracting a reference feature descriptor from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors, and extracting a target feature descriptor from each target feature point in the set of target feature points to obtain a set of target feature descriptors; generating initial descriptor matching pairs between the two sets of descriptors, and obtaining a final set of matching pairs through multiple screenings; calculating a transformation matrix representing the geometric transformation relationship from the reference feature points to the matched target feature points; extracting the rotation and scaling parts from the transformation matrix, and calculating the value of the rotation angle therefrom, so as to achieve the technical effect of quickly and accurately detecting the rotation angle of the paper document.

Description

Technical Field

[0001] This application relates to detecting the rotation angle of a paper document. In particular, it relates to a method, apparatus, system, and computer-readable storage medium for detecting the rotation angle of a paper document to add a physical mark thereto. Background Art

[0002] On the production line of the manufacturing industry, each product's packaging must contain a certificate of conformity. The certificate of conformity usually clearly records important information such as the name and date of the person who stamped it. Traditionally, these stamping operations rely on manual execution. However, the manual stamping process is cumbersome and inefficient. The operator needs to perform fine alignment on each certificate of conformity, which is time-consuming and prone to deviation in the position of the stamp, making it difficult to maintain the uniformity and accuracy of the stamp. With the continuous improvement of production automation and the significant increase in the speed of the manufacturing line, the manual stamping method can no longer meet the requirements of high-efficiency production.

[0003] The pattern and text design of the certificate of conformity often follow a certain regularity and repeatability. Although this feature facilitates the standardized entry of information, it poses challenges to the automated stamping technology. In the field of image processing, the existence of a large number of similar feature points makes the feature matching process complex and difficult, prone to false matching, and affecting the final stamping accuracy.

[0004] In view of this, it is desirable to provide a method for automatically stamping the certificate of conformity, which can quickly and accurately detect the rotation angle of the certificate of conformity to be stamped relative to the stamp reference orientation, so as to achieve precise automated stamping of the certificates of conformity transmitted at high speed on the production line. Summary of the Invention

[0005] This application is proposed in view of the above problems. The main purpose of this application is to provide an apparatus, method, system, and computer-readable storage medium for detecting the rotation angle of a paper document to add a physical mark thereto, so as to solve the technical problem in the prior art that it is difficult to accurately and quickly add physical marks to paper documents with high similarity transmitted at high speed on the production line.

[0006] To achieve the above object, according to one aspect of the present application, a method for detecting the rotation angle of a paper document to add a physical mark to the paper document is provided. The method for detecting the rotation angle of the paper document to add a physical mark to the paper document includes: obtaining a reference image and a target image, where the reference image is obtained by photographing a reference paper document in a predetermined orientation, and the target image is obtained by photographing a target paper document to be marked in an arbitrary orientation. The rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document. Regular patterns and / or texts are provided on the surfaces of the reference paper document and the target paper document; extracting a set of reference feature points of reference feature points from the reference image, and extracting a set of target feature points of target feature points from the target image; extracting corresponding reference feature descriptors from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extracting corresponding target feature descriptors from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image; generating a plurality of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through approximate nearest neighbor search, and obtaining a final matching pair set through multiple screenings. The final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other; calculating a transformation matrix representing the geometric transformation relationship of transforming the reference feature point in each feature point matching pair among the plurality of feature point matching pairs to the matching target feature point, where the transformation matrix includes a rotation and a scaling part; and extracting the rotation and scaling parts in the transformation matrix, and calculating the value of the rotation angle from the rotation and scaling parts.

[0007] In this way, the detection of the rotation angle of the target paper document can be achieved. In particular, through the combined action of the initial rough matching and multiple screenings, the rotation angle between the target paper document to be marked in an arbitrary orientation and the reference paper document can be automatically detected in real time and accurately. Therefore, the above method for detecting the rotation angle of a paper document to add a physical mark to the paper document can be applied to the automatic addition of physical marks to target paper documents on a production line.

[0008] Further, according to an embodiment of the present application, the multiple screenings include: a first screening based on the comparison of the matching distance of each initial descriptor matching pair with the minimum matching distance of the initial descriptor matching pair, and a second screening based on the random sample consensus algorithm with a preset maximum reprojection error threshold.

[0009] In this way, through the synergistic effect of an initial rough matching, a first screening based on the comparison of matching distances, and a second screening using the Random Sample Consensus (RANSAC) algorithm, the real-time and accurate detection of the rotation angle of the target paper document can be achieved. In particular, through the combination of two specific screenings, while reducing the computational complexity of the feature point matching of the two images, the accuracy of the feature point matching can be improved, and the computational complexity of the subsequent rotation angle can also be reduced. Thereby, the fast and accurate detection of the rotation angle of the target paper document is realized.

[0010] Further, according to an embodiment of the present application, through approximate nearest neighbor search, a plurality of initial descriptor matching pairs between the reference feature descriptor set and the target feature descriptor set are generated, and a final matching pair set is obtained through multiple screenings. The final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other, including: generating an initial descriptor matching pair set between the reference feature descriptor set and the target feature descriptor set through the approximate nearest neighbor search algorithm. The initial descriptor matching pair set includes the plurality of initial descriptor matching pairs, and each initial descriptor matching pair includes a matching reference feature descriptor and a target feature descriptor; calculating the matching distance of each initial descriptor matching pair, and determining the minimum value among the matching distances of the initial descriptor matching pairs in the initial descriptor matching pair set as the minimum matching distance; comparing the matching distance of each initial descriptor matching pair with the minimum matching distance, and performing a first screening on the plurality of initial descriptor matching pairs based on the comparison result to obtain an intermediate descriptor matching pair set. The intermediate descriptor matching pair set includes a plurality of intermediate descriptor matching pairs selected from the plurality of initial descriptor matching pairs; identifying a plurality of intermediate feature point matching pairs corresponding to the plurality of intermediate descriptor matching pairs, and each intermediate feature point matching pair includes a matching reference feature point and a target feature point. For the plurality of intermediate feature point matching pairs, using calculation parameters including a preset maximum reprojection error threshold, an affine transformation model that best represents the transformation relationship between the reference feature point and the target feature point is obtained through the Random Sample Consensus (RANSAC) algorithm, and a second screening is performed on the plurality of intermediate feature point matching pairs based on the affine transformation model to obtain a final matching pair set including a plurality of feature point matching pairs.

[0011] In this way, through the synergistic effect of an initial rough matching by approximate nearest neighbor search, a first screening using the descriptor-based matching distance comparison, and a second screening using the Random Sample Consensus (RANSAC) algorithm, the matching speed can be improved while effectively excluding outliers, ensuring that only the feature point matching pairs that truly reflect the document rotation angle are used for subsequent geometric transformation calculations.

[0012] Further, according to an embodiment of the present application, the calculation parameters further include: a confidence level, a maximum number of selection iterations for the selection of the affine transformation model, and a maximum number of refinement iterations for the optimization of the affine transformation model, and the maximum reprojection error threshold is 0.3 pixels, the confidence level is 0.99, the maximum number of selection iterations is 200, and the maximum number of refinement iterations is 10.

[0013] In this way, by presetting a specific maximum reprojection error threshold, a confidence level, a maximum number of selection iterations for the selection of the affine transformation model, and a maximum number of refinement iterations for the optimization of the affine transformation model, the consistency and reliability of the random sample consensus algorithm when processing images of different paper documents can be ensured. In particular, the preset maximum reprojection error threshold of 0.3 pixels covers the positional deviation of physical marks allowed on the paper document and slight deformation of the document (such as a certificate). The confidence level of 0.99 ensures that a full inlier sample can be sampled to cope with high-noise scenarios. The maximum number of selection iterations of 200 is sufficient to avoid timeouts caused by an extremely low inlier ratio. And the maximum number of refinement iterations of 10 can meet the dual requirements of processing speed and marking addition accuracy for scenarios where marks are added to paper documents (such as certificates). Therefore, by setting the above calculation parameters, the quality of the final set of matching pairs and the robustness of the algorithm can be ensured.

[0014] Further, according to an embodiment of the present application, determining the minimum matching distance among the matching distances of the initial descriptor matching pairs in the initial descriptor matching pair set includes: obtaining a predetermined first reference value and a second reference value, where the first reference value is less than the second reference value; comparing the first reference value with the matching distance of each initial descriptor matching pair in the initial descriptor matching pair set, where, in each comparison, in response to the current first reference value being less than the matching distance of the initial descriptor matching pair, updating the current first reference value with the matching distance of the initial descriptor matching pair and using the updated first reference value for the next comparison until the comparison with the matching distances of all initial descriptor matching pairs is completed; comparing the second reference value with the matching distance of each initial descriptor matching pair in the initial descriptor matching pair set, where, in each comparison, in response to the current second reference value being greater than the matching distance of the initial descriptor matching pair, updating the current second reference value with the matching distance of the initial descriptor matching pair and using the updated second reference value for the next comparison until the comparison with the matching distances of all initial descriptor matching pairs is completed; comparing the first reference value after the comparison is completed with the second reference value after the comparison is completed; and in response to the first reference value after the comparison is completed being greater than the second reference value after the comparison is completed, using the second reference value after the comparison is completed as the minimum matching distance.

[0015] In this way, compared with simply traversing all the initial descriptor matching pairs in the initial descriptor matching pair set using a single reference value to find the minimum matching distance, by traversing all the matching distances using two reference values to respectively find the maximum and minimum values among the matching distances of the initial descriptor matching pairs, and only after determining that the first reference value after the comparison is greater than the second reference value after the comparison, taking the second reference value after the comparison as the minimum matching distance, it can be ensured that the set of initial descriptor matching pairs traversed is a non-empty valid set, that is, the effectiveness of the traversal process is ensured, thereby ensuring the robustness of the calculated minimum matching distance.

[0016] Further, according to an embodiment of the present application, extracting the rotation and scaling parts from the transformation matrix and calculating the value of the rotation angle from the rotation and scaling parts includes: extracting the rotation and scaling parts from the transformation matrix as the rotation matrix; normalizing the rotation matrix to obtain the normalized rotation matrix; using the arccosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix; and determining the quadrant in which the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix.

[0017] In this way, through mathematical operations on the transformation matrix, the rotation angle of the paper document can be determined efficiently and accurately, which is applicable to real-time or batch processing scenarios.

[0018] Further, according to an embodiment of the present application, the normalized rotation matrix is a 2×2 matrix. Determining the quadrant in which the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix includes: when the value of the element [0, 0] of the normalized rotation matrix is greater than 0, the value of the element [1, 1] is greater than 0, the value of the element [0, 1] is less than 0, and the value of the element [1, 0] is greater than 0, determining that the rotation angle is in the first quadrant; when the value of the element [0, 0] of the normalized rotation matrix is less than 0, the value of the element [1, 1] is less than 0, the value of the element [0, 1] is less than 0, and the value of the element [1, 0] is greater than 0, determining that the rotation angle is in the second quadrant; when the value of the element [0, 0] of the normalized rotation matrix is less than 0, the value of the element [1, 1] is less than 0, the value of the element [0, 1] is greater than 0, and the value of the element [1, 0] is less than 0, determining that the rotation angle is in the third quadrant; and when the values of the elements of the normalized rotation matrix do not belong to any of the listed cases, determining that the rotation angle is in the fourth quadrant.

[0019] In this way, the rotation direction can be determined by detecting the positive and negative of the elements of the normalized rotation matrix, which increases the accuracy of angle calculation. When processing paper documents with complex rotation directions, the correct physical marking direction can be ensured.

[0020] Further, according to an embodiment of the present application, the transformation matrix is a 2×3 affine transformation matrix, which is represented in the following form:

[0021]

[0022] Wherein, elements a11, a21, a12, and a22 describe rotation and scaling, while elements b1 and b2 describe translation. The final set of feature point matching pairs includes N feature point matching pairs. The coordinates of the reference feature point in the i-th feature point matching pair are xi and yi, and the coordinates of the target feature point in the i-th feature point matching pair are ui and vi. Both i and N are positive integers and 1≤i≤N. Calculating the transformation matrix includes: for the i-th feature point matching pair, constructing the following equations:

[0023] ui = a11 × xi + a12 × yi + b1

[0024] vi = a21 × xi + a22 × yi + b2;

[0025] Combine the equations for all N feature point matching pairs and solve the combined equations using the least squares method to obtain the values of each element in the affine transformation matrix.

[0026] In this way, an optimal estimation of the elements of the affine transformation matrix is ensured, thereby improving the accuracy of subsequent rotation angle calculation and the positioning accuracy of physical markers.

[0027] According to another aspect of the present application, there is provided an apparatus for detecting the rotation angle of a paper document to add a physical mark to the paper document. The apparatus for detecting the rotation angle of a paper document to add a physical mark to the paper document includes: an acquisition module, configured to acquire a reference image and a target image, where the reference image is obtained by photographing a reference paper document in a predetermined orientation, and the target image is obtained by photographing a target paper document to be marked in an arbitrary orientation. The rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document. Regular patterns and / or texts are provided on the surfaces of the reference paper document and the target paper document; a first extraction module, configured to extract a set of reference feature points of reference feature points from the reference image and a set of target feature points of target feature points from the target image; a second extraction module, configured to extract corresponding reference feature descriptors from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extract corresponding target feature descriptors from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image; a matching module, configured to generate multiple initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through approximate nearest neighbor search, and obtain a final matching pair set through multiple screenings. The final matching pair set includes multiple feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other; and a calculation module, configured to: calculate a transformation matrix representing the geometric transformation relationship from the reference feature point in each feature point matching pair in the final matching pair set to the matching target feature point, where the transformation matrix includes a rotation and a scaling part; extract the rotation and scaling parts from the transformation matrix, and calculate the value of the rotation angle from the rotation and scaling parts.

[0028] Further, according to an embodiment of the present application, the multiple screenings include: a first screening based on a comparison of the matching distance of each initial descriptor matching pair with the minimum matching distance among the multiple initial descriptor matching pairs, and a second screening based on the random sample consensus algorithm with a preset maximum reprojection error threshold.

[0029] In this way, through the synergistic effect of the initial rough matching, the first screening based on the matching distance comparison, and the second screening using the random sample consensus algorithm, the real-time and accurate detection of the rotation angle of the target paper document can be achieved. In particular, through the combination of the two specific screenings, while reducing the computational amount of the feature point matching of the two images, the accuracy of the feature point matching can be improved, and the computational amount of the subsequent rotation angle calculation can also be reduced. Thereby, the fast and accurate detection of the rotation angle of the target paper document is realized.

[0030] Further, according to an embodiment of the present application, the matching module includes: an initial matching module configured to generate an initial set of descriptor matching pairs between a set of reference feature descriptors and a set of target feature descriptors through an approximate nearest neighbor search algorithm, the initial set of descriptor matching pairs including a plurality of initial descriptor matching pairs, each initial descriptor matching pair including a matched reference feature descriptor and a target feature descriptor; a first screening module configured to: calculate the matching distance of each initial descriptor matching pair, and determine the minimum value among the plurality of matching distances of the plurality of initial descriptor matching pairs as the minimum matching distance; compare the matching distance of each initial descriptor matching pair with the minimum matching distance, and perform a first screening on the plurality of initial descriptor matching pairs based on the comparison result to obtain an intermediate set of descriptor matching pairs, the intermediate set of descriptor matching pairs including a plurality of intermediate descriptor matching pairs selected from the plurality of initial descriptor matching pairs; and a second screening module configured to: identify a plurality of intermediate feature point matching pairs corresponding to the plurality of intermediate descriptor matching pairs, each intermediate feature point matching pair including a matched reference feature point and a target feature point, for the plurality of intermediate feature point matching pairs, use calculation parameters including a preset maximum reprojection error threshold, obtain an affine transformation model that best represents the transformation relationship between the reference feature points and the target feature points through a random sample consensus algorithm, and perform a second screening on the plurality of intermediate feature point matching pairs based on the affine transformation model to obtain a final set of matching pairs including a plurality of feature point matching pairs.

[0031] Further, according to an embodiment of the present application, the calculation module is configured to: extract the rotation and scaling parts from the transformation matrix as the rotation matrix; normalize the rotation matrix to obtain a normalized rotation matrix; use the arccosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix; and determine the quadrant in which the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix.

[0032] According to another aspect of the present application, there is provided a system for automatically adding physical marks to paper documents, the system for automatically adding physical marks to paper documents including: a conveyor belt for conveying a target paper document to be added with physical marks in any orientation, the surface of the target paper document having regular patterns and / or texts; a camera located above the conveyor belt and configured to capture the target paper document reaching below the camera to generate a target image; a mark adding device located above the conveyor belt and arranged downstream of the camera; a rotation driving device connected to the mark adding device; and a controller connected to the camera and the rotation driving device and configured to: control the rotation driving device to rotate according to the calculated rotation angle according to the method of detecting the rotation angle of the paper document to add physical marks as described above, and cause the mark adding device to add physical marks to the target paper document.

[0033] According to another aspect of the present application, there is provided a computer-readable storage medium having instructions stored thereon, which when executed by a processor, cause the processor to execute the method of detecting the rotation angle of a paper document to add a physical mark to the paper document as described above.

[0034] In an embodiment of the present application, there is provided a method for quickly and accurately detecting the rotation angle of a paper document to add a physical mark to the paper document. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document includes: obtaining a reference image and a target image. The reference image is obtained by photographing a reference paper document in a predetermined orientation, and the target image is obtained by photographing a target paper document to be marked in an arbitrary orientation. The rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document. Regular patterns and / or texts are provided on the surfaces of the reference paper document and the target paper document. Extract a set of reference feature points of the reference feature points from the reference image, and extract a set of target feature points of the target feature points from the target image. Extract corresponding reference feature descriptors from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extract corresponding target feature descriptors from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image. Through approximate nearest neighbor search, generate a plurality of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors, and obtain a final matching pair set through multiple screenings. The final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other. Calculate a transformation matrix representing the geometric transformation relationship of transforming from the reference feature point in each feature point matching pair in the plurality of feature point matching pairs to the matching target feature point. The transformation matrix includes a rotation and a scaling part, and extract the rotation and scaling parts in the transformation matrix, and calculate the value of the rotation angle from the rotation and scaling parts, so as to at least solve the technical problem in the prior art that it is difficult to automatically add physical marks to paper documents with high similarity that are transmitted at high speed on a production line accurately and quickly, thereby realizing precise control of automatically adding physical marks to paper documents, and achieving the technical effect of significantly improving the mark adding efficiency while ensuring the unity of the mark position. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings forming a part of this application are used to provide a further understanding of the application. The illustrative embodiments and descriptions thereof of the application are used to explain the application and do not constitute an improper limitation of the application. In the drawings:

[0036] Figure 1It is a schematic diagram of a system for automatically adding physical marks to paper documents according to an embodiment of the present application. In this system for automatically adding physical marks to paper documents, a method for detecting the rotation angle of a paper document according to the present application is applied to add physical marks to the paper document;

[0037] Figure 2 It is a flowchart of a method for detecting the rotation angle of a paper document according to an embodiment of the present application to add physical marks to the paper document;

[0038] Figure 3 It shows Figure 2 a flowchart showing the detailed steps of step S400 shown;

[0039] Figure 4 It shows Figure 2 a flowchart showing the detailed steps of step S600 shown;

[0040] Figure 5 It is a block diagram of a device for detecting the rotation angle of a paper document according to an embodiment of the present application to add physical marks to the paper document.

[0041] Among them, the above-mentioned drawings include the following reference numerals:

[0042] Detailed implementation manners

[0043] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0044] It should be pointed out that unless otherwise specified, all technical and scientific terms used in the present application have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0045] In the present application, unless otherwise stated, the orientation words such as "upper, lower, top, bottom" are usually in the directions shown in the drawings, or in the vertical, perpendicular or gravitational directions of the components themselves; similarly, for the convenience of understanding and description, "inner, outer" refer to the inner and outer of the contours of the components themselves, but the above orientation words are not used to limit the present application.

[0046] In the present application, a "paper document" refers to a standardized document carried on paper, such as letters, reports, contracts, certificates, bills, brochures, cards, etc. The document can have a regular shape (such as rectangular, circular, oval, triangular, etc.), and there are regular patterns and / or texts on it (such as fixed text / pattern layout, such as Figure 1as shown by the pattern "XXX" on the target paper document 200 therein), and this pattern and / or text may be repetitive. In addition, multiple paper documents of the same type can share the same layout, with only some fields (such as signatures, serial numbers, dates, etc.) being different. This results in a large number of similar feature points possibly existing in the images of the paper documents, making it difficult to distinguish which feature points are truly corresponding during the matching process and prone to false matching and mismatching. In an exemplary embodiment, the paper document may include any one or more of a product's certificate of conformity, a product's warranty card, a logistics label of goods, a product's quality inspection report, and a contract document. Preferably, the paper document may be a product's certificate of conformity.

[0047] In the present application, a "physical mark" refers to a mark added by physical means at a predetermined position on the surface of a paper document to provide identification or information. Physical marks include, but are not limited to, seals (including postmarks), watermarks, barcodes, QR codes, text, numbers, etc. Physical marks can be added by various methods, such as laser etching, fluorescent or UV marking (visible under a specific light source), ink printing (such as inkjet, laser, or thermal transfer), thermal printing (for thermal paper), indentation or embossing (marks providing a tactile and visual texture), etc. Preferably, the physical mark may be a seal.

[0048] Currently, when using computer vision and image processing to automatically stamp the certificates of conformity on a production line, there are the following special requirements: It is necessary to accurately calculate the position of the seal in an extremely short time, which means that the image feature matching algorithm not only needs to have a high degree of accuracy but also must ensure sufficient real-time processing speed to adapt to the rhythm of the high-speed production line. In addition, since the automatic stamping of the certificate of conformity is carried out automatically on the production line, its image will be affected by changing light intensities. Although the Scale-Invariant Feature Transform (SIFT) algorithm has been applied to image matching and the calculation of the distance between matching points, in the case of using the Scale-Invariant Feature Transform algorithm, the calculation of the rotation angle between two images has not yet been achieved.

[0049] The present application can solve the above technical problems. The purpose of the present application is to provide a method for automatically and accurately adding a physical mark to a paper document that is being transmitted at high speed on a production line by quickly and accurately detecting the rotation angle of the paper document relative to the reference orientation of the reference paper document. Through the automatic detection and calculation of this rotation angle, accurate mark addition is achieved for the paper document that is being transmitted at high speed on the production line, thereby improving the production speed of the paper document and the standardization degree of the additional marks on the paper document.

[0050] Figure 1Schematic diagram of a system for automatically adding physical marks to paper documents according to an embodiment of the present application. The system for automatically adding physical marks to paper documents includes a device for detecting the rotation angle of a paper document to add physical marks to the paper document. Figure 1 An exemplary scenario is shown in which the method of detecting the rotation angle of a paper document according to the present application can be applied to add physical marks to the paper document.

[0051] As Figure 1 shown, the system 100 for automatically adding physical marks to paper documents includes a camera 110, a mark adding device 120, a rotation driving device 130 connected to the mark adding device 120, and a controller (not shown in the figure). The controller is connected to the camera 110 and the rotation driving device 130. The controller may include a processor, and may also include an internal memory or may be connected to an external memory, and a reference image obtained by photographing a reference paper document in a predetermined orientation by the camera 110 may be pre-stored in the memory.

[0052] The predetermined orientation may be the orientation of the reference paper document when the mark adding device 120 adds a mark to the reference paper document in the default orientation. In this way, when the mark adding device 120 is rotated according to the detected and calculated rotation angle of the target paper document relative to the reference paper document, it can be ensured that the physical marks on the target paper document and the physical marks on the reference paper document are in the same position and orientation.

[0053] In Figure 1 , a plurality of target paper documents in the same batch (i.e., of the same type as the reference paper document) may be sequentially conveyed on a conveyor belt S. The conveying mode of the conveyor belt S may be preset such that when a target paper document 200 in an arbitrary unknown orientation is conveyed to a position A corresponding to the camera 110, the conveyor belt stops. At this time, the camera 110 photographs the target paper document 200 to generate a target image. The controller obtains the target image from the camera 110 and obtains the reference image from the memory to perform processing such as extraction, matching, and angle calculation of image feature points. Then, the controller can control the rotation driving device 130 to rotate according to the calculated rotation angle, so as to rotate the mark adding device 120 accordingly. Subsequently, the conveyor belt S continues to move until the target paper document 200 is conveyed to a position B corresponding to the mark adding device 120. After the conveyor belt S stops at the position B, the mark adding device 120 may be lowered to be close to or even in contact with the target paper document 200, and / or the light source in the mark adding device 120 may be activated to emit light to drive the mark adding device 120 to add marks to the target paper document 200.

[0054] In this application, the reference image does not have to be the entire original image of the photographed reference paper document, but can be a part of the original image (this part does not have to include the marker addition area), specifically, it is a part of the original image that contains obvious features (such as lines, corner points, arcs, textures, etc.) and is easy to extract feature points. For example, the reference image includes a part of the reference paper document that has a relatively obvious difference from its neighborhood. In other words, the reference image includes an area on the reference paper document with high-contrast features. This area can have a gradient change greater than a preset threshold to facilitate the accurate detection of feature points. In this way, it is not necessary to extract feature points from the entire photographed original image, but only need to intercept a predetermined part of the photographed image as the reference image, and then extract the feature points of the reference image. In this way, while ensuring the recognition accuracy, the computational amount of subsequent feature point extraction and matching can be significantly reduced, the speed of feature point extraction and matching can be increased, and the processing time can be reduced, so as to be able to add physical markers to the paper documents continuously conveyed on the production line in real time.

[0055] Figure 2 is a method for detecting the rotation angle of a paper document to add a physical marker to the paper document according to an embodiment of the present application. The method for detecting the rotation angle of a paper document to add a physical marker to the paper document can be executed by the above-mentioned device for detecting the rotation angle of a paper document to add a physical marker to the paper document (for example, the processor in the device for detecting the rotation angle of a paper document to add a physical marker to the paper document). The method for detecting the rotation angle of a paper document to add a physical marker to the paper document may include the following steps:

[0056] S100: Obtain a reference image and a target image. The reference image is obtained by photographing a reference paper document in a predetermined orientation, and the target image is obtained by photographing a target paper document to be marked in any orientation. The rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document. Regular patterns and / or texts are provided on the surfaces of the reference paper document and the target paper document. [[ID=�]]

[0057] S200: Extract a set of reference feature points of the reference feature points from the reference image, and extract a set of target feature points of the target feature points from the target image.

[0058] In this application, the scale-invariant feature transform (SIFT) algorithm can be executed to extract the respective sets of feature points from the reference image and the target image. By using the scale-invariant feature transform algorithm, the descriptors of the extracted sets of feature points can have rotation and illumination invariance, so it is particularly suitable for adding physical markers to paper documents on the production line.

[0059] For example, the Scale-Invariant Feature Transform (SIFT) algorithm can be used to extract significant feature points in the reference image and the target image respectively as key points, so as to form a reference feature point set and a target feature point set. Then, a gradient direction descriptor (such as a 128-dimensional vector) of the neighborhood of each feature point is constructed to generate a reference feature descriptor set and a target feature descriptor set with rotation, scale, and illumination invariance.

[0060] Specifically, the obtained reference image and target image can be respectively subjected to grayscale transformation to generate corresponding reference grayscale images and target grayscale images. Then, difference-of-Gaussian pyramids can be constructed for the reference grayscale image and the target grayscale image respectively to generate a multi-scale space. In this multi-scale space, local extreme points in the three-dimensional space (x, y, scale) are detected through a Gaussian differential function, and candidate interest points with scale and rotation invariance are identified. Fine positioning is performed on the candidate interest points, and their exact positions and scale parameters are determined through a fitting model (such as a quadratic function model). Based on a contrast threshold, candidate interest points whose stability meets the preset conditions are filtered out to generate a reference image key point set corresponding to the reference image (as the reference feature point set) and a target image key point set corresponding to the target image (as the target feature point set). For each key point in the reference image key point set and the target image key point set, based on the local gradient direction of the image in its neighborhood, a gradient direction histogram is calculated, and a direction parameter is assigned to this key point based on this gradient direction histogram. For example, the main gradient direction and secondary gradient directions that meet the amplitude ratio threshold can be assigned as the direction parameter of this key point to achieve single-direction or multi-direction assignment. Thus, a reference image key point set and a target image key point set can be output, and the position, direction, and scale information of each key point in these sets have been determined.

[0061] Furthermore, after the direction parameter is assigned to the key point, a rotation transformation can be performed on the neighborhood of the key point according to the direction parameter to normalize its direction. According to the scale parameter of this key point, the size of the neighborhood window is adjusted to normalize its scale. The position parameter of this key point is used as the origin of the local coordinate system, and a translation transformation is performed. Through the above transformations, subsequent descriptor calculation and feature matching are both based on the normalized neighborhood, thus being invariant to rotation, scale, and translation.

[0062] S300: Extract corresponding reference feature descriptors from each reference feature point in the reference feature point set to obtain a reference feature descriptor set corresponding to the reference image, and extract corresponding target feature descriptors from each target feature point in the target feature point set to obtain a target feature descriptor set corresponding to the target image.

[0063] Each feature point in the reference feature point set and the target feature point set obtained in S200 has position, orientation, and scale information. For each feature point (i.e., key point) in the obtained reference feature point set (i.e., the reference image key point set) and the target feature point set (i.e., the target image key point set), a corresponding feature point descriptor can be calculated based on the neighborhood gradient information of the feature point. The neighborhood gradient information includes the gradient distribution in the neighborhood, such as the histogram of gradient directions. The following steps can be performed to calculate the descriptor of the feature point: Based on the neighborhood of the feature point (e.g., the neighborhood after scale and orientation normalization), divide the neighborhood into multiple sub-regions; perform statistical modeling on the image gradient direction distribution within each sub-region to generate a local feature vector; combine the local feature vectors in the order of the sub-regions to form the global feature descriptor of the feature point. Thus, a reference feature descriptor set corresponding to the reference image and a target feature descriptor set corresponding to the target image can be obtained. For example, the neighborhood can be divided into 4×4 sub-regions, and the gradient direction distribution of each sub-region is statistically modeled through the histogram of gradient directions, and the global feature descriptor obtained in this way is a 128-dimensional floating-point vector.

[0064] S400: Through approximate nearest neighbor search, generate multiple initial descriptor matching pairs between the reference feature descriptor set and the target feature descriptor set, and obtain the final matching pair set through multiple screenings. The final matching pair set includes multiple feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other.

[0065] Approximate nearest neighbor search (ANNS) refers to an algorithm for quickly finding a set of candidate vectors with high similarity to the target vector in a high-dimensional vector space. Compared with the traditional nearest neighbor search algorithm, the approximate nearest neighbor search algorithm can pre-build a "quick retrieval directory" of feature data and only compare some of the most likely matching candidates, which can increase the search speed by dozens to hundreds of times on the premise of ensuring that most correct matches are found.

[0066] S400 can use the Fast Library for Approximate Nearest Neighbors (FLANN) for matching and combine the Random Sample Consensus (RANSAC) algorithm to eliminate the interference of outliers, so as to achieve accurate matching between feature points. The Fast Library for Approximate Nearest Neighbors (FLANN) is an open-source toolkit designed for efficiently performing approximate nearest neighbor search. It supports the automatic selection and parameter optimization of various index structures such as KD-Tree and K-means tree, and can adaptively construct the optimal retrieval model according to the distribution characteristics of the input data.

[0067] In the present application, as a preferred embodiment, the multiple screening includes: a first screening based on a comparison between the matching distance of each initial descriptor matching pair and the minimum matching distance of the initial descriptor matching pairs, and a second screening based on the Random Sample Consensus algorithm with a preset maximum reprojection error threshold.

[0068] Figure 3 is a flowchart showing Figure 2 the detailed steps of step S400 shown. As Figure 3 shown, S400 may include the following steps:

[0069] S410: Generate an initial descriptor matching pair set between a reference feature descriptor set and a target feature descriptor set through an approximate nearest neighbor search algorithm. The initial descriptor matching pair set includes a plurality of initial descriptor matching pairs, and each initial descriptor matching pair includes a matched reference feature descriptor and a target feature descriptor. This process can be a fast rough matching using a fast approximate nearest neighbor search library. Due to the high similarity of paper documents, there will be a certain false matching rate in this rough matching.

[0070] For example, a FLANN matcher can be trained, and the training process can be optimized using the FlannBased matcher. Then, use the trained FLANN matcher to build an index tree for the reference feature descriptor set. The fast approximate nearest neighbor search library organizes the feature descriptors in the reference feature descriptor set by creating layer upon layer of tree structures, so that when searching for the nearest neighbor, irrelevant data points can be quickly skipped, thereby greatly reducing the search time. Then, the matching parameters of the fast approximate nearest neighbor search library can be set, including the type of search algorithm, such as KD tree or hierarchical clustering, and other optimization parameters during matching, such as the number of trees, the number of checks, the matching distance threshold, etc. These parameters can affect the speed and accuracy of the matching. Next, use the established index tree to match each target feature descriptor in the target feature descriptor set through the fast approximate nearest neighbor search library to find the reference feature descriptor closest to it. Due to the high efficiency of the fast approximate nearest neighbor search library, even when dealing with a large amount of data, the search speed can be maintained within a reasonable range. Combine the target feature descriptor with the reference feature descriptor to form an initial descriptor matching pair, and obtain the initial descriptor matching pair set between the reference feature descriptor set and the target feature descriptor set.

[0071] S420: Calculate the matching distance of each initial descriptor matching pair, and determine the minimum value among the matching distances of the initial descriptor matching pairs in the initial descriptor matching pair set as the minimum matching distance.

[0072] The matching distance of an initial descriptor matching pair refers to the similarity distance between the reference feature descriptor and the target feature descriptor in the matching pair. This distance measures the similarity between two feature descriptors and can be the Euclidean distance.

[0073] Further, S420 may further include: after calculating the matching distance of each initial descriptor matching pair, obtaining a predetermined first reference value and a second reference value, where the first reference value is less than the second reference value; comparing the first reference value with the matching distance of each initial descriptor matching pair in the set of initial descriptor matching pairs. Wherein, in each comparison, in response to the current first reference value being less than the matching distance of the initial descriptor matching pair, updating the current first reference value with the matching distance of the initial descriptor matching pair and using the updated first reference value for the next comparison until the comparison with the matching distances of all initial descriptor matching pairs is completed; comparing the second reference value with the matching distance of each initial descriptor matching pair in the set of initial descriptor matching pairs. Wherein, in each comparison, in response to the current second reference value being greater than the matching distance of the initial descriptor matching pair, updating the current second reference value with the matching distance of the initial descriptor matching pair and using the updated second reference value for the next comparison until the comparison with the matching distances of all initial descriptor matching pairs is completed; comparing the first reference value after the comparison is completed with the second reference value after the comparison is completed; and in response to the first reference value after the comparison is completed being greater than the second reference value after the comparison is completed, using the second reference value after the comparison is completed as the minimum matching distance.

[0074] Compared with simply traversing the matching distances of all initial descriptor matching pairs in the set of initial descriptor matching pairs using one reference value to find the minimum matching distance, through the above steps, it can be ensured that the set of initial descriptor matching pairs traversed is a non-empty valid set, that is, the effectiveness of the traversal process is ensured, thereby ensuring the robustness of the calculated minimum matching distance.

[0075] S430: Compare the matching distance of each initial descriptor matching pair with the minimum matching distance, and perform a first screening on multiple initial descriptor matching pairs based on the comparison result to obtain an intermediate descriptor matching pair set, where the intermediate descriptor matching pair set includes multiple intermediate descriptor matching pairs selected from multiple initial descriptor matching pairs.

[0076] Specifically, the matching distance of each initial descriptor matching pair can be compared with a multiple (for example, 2 times) of the minimum matching distance. If the matching distance is less than 2 times the minimum matching distance, it means that this initial descriptor matching pair represents the descriptors of unique and non-repeating feature points in two images, so such an initial descriptor matching pair is retained as an intermediate descriptor matching pair.

[0077] Through the first screening by comparing the matching distance based on the matching pairs with multiples (especially 2 times) of the minimum matching distance, in the scenario of paper documents with highly similar features (such as certificates of conformity), low-quality matching points can be quickly eliminated, such as mis-matches caused by repeated patterns (such as intersections of text strokes, intersections of table lines, etc.). Moreover, the first screening in S430 can reduce the number of matching pairs for subsequent RANSAC processing and improve the overall processing speed.

[0078] S440: Identify multiple intermediate feature point matching pairs corresponding to multiple intermediate descriptor matching pairs. Each intermediate feature point matching pair includes a matched reference feature point and a target feature point. For the multiple intermediate feature point matching pairs, using calculation parameters including a preset maximum reprojection error threshold, obtain an affine transformation model that best represents the transformation relationship between the reference feature point and the target feature point through the random sample consensus algorithm, and based on this affine transformation model, perform a second screening on the multiple intermediate feature point matching pairs to obtain a final matching pair set including multiple feature point matching pairs.

[0079] The affine transformation model is a mathematical model that describes the geometric deformation relationship in a two-dimensional or three-dimensional space, and its mathematical expression is:

[0080] X' = A·X + b

[0081] Where X is the coordinate vector of the original feature point, X' is the coordinate vector of the target feature point, A is a linear transformation matrix (covering operations such as rotation, scaling, shearing, etc.), and b is a translation vector. The core feature of this affine transformation model is to maintain the parallelism of geometric bodies and the invariance of line ratio, that is, parallel lines remain parallel after transformation, and the relative position ratio of points on a line remains unchanged.

[0082] In this application, S440 may further include:

[0083] S441: Use the Random Sample Consensus algorithm (RANSAC) to obtain an affine transformation model that best represents the transformation relationship between the reference feature points and the target feature points. Specifically, several intermediate feature point matching pairs (the number can be preset, for example, 2, 3, 4, etc.) can be randomly selected from multiple intermediate feature point matching pairs as a group of initial samples; calculate the four affine transformation parameters representing rotation and scaling between the reference feature points and the target feature points of this group of initial samples to form an initial affine transformation model with four degrees of freedom; use this initial affine transformation model to transform all the remaining intermediate feature point matching pairs. For example, project the reference feature points in the remaining intermediate feature point matching pairs through the calculated initial affine transformation model, and calculate the distance between the position of the projected reference feature points and the position of the actual corresponding target feature points. This distance is used as the reprojection error; compare the calculated distance with a preset maximum reprojection error threshold (for example, the maximum reprojection error threshold is set to 0.3 pixels) to evaluate whether each intermediate feature point matching pair meets this maximum reprojection error threshold; mark the intermediate feature point matching pairs that meet the reprojection error threshold as inliers, and mark the remaining intermediate feature point matching pairs as outliers. Based on the proportion of inliers and the preset confidence level, calculate the actual number of iterations required to ensure that the estimation of the affine transformation model reaches the required reliability and stability. For example, if the proportion of inliers increases, the number of iterations may decrease; if the proportion of inliers is lower than expected, the number of iterations may need to increase. Then, repeat the above process for multiple iterations. Each iteration randomly selects different samples to recalculate the initial affine transformation model and evaluate the adaptability of the model and the proportion of inliers until the preset maximum number of iterations is reached or the proportion of inliers meets the confidence level. For example, the preset maximum number of iterations can be 200 times, and the preset confidence level can be 0.99. Next, among all the initial affine transformation models generated by the iterations, select the affine transformation model with the largest number of inliers and the reprojection error meeting the threshold requirements as the affine transformation model for fine optimization. This affine transformation model is the affine transformation model that best represents the transformation relationship between the reference feature points and the target feature points.

[0084] S442: Further improve the accuracy of the affine transformation model through a fine iterative optimization strategy. Specifically, all the feature point matching pairs determined as inliers in S441 can be used to finely tune the parameters of the affine transformation model to reduce the error of the model. In each iteration, update the model parameters by the least squares method to minimize the residuals of all inliers, that is, the reprojection error of the matching pairs. Perform multiple iterations (for example, the maximum number of fine iterations is set to 10 times) until the model parameters reach a stable state (i.e., converge).

[0085] In this application, in addition to the maximum reprojection error threshold, the calculation parameters may further include: a confidence level, a maximum number of selection iterations for the selection of an affine transformation model, and a maximum number of fine iterations for the optimization of the affine transformation model. In a preferred exemplary embodiment, the maximum reprojection error threshold is 0.3 pixels, the confidence level is 0.99, the maximum number of selection iterations is 200, and the maximum number of fine iterations is 10.

[0086] In this application, the setting of the above calculation parameters is closely related to the scenario of paper documents (such as certificates of conformity) with highly similar features and uneven illumination on the production line. For example, the preset maximum reprojection error threshold of 0.3 pixels covers the positional deviation of physical marks allowed on the paper document and slight deformation of the document (such as paper). The confidence level of 0.99 ensures that a full set of inlier samples can be sampled to cope with high-noise scenarios. The maximum number of selection iterations of 200 is sufficient to avoid timeouts caused by an extremely low inlier ratio. The maximum number of fine iterations of 10 can meet the dual requirements of processing speed and marking addition accuracy for the scenario of adding marks to paper documents (such as certificates of conformity). Therefore, by setting the above calculation parameters, the quality of the final set of matching pairs and the robustness of the algorithm can be ensured, thereby ensuring the consistency and reliability of the random sample consensus algorithm when processing images of different paper documents.

[0087] S443: Based on the optimized affine transformation model, perform a second screening to obtain the final set of matching pairs. Specifically, according to the optimized affine transformation model, re-project the reference feature points in all the multiple intermediate feature point matching pairs through the optimized affine transformation model, and evaluate the reprojection error of all intermediate feature point matching pairs based on the re-projected positions; screen out the feature point matching pairs with a reprojection error less than the maximum reprojection error threshold from the multiple intermediate feature point matching pairs as the final multiple feature point matching pairs, so as to obtain the final set of matching pairs including the multiple feature point matching pairs. The position (i.e., planar coordinates) information of the final multiple feature point matching pairs is used for subsequent rotation angle calculation.

[0088] The above second screening based on the random sample consensus algorithm can further filter out incorrect matches caused by local deformation, illumination change, or noise of paper documents (such as certificates of conformity) on the production line.

[0089] Through the synergistic effect of rough matching using the fast approximate nearest neighbor search library, the first screening based on matching distance comparison, and the second screening based on the random sample consensus algorithm, in the scenario of paper documents (such as certificates of conformity) on the production line, the method according to this application can achieve real-time and accurate detection of the rotation angle of the document, and thus is applicable to the automatic addition of document marks.

[0090] S500: Calculate a transformation matrix representing the geometric transformation relationship for transforming a reference feature point in each of a plurality of feature point matching pairs to a matching target feature point. The transformation matrix includes rotation and scaling parts. The transformation matrix may also include a translation part.

[0091] In the present application, the transformation matrix may be a 2×3 affine transformation matrix, which may be represented in the following form:

[0092]

[0093] Wherein, the elements a11, a21, a12, and a22 describe rotation and scaling, while the elements b1 and b2 describe translation.

[0094] When the final set of matching pairs includes N feature point matching pairs, the coordinates (i.e., positions on the plane) of the reference feature point in the i-th feature point matching pair may be xi and yi, and the coordinates of the target feature point in the i-th feature point matching pair may be ui and vi, where both i and N are positive integers and 1 ≤ i ≤ N.

[0095] At this time, S500 may include:

[0096] S510: For the i-th feature point matching pair, construct the following equations:

[0097] ui = a11 × xi + a12 × yi + b1

[0098] vi = a21 × xi + a22 × yi + b2;

[0099] S520: Combine the equations for all N feature point matching pairs and use the least squares method to solve the combined equations to obtain the values of each element in the affine transformation matrix.

[0100] Specifically, these equations for all N feature point matching pairs can be combined into an overdetermined linear system of equations, and the overdetermined linear system of equations is solved using the least squares method.

[0101] S600: Extract the rotation and scaling parts from the transformation matrix and calculate the value of the rotation angle from the rotation and scaling parts.

[0102] In the present application, when the transformation matrix is a 2×3 affine transformation matrix, the rotation and scaling parts are the first two rows and the first two columns of the matrix.

[0103] Figure 4 is a flowchart showing Figure 2 the detailed steps of step S600 shown. As Figure 4As shown, S600 may include: S610: Extract the rotation and scaling parts in the transformation matrix as the rotation matrix; S620: Normalize the rotation matrix to obtain the normalized rotation matrix; S630: Use the arccosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix; S640: Determine the quadrant where the rotation angle is located according to the positive or negative signs of the element values of the normalized rotation matrix.

[0104] S640 may further include: When the value of element [0, 0] of the normalized rotation matrix is greater than 0, the value of element [1, 1] is greater than 0, the value of element [0, 1] is less than 0, and the value of element [1, 0] is greater than 0, it is determined that the rotation angle is in the first quadrant; When the value of element [0, 0] of the normalized rotation matrix is less than 0, the value of element [1, 1] is less than 0, the value of element [0, 1] is less than 0, and the value of element [1, 0] is greater than 0, it is determined that the rotation angle is in the second quadrant; When the value of element [0, 0] of the normalized rotation matrix is less than 0, the value of element [1, 1] is less than 0, the value of element [0, 1] is greater than 0, and the value of element [1, 0] is less than 0, it is determined that the rotation angle is in the third quadrant; And when the values of the elements of the normalized rotation matrix do not belong to any of the listed cases, it is determined that the rotation angle is in the fourth quadrant.

[0105] Thus, by using the calculation of S600, the radian value of the rotation angle and the quadrant where the rotation angle is located can be obtained. Then, the rotation drive device 130 can be controlled to rotate according to the radian value and the quadrant, so that the marking addition device 120 is aligned with the target paper document 200 to add physical marks.

[0106] The quadrant where the rotation angle is located can represent the rotation direction. At this time, there are two control methods. The first is to fix the rotation direction (clockwise or counterclockwise) of the rotation drive device 130, and calculate the rotation angle α of the rotation drive device 130 according to the quadrant where the rotation angle is located and the radian value r. When the rotation angle is in the first or second quadrant, α = r×180° / π; When the rotation angle is in the third or fourth quadrant, α = 360° - r×180° / π. The second is to directly determine the absolute value of the rotation angle α according to the radian value r, that is, α = r×180° / π. Then, according to the quadrant where the rotation angle is located, the rotation drive device is rotated clockwise or counterclockwise. For example, it is rotated clockwise when the rotation angle is in the first or second quadrant, and rotated counterclockwise when the rotation angle is in the third or fourth quadrant.

[0107] In addition, the present application also provides a device for detecting the rotation angle of a paper document to add physical marks to the paper document. Figure 5It is a block diagram of a device for detecting the rotation angle of a paper document to add a physical mark according to an embodiment of the present application.

[0108] As Figure 5 shown, the device 300 for detecting the rotation angle of a paper document to add a physical mark to the paper document includes: an acquisition module 310, configured to acquire a reference image and a target image, the reference image being obtained by photographing a reference paper document in a predetermined orientation, the target image being obtained by photographing a target paper document to be marked in an arbitrary orientation, the rotation angle being the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document, and the surfaces of the reference paper document and the target paper document having regular patterns and / or texts; a first extraction module 320, configured to extract a set of reference feature points of reference feature points from the reference image and a set of target feature points of target feature points from the target image; a second extraction module 330, configured to extract a corresponding reference feature descriptor from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extract a corresponding target feature descriptor from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image; a matching module 340, configured to generate a plurality of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through approximate nearest neighbor search, and obtain a final matching pair set through multiple screenings, the final matching pair set including a plurality of feature point matching pairs, each feature point matching pair including a reference feature point and a target feature point that match each other; and a calculation module 350, configured to: calculate a transformation matrix representing the geometric transformation relationship from the reference feature point in each feature point matching pair in the final matching pair set to the matching target feature point, the transformation matrix including a rotation and a scaling part; extract the rotation and scaling parts from the transformation matrix, and calculate the value of the rotation angle from the rotation and scaling parts; wherein, the multiple screenings include: a first screening based on the comparison of the matching distance of each initial descriptor matching pair with the minimum matching distance among the plurality of initial descriptor matching pairs, and a second screening based on the random sample consensus algorithm with a preset maximum reprojection error threshold.

[0109] Thus, the device 300 for detecting the rotation angle of a paper document to add a physical mark according to the present application can realize the detection of the rotation angle between the target paper document and the reference paper document. In particular, for paper documents on a production line with highly similar features, high real-time requirements, and being affected by light interference, the cascade optimization strategy of multiple screenings can solve the problem of false matching and does not require time-consuming processing, thereby realizing high-precision rotation angle detection and fast processing capabilities.

[0110] The matching module 340 further includes: an initial matching module configured to generate an initial set of descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through an approximate nearest neighbor search algorithm, the initial set of descriptor matching pairs including a plurality of initial descriptor matching pairs, each initial descriptor matching pair including a matched reference feature descriptor and a target feature descriptor; a first screening module configured to: calculate the matching distance of each initial descriptor matching pair, and determine the minimum value among the plurality of matching distances of the plurality of initial descriptor matching pairs as the minimum matching distance; compare the matching distance of each initial descriptor matching pair with the minimum matching distance, and perform a first screening on the plurality of initial descriptor matching pairs based on the comparison result to obtain an intermediate set of descriptor matching pairs, the intermediate set of descriptor matching pairs including a plurality of intermediate descriptor matching pairs selected from the plurality of initial descriptor matching pairs; and a second screening module configured to: identify a plurality of intermediate feature point matching pairs corresponding to the plurality of intermediate descriptor matching pairs, each intermediate feature point matching pair including a matched reference feature point and a target feature point, for the plurality of intermediate feature point matching pairs, use calculation parameters including a preset maximum reprojection error threshold, and obtain an affine transformation model that best represents the transformation relationship between the reference feature points and the target feature points through the random sample consensus algorithm, and perform a second screening on the plurality of intermediate feature point matching pairs based on the affine transformation model to obtain a final set of matching pairs including a plurality of feature point matching pairs.

[0111] The calculation module 350 is further configured to: extract the rotation and scaling parts in the transformation matrix as the rotation matrix; normalize the rotation matrix to obtain a normalized rotation matrix; use the arccosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix; and determine the quadrant in which the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix.

[0112] The foregoing Figures 2 to 4 The method for detecting the rotation angle of a paper document to add a physical mark to the paper document described above can be executed by the device 300 for detecting the rotation angle of a paper document to add a physical mark to the paper document. The device 300 for detecting the rotation angle of a paper document to add a physical mark to the paper document can achieve the steps and effects of the above method, and thus will not be described in detail herein.

[0113] The present application further provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the processor is caused to execute the above method for detecting the rotation angle of a paper document to add a physical mark to the paper document.

[0114] The present application also provides a computer program product, including instructions which, when executed by a processor, cause the processor to execute the method for detecting the rotation angle of a paper document to add a physical mark to the paper document as described above.

[0115] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprise" and / or "include" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0116] It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The nouns and pronouns related to people in this patent application are not limited to a specific gender.

[0117] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting the rotation angle of a paper document to add a physical mark to the paper document, characterized in that, The method for detecting the rotation angle of a paper document to add a physical mark to the paper document includes: Obtaining a reference image and a target image, where the reference image is obtained by photographing a reference paper document in a predetermined orientation, the target image is obtained by photographing a target paper document to be marked in any orientation, the rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document, and regular patterns and / or texts are provided on the surfaces of the reference paper document and the target paper document (S100); Extracting a set of reference feature points of reference feature points from the reference image, and extracting a set of target feature points of target feature points from the target image (S200); Extracting corresponding reference feature descriptors from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extracting corresponding target feature descriptors from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image (S300); Generating a plurality of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through approximate nearest neighbor search, and obtaining a final matching pair set through multiple screenings. The final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other (S400); Calculating a transformation matrix representing the geometric transformation relationship of transforming the reference feature point in each feature point matching pair among the plurality of feature point matching pairs to the matching target feature point, where the transformation matrix includes a rotation and a scaling part (S500); and Extracting the rotation and scaling parts in the transformation matrix, and calculating the value of the rotation angle from the rotation and scaling parts (S600); Wherein, the extracting the rotation and scaling parts in the transformation matrix and calculating the value of the rotation angle from the rotation and scaling parts (S600) includes: Extracting the rotation and scaling parts in the transformation matrix as a rotation matrix (S610); Normalizing the rotation matrix to obtain a normalized rotation matrix (S620); Using the inverse cosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix (S630); and Determining the quadrant where the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix (S640).

2. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 1, wherein, The multiple screenings include: a first screening based on the comparison between the matching distance of each initial descriptor matching pair and the minimum matching distance of the initial descriptor matching pair, and a second screening based on the random sample consensus algorithm with a preset maximum reprojection error threshold.

3. The method for detecting the rotation angle of a paper document and adding a physical mark to the paper document according to claim 2, wherein, Generating a plurality of initial descriptor matching pairs between the reference feature descriptor set and the target feature descriptor set through approximate nearest neighbor search, and obtaining a final matching pair set through multiple screenings. The final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other (S400) includes: Generating the initial descriptor matching pair set between the reference feature descriptor set and the target feature descriptor set through an approximate nearest neighbor search algorithm. The initial descriptor matching pair set includes the plurality of initial descriptor matching pairs, and each initial descriptor matching pair includes a matching reference feature descriptor and a target feature descriptor (S410); Calculating the matching distance of each initial descriptor matching pair, and determining the minimum value among the matching distances as the minimum matching distance (S420); Comparing the matching distance of each initial descriptor matching pair with the minimum matching distance, and performing the first screening on the plurality of initial descriptor matching pairs based on the comparison result to obtain an intermediate descriptor matching pair set. The intermediate descriptor matching pair set includes a plurality of intermediate descriptor matching pairs selected from the plurality of initial descriptor matching pairs (S430); Identifying a plurality of intermediate feature point matching pairs corresponding to the plurality of intermediate descriptor matching pairs. Each intermediate feature point matching pair includes a matching reference feature point and a target feature point. For the plurality of intermediate feature point matching pairs, using calculation parameters including the preset maximum reprojection error threshold, obtaining an affine transformation model that best represents the transformation relationship between the reference feature point and the target feature point through the random sample consensus algorithm, and performing the second screening on the plurality of intermediate feature point matching pairs based on the affine transformation model to obtain the final matching pair set including the plurality of feature point matching pairs (S440).

4. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 3, wherein, The calculation parameters further include: a confidence level, a maximum selection iteration number for the selection of the affine transformation model, and a maximum refinement iteration number for the optimization of the affine transformation model, and The maximum reprojection error threshold is 0.3 pixels, the confidence level is 0.99, the maximum selection iteration number is 200, and the maximum refinement iteration number is 10.

5. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 3 or 4, characterized in that, Determining the minimum value among the matching distances as the minimum matching distance further includes: Obtaining a predetermined first reference value and a second reference value, where the first reference value is less than the second reference value; Compare the first reference value with the matching distances of each of the initial descriptor matching pairs in the set of initial descriptor matching pairs, wherein, in each comparison, in response to the current first reference value being less than the matching distance of the initial descriptor matching pair, update the current first reference value with the matching distance of the initial descriptor matching pair, and use the updated first reference value for the next comparison until the comparison with the matching distances of all the initial descriptor matching pairs is completed; Compare the second reference value with the matching distances of each of the initial descriptor matching pairs in the set of initial descriptor matching pairs, wherein, in each comparison, in response to the current second reference value being greater than the matching distance of the initial descriptor matching pair, update the current second reference value with the matching distance of the initial descriptor matching pair, and use the updated second reference value for the next comparison until the comparison with the matching distances of all the initial descriptor matching pairs is completed; Compare the first reference value after the comparison is completed with the second reference value after the comparison is completed; and In response to the first reference value after the comparison is completed being greater than the second reference value after the comparison is completed, use the second reference value after the comparison is completed as the minimum matching distance.

6. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 1, characterized in that, The normalized rotation matrix is a 2×2 matrix, and determining the quadrant in which the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix (S640) includes: When the value of the element [0, 0] of the normalized rotation matrix is greater than 0, the value of the element [1, 1] is greater than 0, the value of the element [0, 1] is less than 0, and the value of the element [1, 0] is greater than 0, determine that the rotation angle is in the first quadrant; When the value of the element [0, 0] of the normalized rotation matrix is less than 0, the value of the element [1, 1] is less than 0, the value of the element [0, 1] is less than 0, and the value of the element [1, 0] is greater than 0, determine that the rotation angle is in the second quadrant; When the value of the element [0, 0] of the normalized rotation matrix is less than 0, the value of the element [1, 1] is less than 0, the value of the element [0, 1] is greater than 0, and the value of the element [1, 0] is less than 0, determine that the rotation angle is in the third quadrant; and When the values of the elements of the normalized rotation matrix do not belong to any of the listed cases, determine that the rotation angle is in the fourth quadrant.

7. The method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to any one of claims 1 to 4, characterized in that, The transformation matrix is a 2×3 affine transformation matrix, and the affine transformation matrix is represented in the following form: Among them, elements a11, a21, a12, and a22 describe rotation and scaling, while elements b1 and b2 describe translation. Among them, the final matching pair set includes N feature point matching pairs. The coordinates of the reference feature point in the i-th feature point matching pair are xi and yi , and the coordinates of the target feature point in the i-th feature point matching pair are ui and vi , where both i and N are positive integers and 1 ≤ i ≤ N. wherein, calculating the transformation matrix includes: For the i-th feature point matching pair, construct the following equation: Combine the equations for all N feature point matching pairs, and use the least squares method to solve the combined equations to obtain the values of each element in the affine transformation matrix.

8. A device for detecting the rotation angle of a paper document to add a physical mark to the paper document, characterized in that, The device for detecting the rotation angle of a paper document to add a physical mark to the paper document includes: An acquisition module (310) for acquiring a reference image and a target image, where the reference image is obtained by photographing a reference paper document in a predetermined orientation, the target image is obtained by photographing a target paper document with a mark to be added in an arbitrary orientation, the rotation angle is the angle between the orientation of the target paper document and the predetermined orientation of the reference paper document, and the surfaces of the reference paper document and the target paper document have regular patterns and / or texts; A first extraction module (320) configured to extract a set of reference feature points of reference feature points from the reference image and a set of target feature points of target feature points from the target image; A second extraction module (330) configured to extract corresponding reference feature descriptors from each reference feature point in the set of reference feature points to obtain a set of reference feature descriptors corresponding to the reference image, and extract corresponding target feature descriptors from each target feature point in the set of target feature points to obtain a set of target feature descriptors corresponding to the target image; A matching module (340) configured to generate a plurality of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through approximate nearest neighbor search, and obtain a final matching pair set through multiple screenings, the final matching pair set includes a plurality of feature point matching pairs, and each feature point matching pair includes a reference feature point and a target feature point that match each other; and A calculation module (350) configured to: calculate a transformation matrix representing the geometric transformation relationship from the reference feature point in each feature point matching pair in the final matching pair set to the matched target feature point, the transformation matrix includes a rotation and a scaling part; extract the rotation and scaling parts from the transformation matrix, and calculate the value of the rotation angle from the rotation and scaling parts; The calculation module (350) is further configured to: Extract the rotation and scaling parts from the transformation matrix as a rotation matrix; Normalize the rotation matrix to obtain a normalized rotation matrix; Use the inverse cosine function to calculate the radian value of the rotation angle from the element values of the normalized rotation matrix; and Determine the quadrant where the rotation angle is located according to the positive and negative signs of the element values of the normalized rotation matrix.

9. The device for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 8, wherein, The multiple screenings include: a first screening based on the comparison of the matching distance of each initial descriptor matching pair with the minimum matching distance among the multiple initial descriptor matching pairs, and a second screening based on the random sample consensus algorithm with a preset maximum reprojection error threshold.

10. The device for detecting the rotation angle of a paper document to add a physical mark to the paper document according to claim 9, wherein The matching module (340) includes: An initial matching module configured to generate the set of initial descriptor matching pairs between the set of reference feature descriptors and the set of target feature descriptors through an approximate nearest neighbor search algorithm, the set of initial descriptor matching pairs includes the plurality of initial descriptor matching pairs, and each initial descriptor matching pair includes a matched reference feature descriptor and a target feature descriptor; The first screening module is configured to: calculate the matching distance of each of the initial descriptor matching pairs, and determine the minimum value among the multiple matching distances of the multiple initial descriptor matching pairs as the minimum matching distance; compare the matching distance of each of the initial descriptor matching pairs with the minimum matching distance, and perform the first screening on the multiple initial descriptor matching pairs based on the comparison result to obtain a set of intermediate descriptor matching pairs, where the set of intermediate descriptor matching pairs includes multiple intermediate descriptor matching pairs selected from the multiple initial descriptor matching pairs; and The second screening module is configured to: identify multiple intermediate feature point matching pairs corresponding to the multiple intermediate descriptor matching pairs, where each of the intermediate feature point matching pairs includes a matched reference feature point and a target feature point. For the multiple intermediate feature point matching pairs, use calculation parameters including the preset maximum reprojection error threshold, and obtain an affine transformation model that best represents the transformation relationship between the reference feature point and the target feature point through the random sample consensus algorithm, and perform the second screening on the multiple intermediate feature point matching pairs based on the affine transformation model to obtain the final matching pair set including the multiple feature point matching pairs.

11. A system for automatically adding physical markings to paper documents, characterized in that, The system (100) for automatically adding physical marks to a paper document includes: A conveyor belt (S) for conveying a target paper document (200) to which a physical mark is to be added in any orientation, and the surface of the target paper document (200) has a regular pattern and / or text; A camera (110) located above the conveyor belt (S) and configured to capture the target paper document (200) reaching below the camera (110) to generate a target image; A mark adding device (120) located above the conveyor belt (S) and arranged downstream of the camera (110); A rotation driving device (130) connected to the mark adding device (120); A controller, the controller is connected to the camera (110) and the rotation driving device (130) and is configured to: control the rotation driving device (130) to rotate according to the calculated rotation angle according to the method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to any one of claims 1 to 7, and add a physical mark to the target paper document (200) by the mark adding device (120).

12. A computer-readable storage medium, characterized in that, Instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the method for detecting the rotation angle of a paper document to add a physical mark to the paper document according to any one of claims 1 to 7.

Images