A method and device for detecting true or false signature actions for double recording quality inspection
The matching degree between the signature handwriting and the pen tip trajectory is detected through video recording and shape matching algorithms, which solves the problem of the inability to automatically detect the authenticity of signature behavior in the prior art, and realizes the automatic detection of signature behavior in financial activities.
Patent Information
- Application Number
- CN202411459776.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-10-18
AI Technical Summary
The existing technology is difficult to automatically detect the authenticity of signature behavior in financial activities, and it is impossible to effectively identify real signatures and fake signatures.
Using video recording technology, the signature pen is divided by detection and segmentation algorithm, and the nib position is positioned through the key point positioning algorithm, track the nib trajectory, detect the signature handwriting, and judge the matching degree between the signature handwriting and the nib trajectory through the shape matching algorithm, and automatically judge the Whether it is true or not.
It realizes automatic detection of signature behavior in financial activities, improves the accuracy of detection, and can effectively identify authentic signature behaviors.
Smart Images
Figure CN119027858B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of signature action detection, and in particular to a method and device for detecting true or false signature actions for double recording quality inspection. Background Art
[0002] In the process of online and offline financial activities, signatures are usually required to confirm the autonomous will of the participants. In order to determine the true will of the participants, the participants are usually required to sign and confirm, and determine the authenticity of the signature.
[0003] The false signature behaviors here include:
[0004] (1) There is only the act of picking up a pen to sign, but no handwriting of the signature;
[0005] (2) There is the act of picking up a pen to sign, and there is also handwriting of the signature, but the handwriting of the signature is signed in advance.
[0006] Therefore, there is an urgent need for an automatic detection method that can distinguish true from false signatures. Summary of the invention
[0007] In order to solve the technical problem of automatically detecting the authenticity of signatures during financial activities, the present invention provides a method and device for detecting authenticity of signatures for double recording quality inspection. The following technical solutions are adopted:
[0008] A method for detecting true and false signature actions for double recording quality inspection comprises the following steps:
[0009] Step 1: Video recording the signing process;
[0010] Step 2: When a signing action is detected, the signing pen is segmented in the current frame using a detection or segmentation algorithm, and the position of the pen tip is located using a key point positioning algorithm;
[0011] Step 3, in each frame of the video, track the position of the pen tip and obtain the trajectory of the pen tip;
[0012] Step 4, detecting the signature handwriting in each frame of the video;
[0013] Step 5, matching the signature handwriting with the trajectory of the pen tip;
[0014] Step 6, judging whether it is a true signature based on whether the ratio of the number of frames where the signature handwriting matches the pen tip trajectory to the total number of frames in the signing process exceeds a threshold value T.
[0015] By adopting the above technical solution, the signing process is first recorded, then the pen tip of the signature is tracked to obtain its trajectory, then the signature handwriting is detected on the signed document, and finally the pen tip trajectory is matched with the signature result. The signature behavior in the process of financial activities is automatically detected to be true or false with high accuracy.
[0016] The current technology mainly detects whether there is a signature through target detection or action detection, but does not judge whether it is a real signature. The proposed solution judges the signature behavior after detecting the signature behavior and automatically distinguishes true and false signature behaviors.
[0017] Optionally, in step 1, the video recording of the signing process ensures that both the signature handwriting and the pen tip are visible at the same time.
[0018] By adopting the above technical solution, the handwriting and the pen tip are simultaneously visible, providing a basis for subsequent analysis and comparison of the handwriting and the pen tip, and facilitating the use of algorithms to automatically extract and track the positions of the handwriting and the pen tip.
[0019] Optionally, in step 2, YOLO is used to simultaneously detect and segment the signing pen, or R-CNN, FastR-CNN and SSD algorithms are used to detect the position of the signing pen and perform segmentation, and the key points of the signing pen are located through regression and heat map methods to locate the position of the pen tip.
[0020] By adopting the above technical solution, the segmentation algorithm is used to segment the signature pen and locate the position of the pen tip. YOLO can be used to simultaneously detect and segment the signature pen, or algorithms such as R-CNN, FastR-CNN and SSD can be used to detect the position of the signature pen and perform segmentation. The key points of the signature pen are located by regression methods (such as DeepPose, MTCNN, MobileNet) and heat map methods (OpenPose, HRNet, ResNet) to accurately locate the position of the pen tip.
[0021] Optionally, in step 3, an optical flow tracking algorithm is used to track the pen tip to obtain the trajectory of the pen tip from the beginning of signing to the current frame.
[0022] By adopting the above technical solution, the pen tip is tracked using an optical flow tracking algorithm such as the LK optical flow method, the KLT optical flow method or the dense optical flow tracking DOT to obtain the trajectory of the pen tip from the beginning of signing to the current frame.
[0023] Optionally, in step 4, the method for obtaining handwriting of each frame of the video includes the following sub-steps:
[0024] Step 41, starting from the signature start frame to the current frame, for two adjacent frames, a handwriting search range is determined according to the pen tip position;
[0025] Step 42, calculating a frame difference image for the search range;
[0026] Step 43, performing binarization processing on the frame difference image to obtain a binary handwriting segment;
[0027] Step 44, accumulate the binary handwriting segments of all frames to obtain the binary signature handwriting from the beginning of signing to the current frame.
[0028] By adopting the above technical solution, the signature handwriting from the starting frame to the current frame can be effectively located, so as to facilitate real-time comparison with the pen tip trajectory from the starting frame to the current frame, thereby determining whether the signature handwriting matches the pen tip trajectory, that is, whether the signature handwriting is generated by the pen tip trajectory.
[0029] Optionally, in step 4, the handwriting of each frame of the video is obtained by the following method: perform handwriting detection on the current frame to detect all handwritings; obtain handwriting that overlaps with the handwriting position of the current frame; and binarize the handwriting to obtain binary signature handwriting.
[0030] By adopting the above technical solution, all handwritten signature handwriting is first detected, and then the signature handwriting that may be generated by the pen tip trajectory in the current signing behavior is selected according to the handwriting position, so as to facilitate real-time comparison with the pen tip trajectory from the start frame to the current frame, thereby determining whether the signature handwriting matches the pen tip trajectory, that is, whether the signature handwriting is generated by the pen tip trajectory.
[0031] Optionally, in step 5, the signature handwriting is shape-matched with the trajectory of the pen tip using an invariant moment method, a wavelet transform method or a shape context method shape matching algorithm.
[0032] By adopting the above technical solution, the Hu invariant moment method can be used to perform shape matching between the signature handwriting and the pen tip trajectory. The specific steps are as follows:
[0033] Step 1. Thin the binary signature handwriting. You can use the skeletonize function in the morphology submodule of the skimage library to obtain a thinned signature handwriting image.
[0034] Step 2. Extract Hu invariant moments from the pen tip trajectory image and the refined signature handwriting image respectively, and then perform shape matching and calculate the matching distance d. This can be achieved using the matchShapes function of the OpenCV library.
[0035] Step 3. According to the above steps, calculate the distance d between the signature handwriting and the pen tip trajectory, and determine whether d is less than a given threshold T1. If it is less than T1, the two are considered to match, otherwise they are considered to not match.
[0036] The Hu invariant moment is invariant to translation, scaling, and rotation, and can better distinguish the same shape from different shapes.
[0037] Optionally, in step 6, the ratio of the number of frames in which the pen tip trajectory matches the signature handwriting during the signing process to the total number of frames is counted, and when the ratio exceeds a threshold T, it is determined to be a true signature, otherwise it is determined to be a false signature.
[0038] By adopting the above technical solution, at the tth frame, the matching distance d between the pen tip trajectory and the signature handwriting is calculated, and it is determined whether d is less than the threshold value T1. If it is less than T1, it is considered that the two match in this frame. The number of frames n in which the pen trajectory and the signature handwriting match in all frames is counted. Assuming the total number of frames is N, when the ratio n / N exceeds the threshold value T, it is determined to be a true signature, otherwise it is determined to be a false signature.
[0039] A device for detecting true or false signatures for double recording quality inspection comprises a signature photographing device, a memory and a processor. The signature photographing device is used to photograph the signing process to ensure that the signature handwriting and the pen tip appear simultaneously in the video frame of the signing process. The signature photographing device communicates with the memory to exchange the video data of the signing process. The memory stores a detection program designed using a method for detecting true or false signatures for double recording quality inspection. The processor is connected to the memory for communication, inputs the video data of the signing process into the detection program, and runs the detection program to output the detection result.
[0040] Optionally, a display is also included, which is communicatively connected to the processor, and the processor controls the display to display the detection results.
[0041] In summary, the present invention includes at least the following beneficial technical effects:
[0042] The present invention can provide a method and device for detecting true and false signature actions for double recording quality inspection. First, the signing process is recorded in video, then the pen tip of the signature is tracked to obtain its trajectory, then the signature handwriting is detected on the signed document, and finally the pen tip trajectory is matched with the signature result in shape. The signature behavior in the process of financial activities is automatically detected to be true or false with high accuracy. After the signature behavior is detected, the proposed solution judges the signature behavior and automatically distinguishes true and false signature behaviors, solving the problem that the current technology only detects the signature behavior but cannot distinguish true and false signature behaviors. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a flow chart of a method for detecting true and false signature actions for double recording quality inspection of the present invention;
[0044] Figure 2 It is a schematic diagram of the pen tip movement trajectory of a specific embodiment of a method for detecting true or false signatures for double recording quality inspection of the present invention, wherein the dark black represents the pen tip trajectory;
[0045] Figure 3 It is a method for detecting true and false signature actions for double recording quality inspection of the present invention and a schematic diagram of real signature handwriting in a specific embodiment. DETAILED DESCRIPTION
[0046] The present invention is further described in detail below with reference to the accompanying drawings. The embodiment of the present invention discloses a method and device for detecting true or false signature actions for double recording quality inspection.
[0047] Reference Figure 1-Figure 3 , a method for detecting true or false signature actions for double recording quality inspection, comprising the following steps:
[0048] Step 1: Video recording the signing process;
[0049] Step 2: When a signing action is detected, the signing pen is segmented in the current frame using a detection or segmentation algorithm, and the position of the pen tip is located using a key point positioning algorithm;
[0050] Step 3, in each frame of the video, track the position of the pen tip and obtain the trajectory of the pen tip;
[0051] Step 4, detecting the signature handwriting in each frame of the video;
[0052] Step 5, matching the signature handwriting with the trajectory of the pen tip;
[0053] Step 6, judging whether it is a true signature based on whether the ratio of the number of frames where the signature handwriting matches the pen tip trajectory to the total number of frames in the signing process exceeds a threshold value T.
[0054] First, the signing process is recorded, then the pen tip is tracked to obtain its trajectory, then the signature handwriting is detected on the signed document, and finally the pen tip trajectory is matched with the signature result. The signature behavior in the process of financial activities is automatically detected with high accuracy. The current technology mainly detects whether there is a signature behavior through target detection or action detection methods, and does not determine whether it is a real signature. The proposed solution judges the signature behavior after detecting the signature behavior and automatically distinguishes true and false signature behaviors.
[0055] In step 1, the video recording of the signing process ensures that both the signature handwriting and the pen tip are visible at the same time.
[0056] The simultaneous visibility of handwriting and pen tip provides a basis for subsequent analysis and comparison of handwriting and pen tip, making it convenient to use algorithms to automatically extract and track the positions of handwriting and pen tip.
[0057] In step 2, YOLO is used to simultaneously detect and segment the signature pen, or R-CNN, FastR-CNN and SSD algorithms are used to detect the position of the signature pen and perform segmentation. The key points of the signature pen are located through regression and heat map methods to locate the position of the pen tip.
[0058] Use the segmentation algorithm to segment the signature pen and locate the position of the pen tip. You can use YOLO to detect and segment the signature pen at the same time, or use R-CNN, FastR-CNN, SSD and other algorithms to detect the position of the signature pen and perform segmentation. Use regression methods (such as DeepPose, MTCNN, MobileNet) and heat map methods (OpenPose, HRNet, ResNet) to locate the key points of the signature pen and accurately locate the position of the pen tip.
[0059] In step 3, the pen tip is tracked using an optical flow tracking algorithm to obtain the trajectory of the pen tip from the beginning of signing to the current frame.
[0060] An optical flow tracking algorithm such as the LK optical flow method, the KLT optical flow method or the dense optical flow tracking DOT is used to track the pen tip and obtain the trajectory of the pen tip from the beginning of signing to the current frame.
[0061] In step 4, the method for obtaining the handwriting of each frame of the video includes the following sub-steps:
[0062] Step 41, starting from the signature start frame to the current frame, for two adjacent frames, a handwriting search range is determined according to the pen tip position;
[0063] Step 42, calculating a frame difference image for the search range;
[0064] Step 43, performing binarization processing on the frame difference image to obtain a binary handwriting segment;
[0065] Step 44, accumulate the binary handwriting segments of all frames to obtain the binary signature handwriting from the beginning of signing to the current frame.
[0066] The signature handwriting from the start frame to the current frame can be effectively located, and is convenient for real-time comparison with the pen tip trajectory from the start frame to the current frame, so as to determine whether the signature handwriting matches the pen tip trajectory, that is, whether the signature handwriting is generated by the pen tip trajectory.
[0067] In step 4, the handwriting of each frame of the video is obtained by the following method: handwriting detection is performed on the current frame to detect all handwritings; handwriting that overlaps with the handwriting position of the current frame is obtained; and the handwriting is binarized to obtain binary signature handwriting.
[0068] First, all handwritten signatures are detected, and then the signatures that may be generated by the pen tip trajectory in the current signing behavior are selected according to the handwriting position, so as to compare them with the pen tip trajectory from the start frame to the current frame in real time, so as to determine whether the signatures match the pen tip trajectory, that is, whether the signatures are generated by the pen tip trajectory.
[0069] In step 5, the signature handwriting is matched with the trajectory of the pen tip using a shape matching algorithm such as an invariant moment method, a wavelet transform method or a shape context method.
[0070] The Hu invariant moment method can be used to match the signature handwriting and the pen tip trajectory. The specific steps are as follows:
[0071] Step 1. Thin the binary signature handwriting. You can use the skeletonize function in the morphology submodule of the skimage library to obtain a thinned signature handwriting image.
[0072] Step 2. Extract Hu invariant moments from the pen tip trajectory image and the refined signature handwriting image respectively, and then perform shape matching and calculate the matching distance d. This can be achieved using the matchShapes function of the OpenCV library.
[0073] Step 3. According to the above steps, calculate the distance d between the signature handwriting and the pen tip trajectory, and determine whether d is less than a given threshold T1. If it is less than T1, the two are considered to match, otherwise they are considered to not match.
[0074] The Hu invariant moment is invariant to translation, scaling, and rotation, and can better distinguish the same shape from different shapes.
[0075] In step 6, the ratio of the number of frames in which the pen tip trajectory matches the signature handwriting in the signing process to the total number of frames is counted. When the ratio exceeds a threshold value T, it is determined to be a true signature, otherwise it is determined to be a false signature.
[0076] At the tth frame, calculate the matching distance d between the pen tip trajectory and the signature handwriting, and determine whether d is less than the threshold T1. If it is less than T1, it is considered that the two match in this frame. Count the number of frames n in which the pen trajectory and the signature handwriting match in all frames. Assuming the total number of frames is N, when the ratio n / N exceeds the threshold T, it is judged as a true signature, otherwise it is judged as a false signature.
[0077] A device for detecting true and false signature actions for dual-recording quality inspection, comprising a signature shooting device, a memory, and a processor. The signature shooting device is used to shoot the signature process to ensure that both the signature handwriting and the pen tip can appear in the video frames of the signature process video. The signature shooting device communicates and exchanges the signature process video data with the memory. The memory stores a detection program designed by a method for detecting true and false signature actions for dual-recording quality inspection. The processor is communicatively connected to the memory, inputs the signature process video data into the detection program, and runs the detection program to output the detection result.
[0078] It further includes a display, and the display is communicatively connected to the processor, and the processor controls the display to show the detection result.
[0079] The following explains the implementation principle of a method and device for detecting true and false signature actions for dual-recording quality inspection based on embodiments:
[0080] Taking the character "Wang" as an example to illustrate how to judge true and false signature behaviors:
[0081] For the video recording of the signature process, within the shooting range, both the signature handwriting and the pen tip should be visible at the same time, as Figure 2 shown.
[0082] When a signature behavior is detected, the position of the pen tip is detected in the current frame through the YOLO algorithm and the key point positioning algorithm. This frame is also called the start signature frame. In subsequent frames, the optical flow tracking algorithm is used to obtain the position of the pen tip, and the positions of the pen tip from the start frame to the current frame are connected to obtain the trajectory of the pen tip. As Figure 2 shown in the dark black font, since the movement trajectory of the pen tip is drawn, there will be a situation of connected strokes, but at this time, it is impossible to distinguish whether it is tracing or a real signature.
[0083] As Figure 3 shown, by accumulating the frame differences near the pen tip position from the start signature frame to the current frame, the real signature handwriting from the start frame to the current frame can be obtained. The specific acquisition method is as follows:
[0084] Starting from the signature start frame and up to the current frame, for adjacent two frames, a line segment is determined according to the pen tip positions of the front and back two frames. Taking the line segment as the center line, the center line is dilated by one pixel as the search range.
[0085] Calculate the frame difference image within the search range;
[0086] Perform binary processing on the frame difference image to obtain binary handwriting fragments. Here, binary processing can be performed using a threshold, and the threshold is set to 15;
[0087] Accumulate all the binary handwriting fragments of all frames to obtain the binary signature handwriting from the start of the signature to the current frame, as Figure 3 shown;
[0088] The shape of the signature handwriting and the trajectory of the pen tip are matched using an invariant rectangle shape matching algorithm.
[0089] During the signing process, the proportion of frames in which the pen tip trajectory matches the signature handwriting is counted. If the proportion exceeds 80%, it is determined to be a true signature.
[0090] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for detecting true and false signature actions for double recording quality inspection, characterized in that: The following steps are involved: Step 1: Video recording the signing process; Step 2: When a signing action is detected, the signing pen is segmented in the current frame using a detection or segmentation algorithm, and the position of the pen tip is located using a key point positioning algorithm; Step 3, in each frame of the video, track the position of the pen tip and obtain the trajectory of the pen tip; Step 4, detecting the signature handwriting in each frame of the video; Step 5, matching the signature handwriting with the trajectory of the pen tip; Step 6, judging whether it is a true signature based on whether the ratio of the number of frames where the signature handwriting matches the pen tip trajectory to the total number of frames in the signing process exceeds a threshold value T.
2. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 1, the video recording of the signing process ensures that both the signature handwriting and the pen tip are visible at the same time.
3. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 2, the YOLO, R-CNN, FastR-CNN or SSD algorithm is used to detect the position of the signature pen and segment it, and the key points of the signature pen are located through regression method or heat map method to locate the position of the pen tip.
4. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 3, the pen tip is tracked using an optical flow tracking algorithm to obtain the trajectory of the pen tip from the beginning of signing to the current frame.
5. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 4, the method for obtaining the handwriting of each frame of the video includes the following sub-steps: Step 41, starting from the signature start frame to the current frame, for two adjacent frames, a handwriting search range is determined according to the pen tip position; Step 42, calculating a frame difference image for the search range; Step 43, performing binarization processing on the frame difference image to obtain a binary handwriting segment; Step 44, accumulate the binary handwriting segments of all frames to obtain the binary signature handwriting from the beginning of signing to the current frame.
6. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 4, the handwriting of each frame of the video is obtained by the following method: handwriting detection is performed on the current frame to detect all handwritings; handwriting that overlaps with the handwriting position of the current frame is obtained; and the handwriting is binarized to obtain binary signature handwriting.
7. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 5, the signature handwriting is matched with the trajectory of the pen tip using a shape matching algorithm such as an invariant moment method, a wavelet transform method or a shape context method.
8. A method for detecting true or false signatures for double recording quality inspection according to claim 1, characterized in that: In step 6, the ratio of the number of frames in which the pen tip trajectory matches the signature handwriting in the signing process to the total number of frames is counted. When the ratio exceeds a threshold value T, it is determined to be a true signature, otherwise it is determined to be a false signature.
9. A true or false signature action detection device for double recording quality inspection, characterized in that: The invention comprises a signature photographing device, a memory and a processor, wherein the signature photographing device is used to photograph the signing process to ensure that the signature handwriting and the pen tip can appear simultaneously in the video frame of the signing process, the signature photographing device communicates with the memory to exchange the video data of the signing process, the memory stores a detection program designed by a method for detecting true and false signature actions for double recording quality inspection as described in any one of claims 1 to 8, the processor is communicatively connected with the memory, the video data of the signing process is input into the detection program, and the detection program is run to output the detection result.
10. A true or false signature motion detection device for double recording quality inspection according to claim 9, characterized in that: It also includes a display, which is communicatively connected to the processor, and the processor controls the display to display the detection results.