Handwritten electronic signature method and system based on multi-modal biological characteristics

Abstract

The invention discloses a handwritten electronic signature method and system based on multi-modal biological characteristics. The method comprises the following steps: carrying out real-name authentication on a signer before signing an electronic file; if the real-name authentication is passed, biological characteristic verification and evidence storage are carried out; recording handwriting data after biological feature verification is passed and signing is completed, and forming a data packet; the signed electronic file is printed into digital paper, the data packet is embedded into a digital paper file structure, and the digital paper is a layout file simulating real paper. According to the invention, the problem that the signature of the existing signer cannot represent the real identity and needs to depend on the CA certificate is solved, the tedious processes of applying, issuing, managing and the like for the CA certificate are avoided, and the online signature and the offline signature have the same efficacy.

Description

Technical Field

[0001] This invention belongs to the field of electronic signatures, and specifically relates to a handwritten electronic signature method and system based on multimodal biometrics. Background Technology

[0002] In current mainstream digital signature systems based on PKI (Public Key Infrastructure) certification centers, the digital certificate acts as the signer's "identity card" on the internet to meet the requirement of authentic identity under the Electronic Signature Law. While PKI relies on the identity of the private key holder of the digital certificate, it cannot verify whether the actual operator is indeed the certificate holder. Scenarios of identity theft due to stolen private keys still exist, meaning the personal signature does not truly represent the individual's true identity. Furthermore, there are complex management processes for applying for, renewing, and revoking CA certificates. Traditional digital signatures only verify hash values, making it impossible to trace the signing process and failing to address the forensic authentication challenges caused by the "instantaneous" nature of signing. Summary of the Invention

[0003] To address the above problems, the technical solution adopted by this invention is: a handwritten electronic signature method based on multimodal biometrics, the method comprising the following steps:

[0004] Before signing electronic documents, the signatory must undergo real-name authentication.

[0005] If real-name authentication is successful, biometric verification will be performed and evidence will be stored.

[0006] After biometric verification is passed and the signature is completed, the handwriting data is recorded and compiled into a data packet;

[0007] The signed electronic document is printed as digital paper, and the data package is embedded in the digital paper document structure, wherein the digital paper is a layout document that simulates real paper.

[0008] Optionally, the data packet includes the pressure, speed, stroke, slope, and pressure area of ​​the handwriting.

[0009] Optionally, the step of printing the signed electronic document onto digital paper includes the following steps:

[0010] Digital paper is generated based on fiber modeling and fiber deposition to simulate the fibers of real paper.

[0011] The signature is integrated with the digital paper by simulating real ink using digital ink technology.

[0012] Optionally, the step of printing the signed electronic document onto digital paper may further include the following steps:

[0013] Record the paper-ink interaction characteristics between the signature and the digital paper; the paper-ink interaction characteristics include the number of fibers of the digital paper covered by each inflection point of the signature.

[0014] Optionally, the method further includes the following steps:

[0015] Extract handwriting data from the digital paper to be identified;

[0016] Handwriting analysis is performed on the handwriting data;

[0017] The handwriting identification result is compared with the identity certificate stored on the digital paper at the time of signing to determine whether they belong to the same person.

[0018] Optionally, the method further includes the following steps:

[0019] To determine whether the digital paper to be tested has been tampered with;

[0020] If the signature has not been altered, detailed signature information will be displayed, and a digital paper signature evidence chain report will be generated.

[0021] When conducting handwriting verification, the handwriting is first verified based on the evidence chain report signed on the digital paper to determine that the digital paper to be identified has not been tampered with.

[0022] Optionally, the step of determining whether the digital paper to be identified has been tampered with includes:

[0023] Store the signed digital paper and record the storage path or address;

[0024] The summary data of the digital paper file is calculated using the storage path or address as the key and stored in the database;

[0025] During the verification process, the digital paper file to be reviewed is input, the file summary value is calculated, and it is compared with the summary data of the corresponding file in the database. Based on whether the comparison results are the same, it is determined whether the file has been tampered with.

[0026] Optionally, if the digital paper file contains multiple sub-files, the method further includes the following steps:

[0027] Traverse each sub-file in the digital paper, calculate the corresponding summary data using the storage path or address of each sub-file as the key, summarize them into a set, denoted as digestMap, and store it in the database;

[0028] During the identification process, the digital paper file to be reviewed is input, its sub-files are traversed to calculate the digest value, and compared with the digestMap. Based on the comparison results, the tampering information of the sub-files is recorded.

[0029] And, a handwritten electronic signature system based on multimodal biometrics, the system comprising:

[0030] The real-name authentication module is used to verify the real-name identity of the signer of electronic documents uploaded to the system.

[0031] The biometric verification module is used to verify the biometric characteristics of the signatory and store the evidence.

[0032] The handwriting data packaging module is used to record handwriting data and assemble it into a data package after signing is completed;

[0033] The document printing module is used to print signed electronic documents into digital paper and embed the data package into the digital paper document structure, wherein the digital paper is a layout document that simulates real paper.

[0034] Optionally, the system further includes a document authentication module for identifying whether a signed document has been tampered with and for identifying the handwriting of the signature.

[0035] By employing the aforementioned technical solution, this invention offers the following advantages: it resolves the issue that existing signatures cannot represent the true identity of the signer and thus require reliance on a CA certificate, eliminating the cumbersome processes of applying for, issuing, and managing CA certificates, and ensuring that online signatures have the same validity as offline signatures. Furthermore, it enables full integrity verification of signed documents to prevent document tampering.

[0036] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic flowchart of a signature method according to Embodiment 1 of the present invention is shown. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1

[0041] like Figure 1 As shown, the handwritten electronic signature method based on multimodal biometrics according to an embodiment of the present invention, taking the electronic contract signing process as an example, includes the following steps:

[0042] S1: Verify the real name of the signatory before signing electronic documents.

[0043] S2: If real-name authentication is successful, biometric verification will be performed and evidence will be stored.

[0044] S3: Record handwriting data and assemble it into a data packet after biometric verification is passed and signing is completed;

[0045] In this embodiment, since it pertains to contract signing, steps S1 to S3 include authentication and signing processes for both the offeror and offeree. Specifically, the offeror drafts the contract text, initiates the signing process, and uploads the contract document. Verification of the initiator's real-name authentication is required, followed by biometric verification (in this embodiment, facial recognition). After completion, the information is stored and a success receipt is sent. The initiator can then sign the contract, recording the complete handwriting data to form a data packet. The offeror then sends the contract to the offeree. Upon receiving the contract document, the offeree reviews it and, similarly, undergoes real-name authentication and facial recognition before signing. Again, the complete handwriting data is recorded after signing, forming a data packet.

[0046] The data packet includes the pressure, speed, stroke, slope, and pressure area of ​​the handwriting.

[0047] S4: Print the signed electronic document onto digital paper and embed the data package into the digital paper document structure, wherein the digital paper is a layout file simulating real paper. The signing process ends, the electronic contract is finalized, and all parties are notified to conclude the process.

[0048] The printing process for digital paper specifically includes the following steps:

[0049] S401: Digital paper generated from fibers based on fiber modeling and fiber deposition simulation of real paper;

[0050] S402: Based on digital ink technology, simulate real ink to integrate the signature handwriting with the digital paper;

[0051] S403: Record the paper-ink interaction characteristics between the signature and the digital paper. The paper-ink interaction characteristics include the number of fibers on the digital paper covered by each inflection point of the signature.

[0052] In the electronic contract signing process, facial recognition is used at every crucial step. The facial images of the offeror and offeree are compared with authoritative data sources from relevant departments to ensure identity verification at every stage. During handwritten signatures, detailed dynamic signature data, such as speed, pressure, inclination, strokes, and pressure area, are recorded to form a signature package. This package is embedded into the electronic contract structure, comprehensively recording the entire signing process. The handwritten signature in the contract is not merely an image, but a visual representation of signature data. The recorded handwriting data allows for a complete reproduction of the signing process, verifying the formation of the signature image. After signing, the electronic contract is printed on digital paper, ensuring its content cannot be tampered with.

[0053] Example 2

[0054] When verifying an electronic contract and raising objections, the first step is to verify whether the content and signature have been tampered with. If, after verification, no tampering is found, a digital paper electronic signature evidence chain traceability report can be issued for the electronic contract. This report objectively and comprehensively proves the authenticity of the identity, the true intent, the original text's integrity, and the signature's integrity throughout the entire lifecycle of the signing process using a series of stored data. In practice, this includes the following steps:

[0055] S5: Upload the electronic contract to be authenticated to the system to verify whether the electronic contract has been tampered with, i.e., whether the digital paper has been altered. If tampered with, proceed to step S6; if not tampered with, proceed to step S7. The authentication process includes:

[0056] S501: Store the signed digital paper and record the storage path or address;

[0057] S502: Calculate the summary data of the digital paper file using the storage path or address as the key, and store it in the database.

[0058] In some practical situations, a digital paper file contains multiple sub-files. If the digital paper file contains multiple sub-files, the above steps also include: traversing each sub-file in the digital paper, using the storage path or address of each sub-file as the key, calculating the corresponding summary data, summarizing it into a set, denoted as digestMap, and storing it in the database.

[0059] S503: During the identification process, input the digital paper file to be reviewed, calculate the file summary value, and compare it with the summary data of the corresponding file in the database. Based on whether the comparison results are the same, determine whether it has been tampered with.

[0060] After inputting the digital paper file to be reviewed, the system parses the file to obtain the file ID. Based on this ID, the corresponding summary data in the database can be found for comparison.

[0061] In the case of multiple sub-files, if the judgment result in step S503 is different, that is, it has been tampered with, then this step further includes: traversing its sub-files to calculate the digest value and comparing it with the digestMap, recording the sub-file tampering information based on the comparison result, and locating the modified sub-file based on the key value of each sub-file.

[0062] S6: The document is flagged as tampered with, and the tampered part is determined based on the information of the digital paper content of the identified modified sub-document. Specifically, this includes text content comparison and signature information comparison, where signature information comparison includes comparing paper-ink interaction features.

[0063] S7: Display detailed signing information and generate a digital paper signature evidence chain report;

[0064] S8: Extract handwriting data and conduct forensic handwriting identification on the handwriting data to determine the owner of the handwriting;

[0065] S9: Compare the handwriting identification result with the identity certificate stored on the digital paper at the time of signing to determine whether they belong to the same person.

[0066] This solution verifies the signer's true identity through multimodal biometrics rather than through digital certificates issued by a Certificate Authority (CA). This method ensures that the signer's signature truly represents their identity and intent. In the multimodal biometric approach, the electronic signature creation data is the signer's handwriting biometric information, which cannot be created by any means other than a genuine personal signature and cannot be possessed by others. This method solves the problem that existing methods rely on CA certificates because the signer's own signature cannot represent their true identity, eliminating the cumbersome processes of applying for, issuing, and managing CA certificates. Furthermore, handwriting analysis provides further verification in case the signer objects. This makes online signatures as valid as offline signatures.

[0067] Example 3

[0068] A handwritten electronic signature system based on multimodal biometrics is provided, the system comprising:

[0069] The real-name authentication module is used to verify the real-name identity of the signer of electronic documents uploaded to the system.

[0070] The biometric verification module is used to verify the biometric characteristics of the signatory and store the evidence.

[0071] The handwriting data packaging module is used to record handwriting data and assemble it into a data package after signing is completed;

[0072] The document printing module is used to print signed electronic documents into digital paper and embed the data package into the digital paper document structure, wherein the digital paper is a layout document that simulates real paper.

[0073] The document authentication module is used to determine whether signed documents have been tampered with and to authenticate the handwriting of the signature.

[0074] It should be noted that the specific implementation process of the functions and roles of each device, module, and unit in this system can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A handwritten electronic signature method based on multimodal biometrics, characterized in that, The method includes the following steps: Before signing electronic documents, the signatory must undergo real-name authentication. If real-name authentication is successful, biometric verification will be performed and evidence will be stored. After biometric verification is passed and the signature is completed, the handwriting data is recorded and compiled into a data packet; The signed electronic document is printed as digital paper, and the data package is embedded in the digital paper document structure, wherein the digital paper is a layout document that simulates real paper.

2. The handwritten electronic signature method based on multimodal biometrics as described in claim 1, characterized in that, The data packet includes the pressure, speed, stroke, slope, and pressure area of ​​the handwriting.

3. The handwritten electronic signature method based on multimodal biometrics as described in claim 1, characterized in that, The step of printing the signed electronic document onto digital paper includes the following steps: Digital paper is generated based on fiber modeling and fiber deposition to simulate the fibers of real paper. The signature is integrated with the digital paper by simulating real ink using digital ink technology.

4. The handwritten electronic signature method based on multimodal biometrics as described in claim 3, characterized in that, The step of printing the signed electronic document onto digital paper also includes the following steps: Record the paper-ink interaction characteristics between the signature and the digital paper; the paper-ink interaction characteristics include the number of fibers of the digital paper covered by each inflection point of the signature.

5. The handwritten electronic signature method based on multimodal biometrics as described in claim 1, characterized in that, The method further includes the following steps: Extract handwriting data from the digital paper to be identified; Handwriting analysis is performed on the handwriting data; The handwriting identification result is compared with the identity certificate stored on the digital paper at the time of signing to determine whether they belong to the same person.

6. The handwritten electronic signature method based on multimodal biometrics as described in claim 1 or 5, characterized in that, The method further includes the following steps: To determine whether the digital paper to be tested has been tampered with; If the signature has not been altered, detailed signature information will be displayed, and a digital paper signature evidence chain report will be generated. When conducting handwriting verification, the handwriting is first verified based on the evidence chain report signed on the digital paper to determine that the digital paper to be identified has not been tampered with.

7. The handwritten electronic signature method based on multimodal biometrics as described in claim 6, characterized in that, The step of determining whether the digital paper to be identified has been tampered with includes: Store the signed digital paper and record the storage path or address; The summary data of the digital paper file is calculated using the storage path or address as the key and stored in the database; During the verification process, the digital paper file to be reviewed is input, the file summary value is calculated, and it is compared with the summary data of the corresponding file in the database. Based on whether the comparison results are the same, it is determined whether the file has been tampered with.

8. The handwritten electronic signature method based on multimodal biometrics as described in claim 7, characterized in that, If the digital paper file contains multiple sub-files, the method further includes the following steps: Traverse each sub-file in the digital paper, calculate the corresponding summary data using the storage path or address of each sub-file as the key, summarize them into a set, denoted as digestMap, and store it in the database; During the identification process, the digital paper file to be reviewed is input, its sub-files are traversed to calculate the digest value, and compared with the digestMap. Based on the comparison results, the tampering information of the sub-files is recorded.

9. A handwritten electronic signature system based on multimodal biometrics, characterized in that, The system includes: The real-name authentication module is used to verify the real-name identity of the signer of electronic documents uploaded to the system. The biometric verification module is used to verify the biometric characteristics of the signatory and store the evidence. The handwriting data packaging module is used to record handwriting data and assemble it into a data package after signing is completed; The document printing module is used to print signed electronic documents into digital paper and embed the data package into the digital paper document structure, wherein the digital paper is a layout document that simulates real paper.

10. The handwritten electronic signature system based on multimodal biometrics as described in claim 9, characterized in that, The system also includes a document authentication module for identifying whether signed documents have been tampered with and for identifying the handwriting of the signature.

Images