Clinical test informed agreement online signing method and system

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a clinical test informed agreement online signing method, system and equipment, and the method comprises the steps: firstly, obtaining a video clip of a current risk verification step of a subject; secondly, inputting the video frame into a three-dimensional grid generation network for hypothesis generation to obtain a three-dimensional face network hypothesis, and projecting the three-dimensional face network hypothesis into a preset two-dimensional plane to obtain projection key points and projection texture features of the face; then, acquiring biological feature key points and local micro-texture features of the face of the subject from the video frame; then, geometric consistency detection is carried out based on the projection key points, the projection texture features, the biological feature key points and the local micro texture features, and key point alignment errors and texture consistency conditions are obtained; and finally, if the key point alignment error and the texture consistency condition both meet the living body detection condition, continuing the next risk verification step. A non-living body attack can be effectively prevented from counterfeiting an identity, and the accuracy of identity verification is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an online signing method, system and device for informed consent for clinical trials. Background Art

[0002] Informed consent is an essential part of clinical trials, and ensuring that subjects fully understand the purpose, methods, potential risks and other relevant information of the trial before signing the informed consent form is a basic requirement to protect their rights and interests. In related technologies, in order to improve the standardization of the informed consent process and save manpower, the informed consent of subjects is signed using an online system. Subjects can interact with the online system as needed through applications and cameras on their smartphones.

[0003] However, the use of online systems also brings potential risks. Since subjects interact online through cameras, there is a possibility of identity forgery. For example, someone may forge their identity through images or videos. This behavior greatly undermines the standardization of clinical trials and has a huge impact on the scientific nature of the trial results.

[0004] Therefore, it is urgent to propose a new online signing method for informed consent in clinical trials. Summary of the invention

[0005] The present application provides an online signing method, system and device for informed consent for clinical trials, which solves the technical problem of identity forgery using online systems in related technologies and achieves a technical effect of more accurate identity authentication of subjects in online systems.

[0006] In order to achieve the above objectives, the main technical solutions adopted in this application include: In a first aspect, an embodiment of the present application provides an online signing method for informed consent for clinical trials, wherein the online signing process is divided into a plurality of sequentially progressive risk verification steps, and the method comprises: After completing the interactive process of the current risk verification step, obtain the video clip of the subject signing the clinical trial informed consent form online; Inputting the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis, obtaining a three-dimensional facial network hypothesis of the subject, and projecting the three-dimensional facial network hypothesis onto a preset two-dimensional plane to obtain projection key points and projection texture features of the subject's face; Extracting key points and texture features from the video frame to obtain biometric key points and local microtexture features of the subject's face; Performing geometric consistency detection based on the projected key points, the projected texture features, the biometric key points and the local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame; If both the key point alignment error and the texture consistency satisfy the living body detection condition, proceed to the next risk verification step of the current risk verification step.

[0007] Optionally, the plurality of risk verification steps include informed information display and preliminary verification steps, and the method further comprises: In the informed information display and preliminary verification step, if the subject passes the face verification, a preliminary verification interactive interface is displayed; wherein the preliminary verification interactive interface has basic information of the clinical trial; If it is detected that the subject has completed reviewing the basic information, it is confirmed that the informed information display and preliminary verification steps are completed.

[0008] Optionally, the plurality of risk verification steps include key clause explanation and dynamic behavior verification steps, and the method further includes: In the key clause explanation and dynamic behavior verification step, voice data of the key clause explanation is played one by one, and a first action reminder message is issued; wherein, the first action reminder message is used to require the subject to confirm understanding of the key clause through a first specified action behavior.

[0009] Optionally, the plurality of risk verification steps include question-answer confirmation and in-depth verification steps, and the method further comprises: In the question-answer confirmation and deep verification step, obtaining the voice data of the subject's questions regarding the clinical trial; In response to the interrogation voice data, reply voice data for the interrogation voice data is generated, and a second action reminder message is issued; wherein the second action reminder message requires the subject to perform a second specified action behavior to ensure dynamic participation of the living body.

[0010] Optionally, the plurality of risk verification steps include a signing step, and the method further comprises: In the signing step, in response to the electronic signature entry operation, signature record data that cannot be tampered with is generated; The process data of each of the plurality of risk verification steps, the biometrics of the subject and the signed record data are bound and stored in an encrypted manner.

[0011] Optionally, the method further comprises: If either the key point alignment error or the texture consistency condition does not satisfy the living body detection condition, an abnormal alarm is triggered and a re-verification reminder message is sent.

[0012] Optionally, the performing geometric consistency detection based on the projected key points, the projected texture features, the biometric key points and the local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame includes: Matching the projected key points with the biometric key points to obtain an average Euclidean distance, and determining the key point alignment error based on the average Euclidean distance; The projected texture features and the local micro-texture features are compared using a structural similarity index and cosine similarity to obtain the texture consistency.

[0013] Optionally, inputting the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis to obtain a three-dimensional facial network hypothesis of the subject includes: Inputting video frames in the video clip into a three-dimensional mesh generation network for hypothesis generation to obtain a plurality of facial network hypotheses; Each facial network hypothesis is scored, and the facial network hypothesis with the highest score is used as the three-dimensional facial network hypothesis.

[0014] In a second aspect, an embodiment of the present application provides an online signing system for informed consent for clinical trials, the system comprising: A video clip acquisition module is used to acquire a video clip of the subject signing the clinical trial informed consent online when the interactive process of the current risk verification step is completed; A hypothesis generation and projection module is used to input the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis, obtain a three-dimensional facial network hypothesis of the subject, and project the three-dimensional facial network hypothesis into a preset two-dimensional plane to obtain projection key points and projection texture features of the subject's face; A video frame feature extraction module, used to extract key points and texture features from the video frame to obtain biometric key points and local micro-texture features of the subject's face; A geometric consistency detection module performs geometric consistency detection based on the projected key points, the projected texture features, the biometric key points and the local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame; The liveness detection result confirmation module is used to continue the next risk verification step of the current risk verification step if the key point alignment error and the texture consistency both meet the liveness detection conditions.

[0015] In a third aspect, an embodiment of the present application provides a computer device, including: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in any of the above embodiments by executing the computer instructions.

[0016] In the embodiment of the present application, first, after completing the interactive process of the current risk verification step, a video clip of the subject signing the informed consent form online for the clinical trial is obtained; secondly, the video frame in the video clip is input into the three-dimensional mesh generation network for hypothesis generation, and the three-dimensional facial network hypothesis of the subject is obtained, and the three-dimensional facial network hypothesis is projected into a preset two-dimensional plane to obtain the projection key points and projection texture features of the subject's face; then, the video frame is subjected to key point extraction and texture feature extraction to obtain the biometric key points and local micro-texture features of the subject's face; then, geometric consistency detection is performed based on the projection key points, projection texture features, biometric key points and local micro-texture features to obtain the key point alignment error and texture consistency between the three-dimensional facial network hypothesis and the video frame; finally, if the key point alignment error and texture consistency meet the liveness detection conditions, the next risk verification step of the current risk verification step is continued. Through the three-dimensional facial network hypothesis generation and projection technology, combined with the extraction and comparison of biometric key points and local micro-texture features, it is possible to effectively prevent non-live attacks from forging identities, and significantly improve the accuracy of identity authentication in the signing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a clinical trial informed consent system provided in an embodiment of this specification; Figure 2 A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 3 A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 4 A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 5 A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 6A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 7a A flow chart of the online signing method for informed consent for clinical trials provided in the embodiments of this specification; Figure 7b A schematic diagram of a noise estimation model provided in an embodiment of this specification; Figure 8 A schematic diagram of an online signing device for informed consent for clinical trials provided in the embodiments of this specification; Fig. 9 A schematic diagram of the structure of a computer device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0020] In clinical trials, ensuring that subjects fully understand the purpose, methods and risks of the trials is a basic requirement for protecting their rights and interests. In order to improve the standardization of the informed consent process and save human resources, the use of an online system to sign informed consent for clinical trials is a direction worth exploring. Subjects can interact with the system through smartphone applications and cameras to complete the informed consent process. However, online systems also have technical problems. Subjects interacting with cameras may lead to identity forgery, such as using images or videos to impersonate others, which not only affects the standardization of the trial, but may also affect the scientific nature of the results.

[0021] Based on this, the present application proposes an online signing method for informed consent of clinical trials, and the online signing process is divided into multiple risk verification steps in sequence. The method includes: first, after completing the interactive process of the current risk verification step, obtaining a video clip of the subject in the process of online signing of the informed consent form of the clinical trial; secondly, inputting the video frame in the video clip into the three-dimensional mesh generation network for hypothesis generation to obtain the three-dimensional facial network hypothesis of the subject, and projecting the three-dimensional facial network hypothesis into a preset two-dimensional plane to obtain the projection key points and projection texture features of the subject's face; then, performing key point extraction and texture feature extraction on the video frame to obtain the biometric key points and local micro-texture features of the subject's face; then, performing geometric consistency detection based on the projection key points, projection texture features, biometric key points and local micro-texture features to obtain the key point alignment error and texture consistency between the three-dimensional facial network hypothesis and the video frame; finally, if both the key point alignment error and the texture consistency meet the liveness detection conditions, continue to the next risk verification step of the current risk verification step. This method reduces the complexity of the operation by dividing it into multiple progressive risk verification steps; reduces manual intervention through automated processes and intelligent processing, and improves the standardization of the signing of informed consent for clinical trials; through three-dimensional facial network hypothesis generation and projection technology, combined with the extraction and comparison of biometric key points and local microtexture features, it effectively prevents non-living attacks from forging identities and significantly improves the security of the signing process. It provides a low-complexity, highly standardized, and highly secure method for online signing of informed consent for clinical trials.

[0022] This application scenario example proposes an online signing method for informed consent for clinical trials. Figure 1 , this method is applied to the platform management terminal 120 of the clinical trial informed consent system, and the system also includes a first user terminal 110 and a second user terminal 130 that are communicatively connected to the platform management terminal 120. Among them, the platform management terminal 120 is responsible for the overall management of the system, which may include registering the subjects in the system after recruitment, guiding the subjects to perform the informed consent and signing process after passing the preliminary verification. The first user terminal 110 is provided to the subjects for use, and its functions include initiating subject registration applications, obtaining recruitment results, and subsequent clinical trial records. The second user terminal 130 is used by real doctors, and its functions include assessing whether the subjects are fully informed of the clinical trial and confirming their consent to participate in the clinical trial, as well as guiding them to complete the electronic signature entry.

[0023] After the subject initiates the registration application through the first user terminal 110, the platform management terminal 120 receives the request and immediately guides the subject to the informed consent process. First, the subject is prompted that the process requires video acquisition and the subject's consent is obtained; then the subject's face is photographed to confirm whether the light is appropriate, that is, not too bright or too dark, so as not to affect the video acquisition effect. If it is not appropriate, the subject can be prompted by voice to move to a place with appropriate light and confirm. After receiving the confirmation information, the platform management terminal 120 takes a photo of the face again through the camera of the first user terminal 110. After the light check is passed, the informed information display and preliminary verification steps can be performed. Exemplarily, when the subject passes the face verification, the preliminary verification interactive interface is displayed; wherein the preliminary verification interactive interface has the basic information of the clinical trial. The subject can view the long basic information by sliding the screen, and after completing the review, the subject can click the "read" button to submit the information. After the platform management terminal 120 detects that the subject has submitted the information, the video clip of this step is analyzed and detected, and the next step can be entered after the detection is passed.

[0024] The next step may be the step of explaining the key terms and verifying dynamic behaviors. The first user terminal 110 can play the explanation of the key terms one by one by voice. After the playback is completed, it asks the subject whether he has understood it, and sends a first action reminder message (such as blinking, opening the mouth), that is, if the subject has understood it, he needs to make the first action. After the platform management terminal 120 detects that the subject has made the first action, it analyzes and detects the video clip of this step, and can proceed to the next step after the detection passes. If the subject is not detected to make the action within a certain period of time, such as 10 seconds, the verification fails, and the subject can choose to try again or exit.

[0025] The next step may be a question-and-answer confirmation and in-depth verification step. The subject can send questions to the platform management end 120 by voice. After the platform management end 120 obtains the question voice data, it can generate the corresponding answer through intelligent analysis and play it by voice. After the playback is completed, the subject is asked whether he has understood, and a second action reminder message (such as blinking, opening the mouth) is issued, that is, if the subject has understood, he needs to make a second action. After the platform management end 120 detects that the subject has made the second action, it analyzes and detects the video clip of this step. After the detection passes, the last step can be entered. If the subject is not detected to make the action within a certain period of time, such as 10 seconds, the verification fails and the subject can choose to retry or exit.

[0026] The last step may be a signing step, where the platform management end 120 initiates a call request to a real doctor. After the doctor logs in to the second client 130, he communicates with the subject via video through the camera. First, the doctor asks the subject if he has any questions about the clinical trial. If so, the doctor will answer them one by one until the subject confirms that there are no more questions; then, the doctor asks the subject if he agrees to participate in the clinical trial, and the subject needs to confirm clearly; finally, the doctor guides the subject to complete the electronic signature entry. The subject can enter the electronic signature on the screen of the mobile phone end 110, and the first user end 110 can upload the signature to the platform management end 120. After the platform management end 120 detects that the subject has submitted the electronic signature, it analyzes and detects the video clip of the step. After the detection is passed, the process data of each of the multiple risk verification steps, the subject's biometrics, and the signature record data are bound and encrypted for storage.

[0027] This scenario example reduces the complexity of operations by dividing the process into multiple progressive risk verification steps; reduces manual intervention and improves the standardization of informed consent signing for clinical trials through automated processes and intelligent processing; and effectively prevents non-living attacks from forging identities through dynamic behavior verification technology, which can significantly improve the security of the signing process.

[0028] According to an embodiment of the present application, an embodiment of an online signing method for informed consent for a clinical trial is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] See also Figure 2 In this embodiment, a method for online signing of informed consent for clinical trials is provided. The online signing process is divided into a plurality of sequentially progressive risk verification steps. The method includes: S110. After the interactive process of the current risk verification step is completed, a video clip of the subject signing the clinical trial informed consent form online is obtained.

[0030] The risk verification steps can be multiple stages in the online signing process to ensure that the subject fully understands and agrees to the relevant information of the clinical trial. The video clip can be a video record corresponding to any risk verification step during the signing process of the subject, which is usually used to verify the identity of the subject and confirm his / her active participation in the process.

[0031] In some embodiments, the online signing method of informed consent for clinical trials is applied to the platform management end of the informed consent system for clinical trials, and the system also includes a first user end that is connected to the platform management end in communication. During the entire informed consent process of the clinical trial, the platform management end can collect video of the subject through the camera of the first user end. The first risk verification step is the informed information display and preliminary verification step. After the subject passes the face verification and agrees to the basic information of the informed consent of the clinical trial, the platform management end obtains the video clip of the subject in this step; the second risk verification step is the key clause explanation and dynamic behavior verification step. After the subject confirms that he is informed of the key clauses, the platform management end obtains the video clip of the subject in this step; the third risk verification step is the question and answer confirmation and in-depth verification step. After the subject confirms that he is informed of the questions asked, the platform management end obtains the video clip of the subject in this step; the fourth risk verification step is the signing step. After the subject submits the electronic signature, the platform management end obtains the video clip of the subject in this step.

[0032] S120, inputting the video frames in the video clip into the three-dimensional mesh generation network for hypothesis generation, obtaining the three-dimensional facial network hypothesis of the subject, and projecting the three-dimensional facial network hypothesis into a preset two-dimensional plane to obtain the projection key points and projection texture features of the subject's face.

[0033] The three-dimensional mesh generation network is used to generate a three-dimensional facial network hypothesis of the subject from the video frames in the video clip, and can be a deep learning model. The three-dimensional facial network hypothesis can be the three-dimensional geometric structure data of the subject's face. The projected key points and projected texture features refer to the key points and texture information obtained after projecting the three-dimensional facial network hypothesis onto a preset two-dimensional plane. For example, the projected key points may include the key point coordinates of the eyes, nose, mouth and other parts of the face, and the projected texture features may include information such as the skin texture of the face.

[0034] In some embodiments, 3D face modeling technology, such as 3D face alignment network (3D-FAN, 3DFace Alignment Network), can be used to generate hypotheses about the subject's face. 3D-FAN first extracts the subject's facial key points from the video frame; then uses a deep neural network to map the facial information in the 2D image to the 3D space, and obtains the subject's three-dimensional facial network hypothesis, including shape, texture, etc.

[0035] In some embodiments, after obtaining the three-dimensional facial network hypothesis of the subject, the three-dimensional facial network hypothesis of the subject is projected onto a two-dimensional plane using perspective projection technology to obtain a two-dimensional facial projection. In order to extract facial key points from the two-dimensional facial projection, Dlib can be used, which is an open source machine learning library widely used in facial recognition and processing tasks. Dlib provides an efficient facial key point detection algorithm that can accurately identify and locate the 68 feature points of the face. Based on this, Dlib can be used to extract the coordinates of the facial key points from the two-dimensional facial projection image and use them as the projected key points of the subject's face. Finally, a convolutional neural network can be used to extract texture features from the two-dimensional facial projection to obtain projected texture features.

[0036] S130, extracting key points and texture features from the video frames to obtain biometric key points and local micro-texture features of the subject's face.

[0037] Among them, key point extraction can be to identify and extract the coordinates of specific facial feature points of the subject from the video frame, such as eyes, nose, mouth, etc. Texture feature extraction can be to analyze facial details to obtain local micro-texture features, such as skin texture, pore distribution, etc.

[0038] In some implementations, Dlib may be used to extract key points of a subject's face from a video frame, such as 68 points of facial features. After extraction, the coordinates of each key point are obtained as a biometric key point.

[0039] In some embodiments, a convolutional neural network may be used to extract texture features from the face of a subject in a video frame to obtain local micro-texture features of the subject.

[0040] S140, performing geometric consistency detection based on projected key points, projected texture features, biometric key points and local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame.

[0041] Among them, the geometric consistency detection can be to judge the degree of match between the three-dimensional facial network hypothesis and the subject's features in the video frame by comparing the difference between the projected key points and the biometric key points, and comparing the difference between the projected texture features and the local micro-texture features. The key point alignment error can be the error between the projected key points and the biometric key points. The texture consistency can be the difference between the projected texture features and the local micro-texture features.

[0042] In some embodiments, key point alignment is performed by a landmark-based alignment method, such as selecting the eye as a reference to geometrically align the projected key points and the biometric key points. After the alignment is completed, the key point alignment error is calculated, and the Euclidean distance can be calculated. Specifically, the Euclidean distance of each pair of key points can be calculated as a measure of the error, and the mean square error can also be calculated.

[0043] In some embodiments, the texture consistency between the three-dimensional face network hypothesis and the video frame is obtained by comparing the similarity between the projected texture features and the local micro-texture features. Specifically, the Euclidean distance or cosine similarity between the projected texture features and the local micro-texture features can be calculated.

[0044] It should be noted that real faces have natural three-dimensional structures, and their projections at different viewing angles follow strict perspective geometry. In contrast, forged attacks (such as image or video replays) only contain planar information or fixed two-dimensional viewing angles, and therefore cannot present three-dimensional structures that conform to projection rules. Specifically, the three-dimensional facial network assumption of real faces shows natural facial contours and depth changes when projected, while two-dimensional forged images may appear flattened or distorted. Based on this principle, when the three-dimensional key points of a real face are projected into two dimensions, the error with the key points of the face image in the video frame is small. On the contrary, due to the lack of a reasonable three-dimensional structure of forged attacks, their projection error increases significantly. In addition, the texture of real skin usually has a high degree of match with the image in the video frame, while forged attacks often produce obvious visual differences due to material reflection, insufficient texture resolution, or synthesis problems. Therefore, it is possible to determine whether the subject in the video frame is a forged attack by the key point alignment error and texture consistency.

[0045] S150: If both the key point alignment error and the texture consistency meet the living body detection conditions, proceed to the next risk verification step of the current risk verification step.

[0046] Among them, the liveness detection condition can be a pre-set threshold or standard, which is used to determine whether the alignment error of the key points on the subject's face and the consistency of the texture meet the liveness detection requirements. Through these conditions, the authenticity of the facial features can be evaluated to ensure that the subject is actually participating in the signing process and is a living person with biometric features, rather than a forged image or video.

[0047] In some implementations, if the key point alignment error is less than the key point alignment error threshold, and the texture consistency is greater than the texture consistency threshold, it is determined that both the key point alignment error and the texture consistency meet the liveness detection conditions, and the next risk verification step can be continued. For example, in the informed information display and preliminary verification steps, the key point alignment error and texture consistency meet the liveness detection conditions, and the key terms explanation and dynamic behavior verification steps can be continued.

[0048] In some implementations, a video frame is extracted for detection at regular time steps (e.g., 5 seconds), and the key point alignment errors and texture consistency obtained multiple times are averaged and compared with liveness detection conditions to determine whether the subject in the video frame is a forged attack.

[0049] In the above embodiment, by dividing multiple risk verification steps into progressive steps, the complexity of the operation is reduced; by automated processes and intelligent processing, manual intervention is reduced, and the standardization of the signing of informed consent for clinical trials is improved; by three-dimensional facial network hypothesis generation and projection technology, combined with the extraction and comparison of biometric key points and local micro-texture features, it effectively prevents non-living attacks from forging identities, and significantly improves the accuracy of identity verification during the signing process. This embodiment provides a low-complexity, high-standardization, and high-accuracy method for online signing of informed consent for clinical trials.

[0050] See also Figure 3 In some embodiments, the plurality of risk verification steps include an informed information display and a preliminary verification step, and the method further comprises: S210. In the informed information display and preliminary verification step, when the subject passes the face verification, a preliminary verification interactive interface is displayed.

[0051] S220: If it is detected that the subject has completed the review of the basic information, confirm that the informed information display and preliminary verification steps are completed.

[0052] Among them, the informed information display and preliminary verification step is the first step in multiple risk verification steps, which is used to complete the preliminary verification and show the basic information of the clinical trial to the subjects. The preliminary verification interactive interface has the basic information of the clinical trial. The basic information can be the core content of the clinical trial, such as the purpose and background of the study, the situation of the drug, the eligibility of the subjects, and the research process (including adverse reactions, possible benefits, damages, etc.). The subjects must review this information in full to ensure that they fully understand the upcoming trial. Face verification can be to verify the identity of the subject through facial recognition technology.

[0053] In some embodiments, face verification can first obtain an image of the subject through a camera; then perform feature extraction on the face to obtain the facial features of the subject; then obtain the facial image corresponding to the subject's identity information from the ID card information database through an interface, perform feature extraction to obtain the subject's identity features; finally, perform a similarity comparison between the subject's facial features and identity features to determine whether the identities match.

[0054] In some embodiments, when the basic information of the clinical trial is displayed in the preliminary verification interactive interface, the subject can view the relatively long basic information by sliding the screen. After completing the review, the subject can click the "Read" button to submit the information. After the platform management detects that the subject has submitted the information, it confirms that the subject has completed the informed information display and preliminary verification steps, and obtains the video clip of this step. Then, according to steps S120 to S150, a liveness test is performed on the subject in the video clip. If the test passes, proceed to the next step.

[0055] In the above embodiment, the accuracy of the subject's identity is initially confirmed by face verification; after the subject has completed the review of basic information, the accuracy of his / her identity in the steps of informed information display and preliminary verification is further confirmed by geometric consistency detection. The reliability and accuracy of identity confirmation are improved through the double verification mechanism.

[0056] In some embodiments, multiple risk verification steps include key clause explanation and dynamic behavior verification steps, and the method also includes: in the key clause explanation and dynamic behavior verification steps, playing voice data of the key clause explanation one by one, and issuing a first action reminder message.

[0057] Among them, key terms may be content that requires special attention in clinical trials, such as subject eligibility, privacy protection, exit mechanism, etc. The first action reminder message is used to require the subject to confirm understanding of the key terms through a first specified action behavior, such as requiring the subject to nod or blink to confirm understanding of the key terms.

[0058] In some embodiments, the mobile phone plays the voice data of the explanation of the key terms one by one. After the playback is completed, the subject is asked to blink through voice to confirm that he has understood the content of the key terms. The mobile phone first obtains the eye key points through Dlib, and then calculates the aspect ratio (Eye Aspect Ratio, EAR) of the subject's eyes when open and closed, and monitors its dynamic changes to determine whether it is a live operation, effectively preventing non-live attacks. After the test passes, submit the confirmation information to the platform management end. The platform management end obtains the video clip of this step, and then extracts a video frame for detection at a certain time step (such as 5s), takes the average of the key point alignment errors and texture consistency obtained multiple times, and then compares them with the live detection conditions to determine whether the subject in the video frame is a forged attack. If the test passes, proceed to the next step.

[0059] In the above embodiment, in the key clause explanation and dynamic behavior verification steps, a preliminary non-living attack prevention is first performed through the first specified action, and then the prevention is further strengthened through geometric consistency detection, and the double detection is used to ensure the accuracy of the subject's identity.

[0060] See also Figure 4 In some embodiments, the multiple risk verification steps include question-answer confirmation and in-depth verification steps, and the method further includes: S410. In the question-answer confirmation and deep verification steps, voice data of the subjects' questions regarding the clinical trial are obtained.

[0061] S420: In response to the question voice data, generate reply voice data for the question voice data, and send a second action reminder message.

[0062] The question voice data may be a voice record of any question asked by the subject in the question and answer confirmation and depth verification steps. The reply voice data refers to the voice reply generated by the platform management end according to the question of the subject. The second action reminder message requires the subject to perform a second specified action behavior to ensure dynamic participation of the living body.

[0063] In some embodiments, the mobile phone prompts the subject to input questions about the clinical trial through voice input, and collects voice information in real time through the microphone of the mobile phone. After the collection is completed, the question voice data is obtained and submitted to the platform management end. The platform management end first pre-processes the question voice data through natural language processing technology (NLP), including voice recognition (ASR) to convert voice into text, and segmentation, denoising and semantic analysis of the text to accurately extract the core content of the subject's question voice data; then, the platform management end generates targeted reply text for the extracted question content based on the trained clinical trial question and answer model; finally, the platform management end uses speech synthesis technology (TTS) to convert the reply text into natural and fluent reply voice data, and plays it to the subject through the mobile phone end to answer his questions clearly and accurately, ensuring the subject's understanding of the clinical trial content.

[0064] It should be noted that the clinical trial question-answering model can be obtained by fine-tuning the pre-trained Transformer large model. For example, based on GPT, the clinical trial-related question-answer pairs are input as training data to fine-tune it, including the purpose, process, risk, privacy protection measures, etc. of the trial. Through fine-tuning, the large model can learn the specific semantics and contextual relationships in the field of clinical trials, thereby generating more accurate and professional responses.

[0065] In some embodiments, after the mobile phone obtains the subject's confirmation without any doubt, it asks the subject to open his mouth through voice. The mobile phone first obtains the mouth key points through the Dlib tool, and then calculates the aspect ratio (Mouth Aspect Ratio, MAR) of the subject's mouth when it is open and closed, and monitors its dynamic changes to determine whether it is a live operation, effectively preventing non-live attacks. After the test passes, submit the confirmation information to the platform management end. The platform management end obtains the video clip of this step, and then extracts a video frame for detection at a certain time step (such as 5s), takes the average of the key point alignment errors and texture consistency obtained multiple times, and then compares them with the live detection conditions to determine whether the subject in the video frame is a fake attack. If the test passes, proceed to the next step.

[0066] In the above embodiment, in the question-answer confirmation and deep verification steps, a preliminary non-living attack prevention is first performed through the second designated action, and then the prevention is further strengthened through geometric consistency detection, and the double detection is used to ensure the accuracy of the subject's identity.

[0067] See also Figure 5 In some embodiments, the plurality of risk verification steps include a signing step, the method further comprising: S510. In the signing step, in response to the electronic signature entry operation, tamper-proof signature record data is generated.

[0068] S520, binding and encrypting and storing the process data of each of the multiple risk verification steps, the subject's biometrics and the signature record data.

[0069] Among them, electronic signature entry can be the subject signing the informed consent form through electronic signature on the mobile phone. Process data can be the interactive data generated in each risk verification step, such as video clips, voice data, action records, etc. Encrypted storage can be storing data in a secure database through an encryption algorithm to prevent data leakage or tampering. The subject's biometric features can be the biometric key points and local microtexture features of the subject's face.

[0070] In some embodiments, before the electronic signature is entered, the platform management end can automatically call the real doctor responsible for the informed consent of the clinical trial. After the real doctor goes online through the mobile phone, he can communicate with the subject through the video camera. First, the doctor asks the subject if he has any questions about the clinical trial. If so, the doctor will answer them one by one until the subject confirms that there are no more questions; then, the doctor asks the subject whether he agrees to participate in the clinical trial, and the subject needs to confirm clearly; finally, the doctor guides the subject to complete the electronic signature entry.

[0071] In some embodiments, the subject can enter the electronic signature on the mobile phone screen. After the first user detects that the entry is completed, the electronic signature can be uploaded to the platform management end. The platform management end can use blockchain technology to generate tamper-proof signature record data. First, the electronic signature and related signature information (such as the identity of the signatory, the signing timestamp, the signing content, etc.) are hashed to generate a unique hash value, and then the hash value is recorded in the block of the blockchain. Each block forms a chain structure by including the hash value of the previous block to ensure that the data cannot be tampered with once it is on the chain. Any modification to the signature record will cause the hash value to change, which will be detected by the blockchain network. At the same time, the distributed ledger characteristics of the blockchain enable the signature record to be stored on multiple nodes, further enhancing the security and tamper-proof nature of the data, and ultimately achieving the tamper-proof and high credibility of the electronic signature signature record.

[0072] In some embodiments, after the platform management detects that the subject has submitted an electronic signature, it obtains a video clip of this step, and then extracts a video frame for detection at a certain time step (such as 5s), takes the average of the key point alignment error and texture consistency obtained multiple times, and then compares it with the liveness detection conditions to determine whether the subject in the video frame is a forgery attack. If the detection passes, the process data of each of the multiple risk verification steps, the subject's biometrics and the signature record data are bound, encrypted by an encryption algorithm (such as the Advanced Encryption Standard AES), and then stored in a high-security database to achieve traceability of the entire process of informed consent for clinical trials.

[0073] In the above embodiment, the signed record data that cannot be tampered with is generated by electronic signature entry, and the subject's identity is authenticated in combination with liveness detection technology, effectively preventing the risk of identity forgery by non-liveness attacks. At the same time, the signed record data is bound to the process data of the risk verification step and the subject's biometrics and stored securely, achieving the traceability of the entire process of informed consent for clinical trials, and further providing security for the online signing of informed consent for clinical trials.

[0074] In some embodiments, the method further includes: if any one of the key point alignment error and the texture consistency condition does not meet the living body detection condition, triggering an abnormal alarm and sending a re-verification reminder message.

[0075] Among them, abnormal alarms are used to remind platform managers to conduct further intervention and processing, effectively preventing the risk of non-living attacks.

[0076] In some embodiments, if the key point alignment error is less than the key point alignment error threshold, or the texture consistency is less than the texture consistency threshold, an abnormal alarm message is sent to the platform manager, who can further analyze the subject's video clips and test results to determine whether the abnormality is caused by a non-living attack or a defect in the platform itself. If it is a platform defect, the manager can make improvements and optimizations accordingly, thereby continuously improving the accuracy and reliability of the platform and effectively preventing the risk of non-living attacks.

[0077] In some embodiments, the online signing system for informed consent for clinical trials allows the subject to attempt verification up to three times in each step, that is, the number of re-verifications is two. If the previous two verifications fail, the system will send a re-verification reminder message to the subject, and the subject can operate the current step again according to the reminder.

[0078] The above embodiment sets thresholds for key point alignment error and texture consistency, triggers an abnormal alarm when any condition is not met, and allows the subject to re-verify, which can effectively prevent the risk of non-living attacks and improve the accuracy and reliability of identity authentication.

[0079] See also Figure 6 In some embodiments, geometric consistency detection is performed based on projected key points, projected texture features, biometric key points, and local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame, including: S710, matching the projected key points with the biometric key points to obtain an average Euclidean distance, and determining the key point alignment error based on the average Euclidean distance.

[0080] S720: Use the structural similarity index and cosine similarity to compare the projected texture features and the local micro-texture features to obtain texture consistency.

[0081] The average Euclidean distance can be the average of the geometric distances between the corresponding points of the projection keypoint and the biometric keypoint. The structural similarity index can be used to simulate the subjective perception of image quality by the human visual system, combining the comparison of brightness, contrast and structural information. The closer the value is to 1, the higher the similarity is, and the closer it is to 0, the lower the similarity is.

[0082] In some implementations, the Dlib tool is first used to obtain the projected key points and the biometric key points, and then the geometric distances between the corresponding points are calculated and averaged to obtain the average Euclidean distance, which is used as the key point alignment error.

[0083] In some embodiments, statistics representing brightness, contrast, and structural information can be obtained through image features, and then the structural similarity index between two images can be calculated. For example, the brightness similarity, contrast similarity, and structural similarity are obtained through projected texture features and local micro-texture features, and the formulas are: The structural similarity index is the product of brightness similarity, contrast similarity, and structural similarity: Among them, μ x , μ y are the mean values ​​of the two image pixels (reflecting brightness), σ x 2 , σ y2 are the variance of the pixel values ​​of the two images (reflecting the contrast), σ xy is the covariance of the pixel values ​​of the two images (reflecting the structural similarity), C1, C2, and C3 are constants used to avoid the denominator being zero.

[0084] In some embodiments, the cosine similarity between the projected texture feature and the local microtexture feature is calculated by using the projected texture feature vector and the local microtexture feature vector, and the formula is: Among them, A is the projected texture feature vector, B is the local micro-texture feature vector, and C is the cosine similarity.

[0085] In some implementations, the structural similarity index and the cosine similarity are considered comprehensively. For example, when both the structural similarity index and the cosine similarity exceed their respective thresholds, the texture consistency is determined to be consistent, otherwise it is determined to be inconsistent.

[0086] In the above embodiment, the average Euclidean distance is obtained by matching the projected key points with the biometric key points, and the key point alignment error is determined based on the average Euclidean distance. The structural similarity index and cosine similarity are used to compare the projected texture features with the local micro-texture features to obtain the texture consistency, providing strong data support for liveness detection.

[0087] See also Figure 7a In some embodiments, the video frames in the video clip are input into a three-dimensional mesh generation network for hypothesis generation to obtain a three-dimensional facial network hypothesis of the subject, including: S810: Input the video frames in the video clip into a three-dimensional mesh generation network to generate hypotheses, and obtain multiple facial network hypotheses.

[0088] S820, scoring each facial network hypothesis, and taking the facial network hypothesis with the highest score as the three-dimensional facial network hypothesis.

[0089] In some embodiments, the three-dimensional mesh generation network includes a first facial network hypothesis generation module and a second facial network hypothesis generation module. The first facial network hypothesis generation module may be a 3D face alignment network (3D-FAN), which is used to generate an initial three-dimensional facial network hypothesis according to an image frame. The second facial network hypothesis generation module is used to generate multiple facial network hypotheses, which are based on a denoising diffusion probability model (DDPM, Denoising Diffusion Probabilistic Models), and generate multiple facial network hypotheses through a reverse diffusion process.

[0090] The forward diffusion process of DDPM can be to gradually add noise to the initial 3D facial network hypothesis x0, and after T time steps, it gradually becomes a standard Gaussian distributed noise x T , the forward diffusion process can be expressed as: in, .

[0091] By gradually adding noise, It can be expressed as: in, , .

[0092] The reverse diffusion process of DDPM can be obtained from Gaussian distribution Stepwise denoising and generating facial network hypotheses At each time step T i ,from Can get After multiple iterations, the noise is gradually reduced.

[0093] in, is the noise variance, To estimate the noise, ~N(0,1).

[0094] To get , the noise at each step needs to be known , which can be obtained through a noise estimation model, which can output noise estimates at each step and provide guidance for the reverse sampling process. Figure 7b The model consists of an encoding module 811, a hypothesis generation module 812 and a decoding module 806. The encoding module 811 includes an encoder 801 and an encoder 802. The encoder 801 can be a three-dimensional convolutional neural network (3D CNN). Encode to obtain a noisy facial feature vector. The encoder 802 may be a two-dimensional convolutional neural network (2D CNN). The image frame is feature extracted to obtain local features of the face (which may be eyes, nose, forehead, chin, etc.) and global features of the face, which can be used as diffusion conditions to guide the diffusion process. Assume that the generation module 812 includes a multi-head self-attention unit 803, a cross-attention unit 804, and a feedforward network 805. The local features of the face are input into the multi-head self-attention unit 803 together with the noisy facial feature vector, which can help the model better understand the relationship between local areas, thereby more accurately identifying faces; in the cross-attention unit 804, the global features of the face are used as key and value features, and the output of the multi-head self-attention unit 803 is used as a query feature to better fuse global and local information, so that the diffusion process can be adjusted according to the content of the video frame and make a more accurate estimate of the noise. The decoding module 806 may be a multi-layer perceptron for decoding the output noise estimate.

[0095] It should be noted that a training data set can be pre-constructed. The data samples in the training data set use a 3D facial network with added noise and the corresponding 2D face images. The initial noise estimation model is trained using the training data set until the model stop training condition is met to obtain a noise estimation model. The loss function can use the mean square error (MSE) loss function to continuously optimize the model parameters by comparing the noise estimated by the model with the actual noise added.

[0096] In some embodiments, a sample is taken from the standard Gaussian distributed noise obtained in the forward diffusion process, and the sample is used as the initial value, and the reverse diffusion process is iterated, wherein the noise estimation model is used to obtain the noise estimation of each step. ,After multiple iterations, the facial network hypothesis is finally obtained.

[0097] In some embodiments, multiple sampling is performed from the standard Gaussian distributed noise obtained in the forward diffusion process, and the initial value obtained each time is different. Different values ​​are also randomly sampled, and finally multiple different facial network hypotheses can be obtained.

[0098] It should be noted that when the video frame is unclear due to poor lighting, especially when the non-living attack deliberately uses a dark environment, generating a facial network hypothesis can effectively enhance the probability of capturing the correct facial features. In addition, when the subject's face is partially occluded, generating a facial network hypothesis can improve the system's robustness to the occluded area through diversified predictions. Although the unoccluded part can be directly obtained, generating a hypothesis can help the system better infer the characteristics of the occluded area, thereby improving the integrity and accuracy of the overall facial features. Especially in the case of non-living attacks deliberately implementing partial occlusion, generating a facial network hypothesis can effectively reduce the success rate of the attack and enhance the security of the system. Therefore, the three-dimensional facial network hypothesis obtained by the second facial network hypothesis generation module has a higher accuracy than the initial three-dimensional facial network hypothesis obtained by the first facial network hypothesis generation module. After the initial three-dimensional facial network hypothesis is obtained by the first facial network hypothesis generation module, multiple three-dimensional facial network hypotheses are obtained by the second facial network hypothesis generation module.

[0099] In some embodiments, macro-feature scoring is performed on multiple facial network hypotheses, and the facial network hypothesis with the highest score is used as the three-dimensional facial network hypothesis. Specifically, the macro-feature scoring may include a shape consistency score and a facial symmetry score. Exemplarily, a standard face template (e.g., an average face shape) is predefined, which contains a typical face contour and facial features. For each 3D face model, the geometric difference between it and the standard face template is calculated. First, the key points of the 3D face model and the standard face template (such as the positions of the eyes, nose, and mouth) are extracted, and then the Euclidean distance between these key points is calculated. According to the distance value, the shape consistency is evaluated. The smaller the distance value, the higher the shape consistency score. Exemplarily, the left and right symmetrical key points of the 3D face model (such as the left eye and the right eye, the left eyebrow and the right eyebrow, the left corner of the mouth and the right corner of the mouth) are first extracted, and then the Euclidean distance between the above symmetrical key points is calculated. According to the distance difference between the key points, the symmetry is evaluated. The smaller the difference, the higher the facial symmetry score. Exemplarily, the shape consistency score and facial symmetry score of each face network hypothesis are added as its macro-feature score, and the face network hypothesis with the highest score is taken as the three-dimensional face network hypothesis.

[0100] In the above embodiment, by inputting the video frames of the video clip into the three-dimensional mesh generation network, a plurality of facial network hypotheses are generated, which can enhance the probability of capturing the correct facial features. By scoring each facial network hypothesis, the facial network hypothesis with the highest score is used as the three-dimensional facial network hypothesis, which provides a strong data basis for projecting it onto a preset two-dimensional plane and performing microscopic and detailed liveness detection.

[0101] See also Figure 8The embodiment of the present application further provides an online signing system 900 for informed consent for clinical trials, and the online signing system 900 for informed consent for clinical trials includes: The video clip acquisition module 910 is used to acquire the video clip of the subject signing the clinical trial informed consent online when the interactive process of the current risk verification step is completed; A hypothesis generation and projection module 920 is used to input the video frames in the video clip into the three-dimensional mesh generation network to generate a hypothesis, obtain a three-dimensional facial network hypothesis of the subject, and project the three-dimensional facial network hypothesis into a preset two-dimensional plane to obtain projection key points and projection texture features of the subject's face; The video frame feature extraction module 930 is used to extract key points and texture features from the video frame to obtain the biometric key points and local micro-texture features of the subject's face; A geometric consistency detection module 940 performs geometric consistency detection based on projected key points, projected texture features, biometric key points, and local micro-texture features to obtain key point alignment errors and texture consistency between the 3D facial network hypothesis and the video frame; The liveness detection result confirmation module 950 is used to proceed to the next risk verification step of the current risk verification step if both the key point alignment error and the texture consistency meet the liveness detection conditions.

[0102] In some embodiments, the online signing system 900 for clinical trial informed consent further includes: An interactive interface display module is used to display a preliminary verification interactive interface when the subject passes the face verification in the informed information display and preliminary verification steps; wherein the preliminary verification interactive interface contains basic information of the clinical trial; The confirmation module is used to confirm the completion of the informed information display and preliminary verification steps if it is detected that the subject has completed the review of the basic information.

[0103] In some embodiments, the online signing system 900 for clinical trial informed consent further includes: The voice playback module is used to play the voice data of the explanation of key terms one by one during the key terms explanation and dynamic behavior verification steps, and send out a first action reminder message; wherein the first action reminder message is used to require the subject to confirm understanding of the key terms through a first specified action behavior.

[0104] In some embodiments, the online signing system 900 for clinical trial informed consent further includes: The subject voice acquisition module is used to obtain the subject's voice data on questions regarding the clinical trial during the question-answer confirmation and deep verification steps; The reply voice generation module is used to generate reply voice data for the question voice data in response to the question voice data, and send a second action reminder message; wherein the second action reminder message requires the subject to perform a second specified action behavior to ensure dynamic participation of the living body.

[0105] In some embodiments, the online signing system 900 for clinical trial informed consent further includes: A signing record generating module, used to generate tamper-proof signing record data in response to the electronic signature input operation in the signing step; The data binding and encryption module is used to bind and encrypt the process data of each of the multiple risk verification steps, the biometrics of the subjects and the signature record data for storage.

[0106] In some embodiments, the online signing system 900 for clinical trial informed consent further includes: The alarm module is used to trigger an abnormal alarm and send a re-verification reminder message if any of the key point alignment error and texture consistency does not meet the living body detection condition.

[0107] In some implementations, the geometric consistency detection module 940 further includes: an alignment error determination unit, used for matching the projected key points with the biometric key points to obtain an average Euclidean distance, and determining the key point alignment error based on the average Euclidean distance; The texture condition determination unit is used to compare the projected texture feature with the local micro-texture feature using the structural similarity index and the cosine similarity to obtain the texture consistency.

[0108] In some implementations, it is assumed that the projection generation module 920 further includes: A video frame input unit, used for inputting video frames in a video clip into a three-dimensional mesh generation network for hypothesis generation, and obtaining a plurality of facial network hypotheses; The scoring unit is used to score each facial network hypothesis, and the facial network hypothesis with the highest score is used as the three-dimensional facial network hypothesis.

[0109] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0110] The online signing system for informed consent for clinical trials in this embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0111] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application, such as Fig. 9 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0112] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0113] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0114] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0115] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0116] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0117] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0118] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0119] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

[0120] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0121] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0122] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0123] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0124] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0125] It is understandable that in the specific implementation of the present application, related data such as user information, location information, navigation data, etc. are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0126] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0127] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0134] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0135] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for online signing of informed consent for clinical trials, characterized in that: The online signing process is divided into a plurality of risk verification steps in a sequential manner, and the method includes: After completing the interactive process of the current risk verification step, obtain the video clip of the subject signing the clinical trial informed consent form online; Inputting the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis, obtaining a three-dimensional facial network hypothesis of the subject, and projecting the three-dimensional facial network hypothesis onto a preset two-dimensional plane to obtain projection key points and projection texture features of the subject's face; Extracting key points and texture features from the video frame to obtain biometric key points and local microtexture features of the subject's face; Performing geometric consistency detection based on the projected key points, the projected texture features, the biometric key points, and the local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame; If both the key point alignment error and the texture consistency satisfy the living body detection condition, proceed to the next risk verification step of the current risk verification step.

2. The method according to claim 1, characterized in that The plurality of risk verification steps include informed information display and preliminary verification steps, and the method further comprises: In the informed information display and preliminary verification step, if the subject passes the face verification, a preliminary verification interactive interface is displayed; wherein the preliminary verification interactive interface contains basic information of the clinical trial; If it is detected that the subject has completed reviewing the basic information, it is confirmed that the informed information display and preliminary verification steps are completed.

3. The method according to claim 1, characterized in that The plurality of risk verification steps include key clause explanation and dynamic behavior verification steps, and the method further includes: In the key clause explanation and dynamic behavior verification step, voice data of the key clause explanation is played one by one, and a first action reminder message is issued; wherein, the first action reminder message is used to require the subject to confirm understanding of the key clause through a first specified action behavior.

4. The method according to claim 1, characterized in that The plurality of risk verification steps include question-answer confirmation and in-depth verification steps, and the method further includes: In the question-answer confirmation and deep verification step, obtaining the voice data of the subject's questions regarding the clinical trial; In response to the interrogation voice data, reply voice data for the interrogation voice data is generated, and a second action reminder message is issued; wherein the second action reminder message requires the subject to perform a second specified action behavior to ensure dynamic participation of the living body.

5. The method according to claim 1, characterized in that A plurality of the risk verification steps include a signing step, and the method further includes: In the signing step, in response to the electronic signature entry operation, signature record data that cannot be tampered with is generated; The process data of each of the plurality of risk verification steps, the biometrics of the subject and the signed record data are bound and stored in an encrypted manner.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: If either the key point alignment error or the texture consistency condition does not satisfy the living body detection condition, an abnormal alarm is triggered and a re-verification reminder message is sent.

7. The method according to any one of claims 1 to 5, characterized in that: The geometric consistency detection is performed based on the projected key points, the projected texture features, the biometric key points and the local micro-texture features to obtain the key point alignment error and texture consistency between the three-dimensional facial network hypothesis and the video frame, including: Matching the projected key points with the biometric key points to obtain an average Euclidean distance, and determining the key point alignment error based on the average Euclidean distance; The projected texture features and the local micro-texture features are compared using a structural similarity index and cosine similarity to obtain the texture consistency.

8. The method according to any one of claims 1 to 5, characterized in that: The step of inputting the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis to obtain a three-dimensional facial network hypothesis of the subject includes: Inputting video frames in the video clip into a three-dimensional mesh generation network for hypothesis generation to obtain a plurality of facial network hypotheses; Each facial network hypothesis is scored, and the facial network hypothesis with the highest score is used as the three-dimensional facial network hypothesis.

9. An online signing system for informed consent of clinical trials, characterized in that: The system comprises: A video clip acquisition module is used to acquire a video clip of the subject signing the clinical trial informed consent online when the interactive process of the current risk verification step is completed; A hypothesis generation and projection module is used to input the video frames in the video clip into a three-dimensional mesh generation network to generate a hypothesis, obtain a three-dimensional facial network hypothesis of the subject, and project the three-dimensional facial network hypothesis into a preset two-dimensional plane to obtain projection key points and projection texture features of the subject's face; A video frame feature extraction module, used to extract key points and texture features from the video frame to obtain biometric key points and local micro-texture features of the subject's face; A geometric consistency detection module performs geometric consistency detection based on the projected key points, the projected texture features, the biometric key points and the local micro-texture features to obtain key point alignment errors and texture consistency between the three-dimensional facial network hypothesis and the video frame; The liveness detection result confirmation module is used to continue the next risk verification step of the current risk verification step if the key point alignment error and the texture consistency both meet the liveness detection conditions.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 8 by executing the computer instructions.

Images