Method and device for performing user authentication during telemedicine

Abstract

An embodiment of the present invention relates to a method and a device for performing user authentication during telemedicine, and, to a method and a device for performing user authentication during telemedicine, which are capable of: performing user authentication for a patient or medical staff during telemedicine; and linking and storing data about user authentication in electronic health records (EHR) or electronic medical records (EMR) of the patient.

Description

Method and device for performing identity verification during remote medical consultation

[0001] The present invention relates to a method and apparatus for identity verification during remote medical treatment, and more specifically, to a method and apparatus for identity verification during remote medical treatment using facial recognition for the identity verification of a patient or medical staff during remote medical treatment.

[0002] Recently, telemedicine is being actively introduced to enhance the convenience and efficiency of medical services. Telemedicine refers to a system that allows patients to receive consultation and treatment from medical staff using video or voice communication without having to visit a hospital or clinic in person.

[0003] In such a telemedicine environment, accurately verifying the patient's identity is essential. Reliable identity authentication must be ensured to prevent fraudulent activities, such as a third party requesting medical services, unauthorized access to or modification of medical information by someone other than a medical professional, or obtaining medication prescriptions under a false identity.

[0004] Furthermore, from the perspective of medical staff, a procedure is required to authenticate whether the medical professional accessing the telemedicine system is the actual medical professional in order to accurately verify the patient's medical history, disease information, and prescription history, and to provide safe treatment.

[0005] In telemedicine systems, technologies such as ID / password input, one-time authentication codes (OTP), mobile phone text authentication, fingerprint recognition, and simple facial recognition have traditionally been used.

[0006] While the ID / password input method is the most common, it carries a risk of password leakage or theft. Additionally, security can be significantly compromised if passwords are not changed regularly.

[0007] While the One-Time Password (OTP) method is relatively secure because it uses a one-time authentication code, it requires a device capable of receiving the code, which also carries the risk of leakage. Additionally, the fingerprint recognition method has the problem of physical limitations due to the requirement of a separate fingerprint sensor.

[0008] On the other hand, facial recognition has the advantage of being relatively convenient and capable of performing contactless authentication. However, there is a problem in that if the facial image itself is leaked, the user's identity is easily exposed, and furthermore, forgery attacks using the leaked image become possible.

[0009] The method and device for identity verification during remote medical treatment according to an embodiment of the present invention are intended to perform identity verification using facial recognition during remote medical treatment.

[0010] In addition, the method and device for identity verification during remote medical treatment according to an embodiment of the present invention are intended to improve the security of facial recognition data.

[0011] However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist.

[0012] As a technical means for achieving the aforementioned technical task, a method for performing identity authentication of a patient or medical staff during remote medical treatment according to an embodiment of the present invention comprises the steps of: capturing a user's facial image using a shooting device equipped on a user's terminal; transmitting the user's facial image to a facial recognition management server; dividing the user's facial image into a plurality of zones; generating a modified facial image by performing shape modification on one or more of the plurality of zones; extracting reference feature points based on the modified facial image; and generating encrypted security feature point data by performing encryption on the reference feature points.

[0013] In addition, a method for performing identity authentication of a patient or medical staff during remote medical treatment according to an embodiment of the present invention includes the steps of: capturing a facial image of a user using a shooting device equipped on a user terminal; transmitting the user facial image to a facial recognition management server; extracting reference feature points from the user facial image; dividing the reference feature points into a plurality of zones; and generating encrypted security feature point data by performing shape deformation on one or more of the plurality of zones.

[0014] Additionally, a computing device according to an embodiment of the present invention includes at least one memory and at least one processor that executes at least one application stored in the memory to perform security processing of facial recognition data, and the instructions of the processor include the steps of: capturing a facial image of a user using a shooting device provided in a user terminal; transmitting the user facial image to a facial recognition management server; dividing the user facial image into a plurality of regions; generating a deformed facial image by performing shape deformation on one or more of the plurality of regions; extracting reference feature points based on the deformed facial image; and generating security feature point encrypted data by performing encryption on the reference feature points.

[0015] In addition, a computing device according to an embodiment of the present invention comprises at least one memory and at least one processor that executes at least one application stored in the memory to perform security processing of facial recognition data, wherein the instructions of the processor include the steps of: capturing a facial image of a user using a shooting device provided in a user terminal; transmitting the user facial image to a facial recognition management server; extracting reference feature points from the user facial image; dividing the reference feature points into a plurality of regions; and performing shape deformation on one or more of the plurality of regions to generate security feature point encryption data.

[0016] The method and device for identity verification during remote medical treatment according to an embodiment of the present invention can perform identity verification using facial recognition during remote medical treatment.

[0017] In addition, the method and device for identity verification during remote medical treatment according to an embodiment of the present invention can improve the security of facial recognition data.

[0018] FIG. 1 is an exemplary diagram showing the communication connection of a remote medical authentication device according to an embodiment of the present invention.

[0019] FIG. 2 is a configuration diagram of a remote medical identity authentication management server according to an embodiment of the present invention.

[0020] FIG. 3 is a configuration diagram of a terminal according to an embodiment of the present invention.

[0021] FIG. 4 is a conceptual diagram illustrating the functions of a processor according to an embodiment of the present invention.

[0022] FIGS. 5 to 7 are flowcharts of a method for generating reference feature points according to an embodiment of the present invention.

[0023] FIGS. 8 and 9 are flowcharts of a method for authenticating identity during remote medical treatment according to an embodiment of the present invention.

[0024] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0025] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0026] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that all modifications, equivalents, and substitutions included within the concept and technical scope of the present invention are included.

[0027] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0028] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0029] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0030] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0031] Hereinafter, a device for authenticating identity during remote medical treatment according to an embodiment of the present invention will be described with reference to FIG. 1.

[0032] FIG. 1 is an exemplary diagram showing a communication connection between a remote medical authentication management server (100) and a terminal (200) according to an embodiment of the present invention.

[0033] Referring to FIG. 1, the remote medical identity authentication management server (100) is connected to a terminal (200) via a communication network. At this time, the remote medical identity authentication management server (100) refers to a device that performs data processing for identity authentication of a patient or medical staff during remote medical treatment, and may correspond to a computing device, a server device, etc.

[0034] The terminal (200) is a device used to perform remote medical services by relaying data between users (medical staff, patients), and may mean a device capable of transmitting data that is input into the terminal (200) or collected using the terminal (200) to the identity authentication management server (100), or displaying data received from the remote medical identity authentication management server (100).

[0035] During remote medical treatment, the patient and the medical staff can perform remote medical treatment using the terminal (200) each possesses. At this time, for remote medical treatment, the terminal (200) includes a shooting device for capturing an image of the user (patient, medical staff).

[0036] When a patient and medical staff use a terminal (200) to perform remote medical treatment, the terminal (200) can capture an image of the user's face and provide it to a remote medical treatment identity verification management server (100).

[0037] The remote medical service identity verification management server (100) performs identity verification of a patient or medical staff based on a facial image of a user received from a terminal (200), and when identity verification is completed, allows the remote medical service using the terminal (200) to continue.

[0038] On the other hand, the remote medical authentication management server (100) performs authentication of the patient or medical staff based on the facial image of the user received from the terminal (200). If the authentication of the patient or medical staff fails, the server may provide a notification to the terminal (200) that the authentication has failed. Additionally, if the authentication fails multiple times, the remote medical service using the terminal (200) may be blocked, or a warning notification may be provided to the terminal (200).

[0039] The terminal (200) may refer to any type of handheld-based wireless communication device, such as a laptop, desktop, laptop, wireless communication device with guaranteed portability and mobility, or a smartphone, tablet PC, etc., equipped with a web browser, or a device used for remote medical treatment.

[0040] In addition, the communication network illustrated in FIG. 1 can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as any type of wireless network such as a mobile radio communication network or a satellite communication network.

[0041] Hereinafter, the structure of a remote medical authentication management server according to an embodiment of the present invention will be described with reference to FIG. 2.

[0042] FIG. 2 is a structural diagram illustrating the structure of a remote medical authentication management server (100) according to an embodiment of the present invention.

[0043] Referring to FIG. 2, the remote medical identity verification management server (100) includes a communication module (110), memory (120), and a processor (140), and may further include a database (130). The communication module (110) performs information transmission and reception with a terminal (200). The communication module (110) may include a device including hardware and software necessary to transmit and receive signals, such as control signals or data signals, using a wired or wireless connection with another network device.

[0044] The memory (120) stores a remote medical identity verification program or a remote medical identity verification application. The name of the remote medical identity verification program is set for convenience of explanation and does not limit the function of the program by the name itself.

[0045] The memory (120) can store at least one of the information and data input to the communication module (110), the information and data required for the function performed by the processor (140), and the data generated by the execution of the processor (140).

[0046] The term "memory" (120) should be interpreted as a collective term for a non-volatile storage device that retains stored information even when power is not supplied, and a volatile storage device that requires power to retain stored information. Additionally, the memory (120) can perform the function of temporarily or permanently storing data processed by the processor (140). The memory (120) may include magnetic storage media or flash storage media in addition to a volatile storage device that requires power to retain stored information, but the scope of the present invention is not limited thereto.

[0047] The database (130) can store user account information, personal information, user facial images, feature point data for facial recognition, and encryption and decryption related data. The database (130) may constitute a part of the memory (120), but is not necessarily located inside the remote medical identity authentication management server (100), and may be connected to the outside of the remote medical identity authentication management server (100) to perform data transmission and reception using a communication connection.

[0048] Additionally, the database (130) can be configured to store data on a medical institution (hospital) basis.

[0049] The processor (140) is configured to execute a remote medical identity verification program or a remote medical identity verification application stored in memory (120). The processor (140) may include various types of devices for controlling and processing data. The processor (140) may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in the program.

[0050] In one example, the processor (140) may be implemented in the form of a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the invention is not limited thereto.

[0051] The processor (140) is configured to execute a remote medical identity verification program and perform the following functions and procedures.

[0052] When a user performs remote medical treatment using a terminal (200), the processor (140) receives a facial image of the user from the terminal (200) and performs identity verification based on the received facial image.

[0053] Specifically, the processor (140) can divide the user's facial image into multiple zones and perform shape deformation on one or more of the multiple zones to generate a deformed facial image.

[0054] Additionally, the processor (140) extracts reference feature points based on the deformed facial image and performs encryption on the extracted first reference feature points to generate and store secure feature point encryption data.

[0055] Additionally, the processor (140) can receive a facial image of a user from a terminal (200) when the user performs remote medical treatment using the terminal (200) based on security feature point encryption data, extract comparison feature points from the received facial image, and perform user authentication by comparing the extracted comparison feature points with reference feature points.

[0056] Additionally, the processor (140) can set a zone division criterion by combining the geometric features of the centerline of the face, eyes, nose, and mouth included in the user's face image with a random number value.

[0057] Additionally, the processor (140) can perform one or more shape deformations, such as rotation, shading, enlargement, reduction, and distortion, on one or more of the separated multiple zones.

[0058] Additionally, the processor (140) can generate a deformed facial image by applying different types of shape deformation to two or more of the multiple zones.

[0059] The processor (140) can divide the deformed facial image into multiple vector extraction zones and generate a reference feature point by generating a feature point vector for each of the multiple vector extraction zones.

[0060] Additionally, the processor (140) can divide the first reference feature point into multiple regions and perform shape deformation on one or more of the multiple regions to perform encryption of the reference feature point.

[0061] Additionally, the processor (140) can generate security feature point encryption data by extracting reference feature points from a user's facial image, dividing the second reference feature points into multiple zones, and performing shape deformation on one or more of the multiple zones.

[0062] FIG. 3 is a block diagram illustrating the configuration of a terminal (200).

[0063] Referring to FIG. 3, the terminal (200) includes a memory (220), an input / output module (230), and a processor (240), and may further include a communication module (210).

[0064] The communication module (210) can perform information transmission and reception with an external database or an external device. Here, the external device may be the remote medical identity authentication management server (100 in FIG. 1) described above. The memory (220) stores the remote medical identity authentication program. The name of the remote medical identity authentication program is set for convenience of explanation and does not limit the function of the program by the name itself.

[0065] The input / output module (230) can receive information, data, etc. transmitted from the outside to the terminal (200), capture an image of the user, or output information, data, etc. held by the terminal (200) to the outside. For example, the input / output module (230) may include a display, a touchpad, a camera, etc. The processor (240) executes a remote medical identity verification program stored in the memory (220). Additional descriptions of the communication module (210), memory (220), and processor (240) will be replaced by the descriptions of the communication module (110 in FIG. 2), memory (120 in FIG. 2), and processor (140 in FIG. 2) described above with reference to FIG. 2.

[0066] Additionally, the processor (240) of the terminal (200) can execute a remote medical identity verification program to capture a facial image of the user during remote medical treatment and transmit it to the remote medical identity verification management server (100).

[0067] Additionally, the processor (240) of the terminal (200) can provide a notification when identity verification fails or stop remote medical treatment.

[0068] Hereinafter, the operation and function of the processor (140) according to an embodiment of the present invention will be described in detail with reference to FIG. 4.

[0069] FIG. 4 is a conceptual diagram illustrating the functions of a remote medical identity authentication management server according to an embodiment of the present invention.

[0070] Referring to FIG. 4, the processor (140) includes a facial image management module (141), a shape deformation management module (142), a feature point management module (143), and an encryption module (147).

[0071] The facial image management module (141) can perform the function of receiving a user's facial image from the terminal (200) and storing the received facial image in conjunction with the user's account information.

[0072] Additionally, the facial image management module (141) can perform the function of storing the patient's facial image received from the terminal (200) by linking it to the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0073] The shape deformation management module (142) can divide a facial image or feature points derived from a facial image into multiple zones and perform shape deformation on one or more of the divided zones.

[0074] Specifically, the shape deformation management module (142) recognizes faces included in a face image using an algorithm capable of recognizing faces, such as a Hear Cascade classifier, HOG (Histogram of Oriented Gradients) based detection, and MTCNN, and sets a distinction criterion by combining the geometric features of the recognized face centerline, eyes, nose, and mouth with random values ​​in the user's face image.

[0075] The shape deformation management module (142) establishes a centerline of the face based on the direction connecting the center point of the two eyes, the tip of the nose, and the tip of the chin. The shape deformation management module (142) can establish a reference coordinate system within the face through normalization.

[0076] Normalization refers to the process of correcting cases where the face is tilted or captured at various sizes to achieve a consistent size and rotation. This involves performing horizontal rotation alignment based on the positions of the two eyes or using preprocessing algorithms to adjust the face size to a uniform pixel size.

[0077] The shape deformation management module (142) sets a basic dividing line based on the distance, angle, or coordinate between the facial centerline and the eyes, nose, and mouth, and can additionally dynamically adjust the position or angle by applying a random number.

[0078] The shape deformation management module (142) distinguishes left and right based on the facial centerline and sets approximate division zones in the upper and lower directions, such as the forehead / eye area, nose area, mouth area, and chin area. That is, it sets a reference dividing line based on a line segment or boundary connecting the facial centerline and geometric reference points such as the eyes, nose, and mouth.

[0079] The shape deformation management module (142) can set geometric reference points by extracting the center coordinates or maximum / minimum coordinates of the facial centerline and eyes, nose, mouth, etc., and can store or manage information regarding the geometric reference points in a separate table.

[0080] The shape transformation management module (142) generates a plurality of random numbers. At this time, the random numbers are determined by a seed value, and the seed can be set differently per user or per session. In addition, the random numbers can be generated using a cryptographic random number generator to enhance security and can be set to have a real number range between 0 and 1, or a value in the range of -α to α.

[0081] And, the shape deformation management module (142) performs deformation of the reference dividing line using the following mathematical formula 1.

[0082] [Mathematical Formula 1]

[0083]

[0084] In the aforementioned mathematical formula 1 represents the coordinates of the dividing line modified by applying a random number, L represents the coordinates of the reference dividing line, and r is a random number r (0≤r≤1), represents the pre-set maximum movement value.

[0085] At this time, the dividing line may be in the form of a curve rather than a straight line, and in this case, the shape deformation management module (142) may divide multiple zones into more complex shapes by applying a random number-based offset to the control points of the curve.

[0086] The shape deformation management module (142) applies the deformed dividing line derived using mathematical formula 1 to the face image to divide the face image into multiple zones, and performs shape deformation on the divided zones.

[0087] The shape deformation management module (142) can perform one or more shape deformations, such as rotation, shading, enlargement, reduction, and distortion, on one or more of the divided multiple zones. Additionally, the shape deformation management module (142) can apply different types of shape deformations to the divided multiple zones.

[0088] The shape deformation management module (142) can select one or more of the multiple zones by applying a random number or separate key information and perform a rotation transformation (an operation to rotate the zone to a specific angle) on the zone, or perform transformations such as mirroring (a left-right flipping operation) and scaling (size adjustment).

[0089] Additionally, the shape deformation management module (142) can perform the function of generating a deformed facial image by applying the shape-deformed area, and storing the generated deformed facial image by linking it with the user's account information or storing the patient's facial image by linking it to the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0090] At this time, parameters used for image restoration and authentication, such as random values ​​for section division, basic dividing lines, modified dividing lines, and modified section information (rotation angle, scale, flip status, modified section information), can be encrypted and stored together.

[0091] As described above, by dividing the facial image into multiple regions and performing shape deformation, it is difficult to easily reproduce the original face using only a part of the image. Furthermore, if the server and the client share a random seed or region division information, it is possible to effectively block attempts at spoofing by verifying whether it corresponds exactly to the original user's facial structure.

[0092] The feature point management module (143) extracts reference feature points and comparison feature points for face recognition based on the user's face image or modified face image. A reference feature point refers to a feature point extracted from a face image set by the user to establish initial face authentication, which serves as a reference for face recognition.

[0093] Comparison feature points refer to feature points extracted from a facial image captured by a user for identity verification, which are used to perform identity verification by comparing them with reference feature points.

[0094] The feature point management module (143) extracts feature points for facial recognition based on the user's facial image or the deformed facial image generated using the shape deformation management module (142).

[0095] The feature point management module (143) inputs the user's face image into a feature point extraction algorithm such as CNN (Convolutional Neural Networks), LBP (Local Binary Patterns), SIFT (Scale-Invariant Feature Transform), etc., to extract feature points from the face image.

[0096] Then, the feature point management module (143) generates face embeddings using deep learning embedding models (FaceNet, ArcFace, CosFace, etc.) and expresses the feature points in vector form.

[0097] The feature point management module (143) sets the feature point (vector) derived based on the reference facial image initially entered by the user for identity verification as the reference feature point (vector), and stores the reference feature point (vector) information by linking it with the user's account information or by linking it with the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0098] And, the feature point management module (143) sets the feature point (vector) derived based on the comparison facial image taken by the user who connected for remote medical treatment using the terminal (200) as the comparison feature point (vector), and performs identity authentication by comparing the reference feature point and the comparison feature point.

[0099] At this time, the feature point management module (143) performs identity authentication by comparing the reference feature point and the comparison feature point using similarity measurement methods such as cosine similarity and Euclidean distance to measure the similarity between the reference feature point and the comparison feature point.

[0100] On the other hand, the feature point management module (143) extracts feature points (vectors) using a CNN (Convolutional Neural Networks) trained based on a deformed facial image, or using a model such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), LBP (Local Binary Patterns), and Rotation Invariant LBP (RI-LBP).

[0101] In the case of the SIFT model, it is invariant to changes in scale and rotation, and since it detects specific keypoints and generates feature points and feature vectors based on the pixel distribution around those keypoints, it can extract feature points based on deformed facial images.

[0102] Models such as SURF, ORB, LBP, and Rotation Invariant LBP can also extract features based on deformed images, and since they extract features by analyzing local pixel patterns, they can extract feature points based on deformed facial images.

[0103] The encryption module (147) can divide a reference feature point into multiple regions and perform shape transformation on one or more of the divided regions to perform encryption of the feature point or decryption of the encrypted feature point.

[0104] Additionally, the encryption module (147) can perform encryption on the feature point data by performing shape transformation on the reference feature points (vectors) derived from the user's face image or modified face image.

[0105] The encryption module (147) divides the derived reference feature point data (vector, coordinate information) into multiple zones and generates security feature point data by performing a rotation transformation (an operation to rotate the zone by a specific angle) or performing a transformation such as mirroring (a left-right flipping operation) or scaling (size adjustment) on one or more of the divided zones.

[0106] Security feature point data and parameters for generating security feature points (transformation zone data, random number, key, scale, offset, etc.) can be stored linked to the user's account information or linked to the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0107] Specifically, the feature point management module (143) can transform the embedding vector by multiplying it by a specific rotation matrix R(n / xn) when the embedding vector exists in an n-dimensional space. For example, a 32 / x32 rotation matrix is ​​applied only to region A (1 to 32 dimensions), and the rotation matrix must be an orthonormal matrix and can be generated using a random number-based or key-based method.

[0108] In addition, the encryption module (147) can apply scaling and shifting by applying a constant scale s and an offset b to the vector elements in each zone as shown in mathematical formula 2 below.

[0109] [Mathematical Formula 2]

[0110]

[0111] represents the embedding vector value included in the deformation target area, and represents the embedding vector value to which scaling and shifting have been applied. The scales s and offset b are determined by random numbers or managed in an encrypted form using a secure key.

[0112] Hereinafter, a method for authenticating identity during remote medical treatment according to an embodiment of the present invention will be described in detail with reference to FIG. 5.

[0113] FIGS. 5 to 7 are flowcharts of a method for generating reference feature points according to an embodiment of the present invention.

[0114] According to an embodiment of the present invention, there are two methods for generating reference feature points: one for extracting reference feature points after performing shape deformation on a facial image, and another for extracting reference feature points based on a general facial image.

[0115] Referring to FIG. 5, a method for extracting reference feature points after performing shape deformation of a facial image includes a facial data collection step (S100), a deformed facial image generation step (S200), and a reference feature point extraction step (S300).

[0116] In the facial data collection step (S100), the remote medical identity authentication management server (100) collects a reference facial image taken by a user using a terminal (200) to set up facial authentication.

[0117] In the modified facial image generation step (S200), the remote medical identity verification management server (100) divides the facial image into multiple zones and performs shape modification on one or more of the divided multiple zones.

[0118] In the modified facial image generation step (S200), the remote medical authentication management server (100) recognizes a face included in the facial image using an algorithm capable of recognizing a face, such as a Hear Cascade classifier, HOG (Histogram of Oriented Gradients) based detection, MTCNN, etc., and sets a distinction criterion by combining the geometric features of the recognized facial centerline, eyes, nose, and mouth with a random number value in the user's facial image.

[0119] In the deformed facial image generation step (S200), the remote medical identity verification management server (100) sets the centerline of the face based on the direction connecting the center point of the two eyes, the tip of the nose, and the tip of the chin. The shape deformation management module (142) can set a reference coordinate system within the face through normalization.

[0120] In this context, normalization refers to the process of correcting cases where the face is tilted or captured at various sizes to achieve a consistent size and rotation. This involves performing horizontal rotation alignment based on the positions of the two eyes or using preprocessing algorithms to adjust the face size to a uniform pixel size.

[0121] In the modified facial image generation step (S200), the remote medical authentication management server (100) sets a basic dividing line based on the distance, angle, or coordinate between the facial centerline and the eyes, nose, and mouth, and additionally applies a random number to dynamically adjust the position or angle.

[0122] In the modified facial image generation step (S200), the remote medical identity authentication management server (100) distinguishes the left and right sides based on the facial centerline and sets approximate division areas in the upper and lower directions, such as the forehead / eye area, nose area, mouth area, and chin area. That is, a reference dividing line is set based on a line segment or boundary connecting the facial centerline and geometric reference points such as the eyes, nose, and mouth.

[0123] In the modified facial image generation step (S200), the remote medical authentication management server (100) can set geometric reference points by extracting the center coordinates or maximum / minimum coordinates of the facial centerline and eyes, nose, mouth, etc., and can store or manage information regarding the geometric reference points in a separate table.

[0124] In the modified facial image generation step (S200), the remote medical identity authentication management server (100) generates a plurality of random numbers. At this time, the random numbers are determined by a seed value, and the seed can be set differently for each user or session. In addition, the random numbers can be generated using a cryptographic random number generator to enhance security and can be set to have a real number range between 0 and 1, or a value in the range of -α to α.

[0125] And, in the modified facial image generation step (S200), the remote medical identity authentication management server (100) performs modification of the reference dividing line using the mathematical formula 1 described above.

[0126] The dividing line may be in the form of a curve rather than a straight line, and in this case, the shape deformation management module (142) may divide multiple zones into more complex shapes by applying a random number-based offset to the control points of the curve.

[0127] In the modified facial image generation step (S200), the remote medical identity authentication management server (100) applies a modified dividing line derived using mathematical formula 1 to the facial image to divide the facial image into multiple zones, and performs shape deformation on the divided zones.

[0128] In the modified facial image generation step (S200), the remote medical identity authentication management server (100) can perform one or more shape modifications, such as rotation, shading, enlargement, reduction, and distortion, on one or more of the divided multiple zones. Additionally, the shape modification management module (142) can apply different types of shape modifications to the divided multiple zones.

[0129] In the modified facial image generation step (S200), the remote medical authentication management server (100) may select one or more of the multiple zones by applying a random number or separate key information and perform a rotation transformation (an operation to rotate the zone to a specific angle) on the zone, or perform transformations such as mirroring (a left-right flipping operation) or scaling (size adjustment).

[0130] And, in the modified facial image generation step (S200), the remote medical authentication management server (100) generates a modified facial image by applying a shape-deformed area, and stores the generated modified facial image by linking it with the user's account information or stores the patient's facial image by linking it to the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0131] At this time, parameters used for image restoration and authentication, such as random values ​​for section division, basic dividing lines, modified dividing lines, and modified section information (rotation angle, scale, flip status, modified section information), can be encrypted and stored together.

[0132] In the reference feature point extraction step (S300), the remote medical identity authentication management server (100) extracts reference feature points from a deformed facial image using a CNN (Convolutional Neural Networks) trained based on the deformed facial image, or using a model such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), LBP (Local Binary Patterns), and Rotation Invariant LBP (RI-LBP).

[0133] The extracted basic feature point data is used to generate face embeddings using deep learning embedding models (FaceNet, ArcFace, CosFace, etc.) and the feature points are stored in the form of vectors. The remote medical authentication management server (100) stores the reference feature point (vector) information by linking it with the user's account information or by linking it with the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0134] Referring to FIG. 6, a method for extracting reference feature points from a facial image includes a facial data collection step (S100), a reference feature point extraction step (S301), and a secure feature point encryption data generation step (S401).

[0135] In the reference feature point extraction step (S301), the remote medical authentication management server (100) extracts reference feature points based on the user's facial image obtained from the terminal (200).

[0136] In the reference feature point extraction step (S301), the remote medical authentication management server (100) inputs the user's facial image into a feature point extraction algorithm such as CNN (Convolutional Neural Networks), LBP (Local Binary Patterns), SIFT (Scale-Invariant Feature Transform), etc., to extract reference feature points from the facial image.

[0137] And, the remote medical identity authentication management server (100) generates face embeddings using a deep learning embedding model (FaceNet, ArcFace, CosFace, etc.) and expresses reference feature points in the form of vectors.

[0138] The reference feature points derived in the reference feature point extraction step (S301) are also stored by linking with the user's account information or by linking with the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR) to store the patient's facial image.

[0139] In the security feature point encryption data generation step (S401), the remote medical identity authentication management server (100) divides the reference feature point into multiple regions and performs shape transformation on one or more of the divided regions to perform encryption of the feature point.

[0140] In the security feature point encryption data generation step (S401), the remote medical authentication management server (100) divides the reference feature point data (vector, coordinate information) into multiple zones, and generates security feature point data by performing a rotation transformation (an operation to rotate the zone by a specific angle) or performing a transformation such as mirroring (a left-right flipping operation) or scaling (size adjustment) on one or more of the divided zones.

[0141] In the security feature point encryption data generation step (S401), the remote medical identity authentication management server (100) can transform the embedding vector by multiplying it by a specific rotation matrix R(n / xn) if the embedding vector exists in an n-dimensional space. For example, a 32 / x32 rotation matrix is ​​applied only to area A (1 to 32 dimensions), and the rotation matrix must be an orthonormal matrix and can be generated using a random number-based or key-based method.

[0142] In addition, scaling and shifting can be applied to the separated areas by applying the aforementioned mathematical formula 2.

[0143] Security feature point data and parameters for generating security feature points (transformation zone data, random number, key, scale, offset, etc.) can be stored linked to the user's account information or linked to the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0144] Additionally, as illustrated in FIG. 7, after the reference feature point extraction step (S300), a security feature point encryption data generation step (S401) can be performed to perform encryption on the base feature points extracted based on the deformed facial image.

[0145] Hereinafter, a method for authenticating identity during remote medical treatment according to an embodiment of the present invention will be described in detail with reference to FIGS. 8 and 9.

[0146] Referring to FIG. 8, the method for authenticating identity during remote medical treatment includes a remote medical treatment execution step (S110), a facial data collection step (S210), a comparison feature point extraction step (S310), a security feature point encrypted data decryption step (S410), and a step for deriving the match rate between reference feature points and comparison feature points (S500).

[0147] In the remote medical treatment execution step (S110), the terminal (200) detects whether a remote medical treatment program (application) is executed using the terminal (200) or whether remote medical treatment is executed using a remote medical treatment channel.

[0148] In the facial data collection step (S210), the terminal (200) captures a facial image of the user for identity verification when the user runs a remote medical treatment program, and transmits the captured facial data to the remote medical treatment identity verification management server (100).

[0149] In the comparison feature point extraction step (S310), the remote medical identity authentication management server (100) extracts comparison feature points from the facial image of the user taken for identity authentication.

[0150] In the comparison feature point extraction step (S310), reference feature points are extracted from a facial image by inputting it into a feature point extraction algorithm such as CNN (Convolutional Neural Networks), LBP (Local Binary Patterns), or SIFT (Scale-Invariant Feature Transform), just like in the reference feature point extraction step (S310) described above.

[0151] In the security feature point encrypted data decryption step (S410), the remote medical authentication management server (100) performs decryption of the security feature point encrypted data based on the security feature point data stored in conjunction with the user's account information or stored in conjunction with the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR), and parameters for generating security feature points (variation zone data, random number, key, scale, offset, etc.) to derive basic feature points.

[0152] In the step of deriving the match rate between the reference feature point and the comparison feature point (S500), the remote medical authentication management server (100) derives the match rate between the reference feature point and the comparison feature point, and completes the authentication if the match rate is greater than or equal to a preset threshold value. On the other hand, in the step of deriving the match rate between the reference feature point and the comparison feature point (S500), if the match rate is less than or equal to a preset threshold value, the remote medical authentication management server (100) provides an authentication failure notification to the terminal (200), or if the authentication fails more than a preset number of times, stops the operation of the remote medical program running on the terminal (200).

[0153] Additionally, if the security feature point encryption data stored in the user account is generated based on a modified facial image, a modified facial image generation step (S211) is additionally performed after the facial data collection step (S210) as illustrated in FIG. 9.

[0154] In the modified facial image generation step (S211), the facial image of a user for identity authentication is divided into multiple zones in the same manner as in the modified facial image generation step (S300) described above, and shape modification is performed on one or more of the divided zones.

[0155] In the modified facial image generation step (S211), the remote medical authentication management server (100) generates a modified facial image using the user's facial image for authentication by using parameters (random value for zone division, basic dividing line, modified dividing line, modified zone information) in the user's account information or the patient's Electronic Health Records (EHR) or Electronic Medical Record (EMR).

[0156] In the comparison feature point extraction step (S311), the deformed face image generated in the deformed face image generation step (S211) is input into a feature point extraction model such as CNN (Convolutional Neural Networks), SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), LBP (Local Binary Patterns), and Rotation Invariant LBP (RI-LBP), in the same way as the reference feature point extraction step (S300) described above, to extract comparison feature points.

[0157] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0158] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

[0159] The mode for carrying out the invention is the same as the best mode for carrying out the invention described above.

[0160] The present invention relates to a method and device for identity verification during remote medical treatment, and since it is applicable to medical institutions and medical-related industries, it has industrial applicability.

Claims

1. In a method for performing identity verification of a patient or medical staff during telemedicine, A step of capturing a facial image of the user using a shooting device equipped on the user's terminal, A step of transmitting the above user face image to a face recognition management server, A step of dividing the above user facial image into multiple zones, A step of generating a deformed facial image by performing shape deformation on one or more of the plurality of zones above, A step of extracting reference feature points based on the above-mentioned deformed facial image, and Step of generating secure feature point encrypted data by performing encryption on the above reference feature points A method for verifying identity using facial recognition during remote medical treatment, including 2. In Paragraph 1, The step of dividing the above user facial image into multiple zones is: A step of establishing a distinction criterion by combining geometric features of the facial centerline, eyes, nose, and mouth included in the user facial image with random values, A facial recognition data security method including 3. In Paragraph 2, The step of generating the above-mentioned deformed facial image is, A step of performing a shape transformation among rotation, shading, enlargement, reduction, and distortion on one or more of the above plurality of zones. A method for verifying identity using facial recognition during remote medical treatment, including 4. In Paragraph 3, The step of generating the above-mentioned deformed facial image is, A step of generating the deformed facial image by applying different shape deformation methods to two or more of the plurality of zones. A method for verifying identity using facial recognition during remote medical treatment, including 5. In Paragraph 4, The step of extracting the above reference feature points is, A step of deriving reference feature points from the transformed facial image using one or more of the SIFT (Scale-Invariant Feature Transform) model, SURF (Speeded-Up Robust Features) model, ORB (Oriented FAST and Rotated BRIEF) model, LBP (Local Binary Patterns) model, and Rotation Invariant LBP (RI-LBP) model. A method for verifying identity using facial recognition during remote medical treatment, including 6. In Paragraph 1, The step of generating security feature point encryption data is, A step of dividing the above reference feature points into multiple regions, and A step of generating security feature point encryption data by performing shape transformation on one or more of the aforementioned multiple regions. A method for verifying identity using facial recognition during remote medical treatment, including 7. In a method for performing identity verification of a patient or medical staff during telemedicine, A step of capturing a facial image of the user using a shooting device equipped on the user's terminal, A step of transmitting the above user face image to a face recognition management server, A step of extracting reference feature points from the above user facial image, A step of dividing the above reference feature points into multiple zones, and A step of generating security feature point encryption data by performing shape deformation on one or more of the above plurality of zones, A method for verifying identity using facial recognition during remote medical treatment, including 8. A device that performs identity verification of a patient or medical staff during remote medical treatment, At least one memory, and It includes at least one processor that executes at least one application stored in the memory to perform security processing of facial recognition data, and The instructions of the above processor are, A step of capturing a facial image of the user using a shooting device equipped on the user's terminal, A step of transmitting the above user face image to a face recognition management server, A step of dividing the above user facial image into multiple zones, A step of generating a deformed facial image by performing shape deformation on one or more of the plurality of zones above, A step of extracting reference feature points based on the above-mentioned deformed facial image, and A computing device comprising an instruction that performs the step of generating secure feature point encrypted data by performing encryption on the above reference feature points.

9. A device that performs identity verification of a patient or medical staff during remote medical treatment, At least one memory, and It includes at least one processor that executes at least one application stored in the memory to perform security processing of facial recognition data, and The instructions of the above processor are, A step of capturing a facial image of the user using a shooting device equipped on the user's terminal, A step of transmitting the above user face image to a face recognition management server, A step of extracting reference feature points from the above user facial image, A step of dividing the above reference feature points into multiple zones, and A step of generating security feature point encryption data by performing shape deformation on one or more of the above plurality of zones, A computing device containing instructions to perform.

Images