Multi-mode identity authentication method and system based on cloud side-end cooperation

Abstract

The invention belongs to the technical field of identity fusion authentication, and provides a multi-mode identity authentication method and system based on cloud side-end cooperation, and the method comprises the steps: obtaining various identity data; the edge computing node performs data analysis after preprocessing various identity data to obtain a primary authentication result; the edge computing node judges the communication state of the cloud server, and sends a primary authentication result and identity data to the cloud server when the communication is normal; and the cloud server performs product weighting operation on the multiple identity feature data and the primary authentication result to obtain fused identity feature data, obtains corresponding two groups of identity feature data according to two groups of data to be authenticated and storage data after identity registration, compares the two groups of identity feature data, obtains a secondary authentication result according to a comparison result, and sends the secondary authentication result to the cloud server. And feeding back the secondary authentication result to the edge computing node. According to the invention, resources can be reasonably scheduled and allocated, and the authentication performance, efficiency and security are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of identity fusion authentication, and particularly relates to a multi-modal identity authentication method and system based on cloud-edge-end collaboration. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Identity authentication is an important defense line for information security. Multiple information security verifications, large-scale efficient authentication, and unified management of authentication data are three important features of a new generation of identity authentication systems. Among them, multiple information security verifications are the core foundation of the identity authentication system. Applying multiple identity authentication technologies can provide a higher level of security guarantee and effectively cope with various information security threats. Large-scale efficient authentication is the guarantee. Through powerful computing and storage capabilities, it can efficiently process a large number of concurrent identity authentication requests, achieve effective diversion and load balancing of concurrent requests, quickly compare and analyze a large amount of biometric data, and ensure the accuracy and efficiency of authentication. Therefore, improving the security and authentication efficiency of identity authentication is the key link to overcome the new generation of identity authentication systems.

[0004] With the rapid development of digitalization, networking, and intelligence, in recent years, the identity authentication field has vigorously strengthened the implementation and application of various identity authentication technologies. Technologies such as big data centers and multi-modal biometrics have enabled the rapid large-scale application of identity authentication technologies. However, with the sharp increase in the scale of identity authentication data and the rapid improvement of people's security awareness of identity information, the security and authentication efficiency of existing identity authentication technologies are difficult to meet the needs of society. Therefore, deeply integrating new technologies such as intelligent acquisition devices based on the Internet of Things, cloud-edge-end collaborative big data analysis, and deep learning multi-modal fusion with existing identity authentication work is the key to realizing a more secure and efficient identity authentication system.

[0005] The biometric technology industry has covered many important fields such as finance, telecommunications, information security, and e-government. However, the storage requirements of current identity authentication systems are increasing rapidly, the security requirements are constantly improving, and it is gradually difficult to handle a large amount of data processing. Especially for some data with high confidentiality requirements, its security and efficient analysis have not been well guaranteed for a long time. Therefore, it is urgent to accelerate the construction of a multi-modal identity authentication system based on cloud-edge-end collaboration to realize a more secure and efficient identity authentication system. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes a multi-modal identity authentication method and system based on cloud-edge-end collaboration. The present invention can achieve reasonable scheduling and allocation of resources, and improve the performance, efficiency, and security of authentication.

[0007] According to some embodiments, the present invention adopts the following technical solutions: A multi-modal identity authentication method based on cloud-edge-end collaboration, comprising the following steps: Obtain various identity data; After preprocessing various identity data by an edge computing node, perform data analysis to obtain a primary authentication result; The edge computing node judges the communication status with the cloud server, and when the communication is normal, sends the primary authentication result and the identity data to the cloud server; The cloud server performs a product weighting operation on various identity feature data and the primary authentication result to obtain fused identity feature data. According to two groups of data, namely the data to be authenticated and the data stored after identity registration, obtain the corresponding two groups of identity feature data, perform a comparison, and obtain a secondary authentication result according to the comparison result, and feedback the secondary authentication result to the edge computing node.

[0008] As an alternative implementation, the identity data includes at least two of face, handwriting, fingerprint, finger vein, palmprint, palm vein, voiceprint, and digital certificate.

[0009] As an alternative implementation, the process by which the cloud server performs a product weighting operation on various identity feature data and the primary authentication result includes: Normalize the identity feature data of different modalities respectively; Use a multi-modal fusion model to perform weighting by multiplying the comparison scores obtained from the primary authentication result with the normalized identity feature data; The multi-modal fusion model processes the weighted identity feature data to obtain fused identity feature data.

[0010] As a further implementation, the process of normalizing the identity feature data of different modalities respectively includes: the normalization operation expression of the identity feature data of the nth modality is:

[0011] where, represents the minimum value operation, represents the maximum value operation, and n is a positive integer.

[0012] As an alternative implementation, the process of obtaining the corresponding two groups of identity feature data, performing a comparison, and obtaining a secondary authentication result according to the comparison result includes: according to two groups of data, namely the data to be authenticated and the data stored after identity registration, obtain the corresponding two groups of identity feature data, compare two feature data in the two groups of biometric data using cosine similarity to obtain a comparison score; obtain a secondary authentication result according to the comparison score and a set comparison threshold.

[0013] As an alternative embodiment, the multimodal fusion model includes a product weighting operation of multimodal identity features and a primary authentication result connected in sequence, a one-dimensional convolutional layer, a plurality of Mamba modules, and a fully connected layer.

[0014] As a further embodiment, the product weighting operation is as follows:

[0015] where, represents the weighted identity feature; represents the comparison score in the primary authentication result.

[0016] As a further embodiment, the Mamba module includes two branches. The first branch includes a one-dimensional convolutional module, a tanh activation function, a state space model module, and a one-dimensional convolutional module connected in sequence; the second branch includes a one-dimensional convolutional module and a tanh activation function connected in sequence.

[0017] As a further embodiment, the input and output dimensions of the state space model module are both [N, C], and its expression is: ;

[0018] where, is the feature dimension in the identity feature; is the hidden state of the identity feature; are both state matrices; is the input identity feature.

[0019] As a further embodiment, the multimodal fusion model uses the Softmax function to guide the multimodal fusion model to learn relevant features, and the expression is:

[0020] where, represents the fused identity feature of the i-th sample.

[0021] A multimodal identity authentication system based on cloud-edge-end collaboration includes: An identity acquisition device for obtaining various identity data; An edge computing node connected to the identity acquisition device and the cloud server, for preprocessing various identity data, performing data analysis to obtain a primary authentication result; judging the communication status of the cloud server, and sending the primary authentication result and identity data to the cloud server when the communication is normal; A cloud server is used to perform a product weighting operation on multiple identity feature data and a primary authentication result to obtain fused identity feature data. Based on two groups of data, namely the data to be authenticated and the data stored after identity registration, two corresponding groups of identity feature data are obtained and compared. According to the comparison result, a secondary authentication result is obtained and fed back to the edge computing node.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: Aiming at the problems of the existing identity authentication system, such as excessive dependence on the cloud computing resource center and the use of a single identity authentication method, which lead to low efficiency, low bandwidth, low security, and low accuracy, the present invention studies a multi-modal fusion identity authentication technology based on cloud-edge-end collaboration, reasonably schedules and allocates resources, and improves the performance, efficiency, and security of authentication.

[0023] The present invention performs preprocessing operations such as filtering and interpolation on the original data sent by the identity acquisition device to achieve the purposes of removing noise and supplementing missing data. At the same time, the identity data collected by the identity acquisition device is fused and processed, improving the utilization of multi-modal information, reducing the storage requirements of identity data, and increasing the speed and efficiency of identity authentication.

[0024] By fusing multiple identity authentication methods, the present invention improves the accuracy and security of identity authentication, and effectively reduces the risks of identity theft and fraud.

[0025] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, provides a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0027] Figure 1 It is an overall schematic diagram of a multi-modal identity authentication system based on cloud-edge-end collaboration provided by an embodiment; Figure 2 It is an overall schematic diagram of an authentication method for multi-modal identity information fusion provided by an embodiment; Figure 3 It is an overall schematic diagram of a multi-modal fusion model provided by an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The present invention will be further described below in conjunction with the drawings and embodiments.

[0029] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention pertains.

[0030] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0031] In the case of no conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0032] Embodiment 1 A multi-modal identity authentication system based on cloud-edge-end collaboration, as Figure 1 shown, includes a cloud server, at least one edge computing node connected to the cloud server, and an identity acquisition device.

[0033] There are multiple edge computing nodes, which are distributed in the areas required for authentication. The identity acquisition device is installed in the specified authentication scenario and is connected to the edge computing nodes.

[0034] The identity acquisition device is connected to the edge computing node and includes corresponding acquisition devices for eight types of identity information, namely face, handwriting, fingerprint, finger vein, palmprint, palm vein, voiceprint, and digital certificate.

[0035] In this embodiment, at least two of the information are acquired.

[0036] The identity acquisition device acquires identity data such as the face, fingerprint, handwriting, and voice of relevant personnel and transmits the identity data to the edge computing node in real time.

[0037] In this embodiment, only one edge computing node needs to be deployed in the same identity authentication scenario. Its function is to save, process, and analyze the identity data uploaded by various acquisition devices in this scenario, and communicate with the cloud server for data upload and reception.

[0038] The edge computing node includes an edge computing device, which is placed in the identity authentication scenario, used for preprocessing the identity data, preprocessing and preliminarily analyzing the identity data to obtain a primary authentication result, and packaging the processed identity data and the primary authentication result and transmitting them to the central cloud for fusion analysis, and used for monitoring the secondary authentication result feedback from the central cloud and performing the result feedback of the relevant personnel's identity authentication.

[0039] The central cloud, including a cloud server, is placed in the computer room and is used for calculating and storing data, and for fusing and analyzing the identity data transmitted by the edge computing nodes to obtain a secondary authentication result, and feeding back the secondary authentication result to the edge computing nodes.

[0040] The functions of the cloud server include: storing historical primary authentication data, historical identity data, and a multi-identity fusion authentication model; performing calculations and analyses through the high-performance graphics processors installed in the cloud server cluster, and feeding back the final authentication result obtained by fusing multiple identity data based on the primary authentication result to the edge computing nodes.

[0041] The high-performance graphics processors installed in the cloud server have a large memory and a large number of computing cores, providing computing power support for the parallel calculation and analysis of identity data, and multiple graphics processors can execute computing tasks in parallel.

[0042] The identity acquisition device acquires various identity data of relevant personnel and transmits the identity data to the edge computing nodes.

[0043] Embodiment 2 A multi-modal identity information fusion authentication method is applied to a multi-modal identity authentication system based on cloud-edge-end collaboration, as Figure 2 shown, and includes the following steps: S1: The identity acquisition device acquires a total of eight types of identity data and sends the identity data to the edge computing nodes; S2: After preprocessing various identity data, the edge computing node performs data analysis to obtain identity feature data and a primary authentication result; The preprocessing includes operations such as filtering and interpolation.

[0044] For different identity data, different methods can be used to extract feature representations and perform authentication, such as: The digital certificate is authenticated using the public key of the CA; The Haar features are extracted from the face data to represent the identity information, and the cosine distance is used for authentication; The voiceprint and handwriting data are authenticated using dynamic time warping; The finger vein, palmprint, palm vein, and fingerprint data extract minutiae features through structural analysis and are authenticated using the Manhattan distance.

[0045] S3: The edge computing node judges the communication status of the cloud server. When the communication is normal, it sends the primary authentication result and the identity feature data to the cloud server and enters step S4; In this step, the edge computing node sends a request to the cloud server through the TCP protocol. If the communication status is normal, the cloud server will return a response, and the edge computing node can judge whether the communication status is normal by whether it receives the response.

[0046] S4: Based on the result of the first authentication, the cloud server splices the identity features through a multi-modal identity fusion algorithm for fusion and analysis to obtain the result of the second authentication.

[0047] Among them, step S4, which is deployed on the cloud server to obtain the result of the second authentication, includes the following steps: S41: Normalize the identity feature data of different modalities respectively; S42: The multi-modal fusion model weights by multiplying the comparison scores obtained from the result of the first authentication with the normalized identity feature data; S43: The multi-modal fusion model processes the weighted identity feature data to obtain the fused identity feature data; S44: According to two groups of data, namely the data to be authenticated and the data stored after identity registration, obtain the corresponding two groups of identity feature data, compare two feature data in the two groups of biometric data using cosine similarity to obtain a comparison score; obtain the result of the second authentication according to the comparison score and the set comparison threshold.

[0048] When a user starts using the authentication system, they need to register their identity. During the registration process, different identity information of the user is collected, corresponding features are extracted and stored in the cloud server. In subsequent authentication, the edge computing node obtains the corresponding registered feature information from the cloud server for the first identity authentication. The cloud server performs the second authentication based on the comparison of the registered feature information and the identity data to be authenticated on the basis of the first authentication.

[0049] In step S41, normalizing each modality can align the features of different modalities, which is beneficial to the feature fusion of different modalities.

[0050] The normalization operation expression of the identity feature data of the nth modality is:

[0051] Among them, represents the operation of taking the minimum value, represents the operation of taking the maximum value; As Figure 3 shown, in steps S42 and S43, the multi-modal fusion model includes a product weighting operation of multi-modal identity features and the result of the first authentication, a one-dimensional convolutional layer, multiple Mamba modules, and a fully connected layer connected in sequence; The product weighting operation expression in step S42 is:

[0052] Among them, represents the weighted identity feature; Represents the comparison score in the one-time authentication result.

[0053] The Mamba module includes two branches. The first branch includes a one-dimensional convolutional module, a tanh activation function, a state space model module, and a one-dimensional convolutional module connected in sequence; the second branch includes a one-dimensional convolutional module and a tanh activation function connected in sequence. The input and output dimensions of the state space model (SSM) module of the Mamba module are both [N, C], and its expression is: ;

[0054] Among them, Is the feature dimension in the identity feature; Is the hidden state of the identity feature; Are both state matrices; Is the input identity feature.

[0055] The target loss function uses the Softmax function to guide the multi-modal fusion model to learn relevant features, and the expression is:

[0056] Among them, Represents the fused identity feature of the i-th sample.

[0057] In step S44, the cosine similarity is used to measure the relationship between two feature data to obtain the comparison score, and the expression is: ; Among them, and Respectively represent the fused features of two groups of data.

[0058] The secondary authentication result is obtained according to the comparison score and the set comparison threshold.

[0059] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD - ROM , optical memory, etc.).

[0060] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0061] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in one block or multiple blocks.

[0063] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made by those skilled in the art without creative efforts within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-modal identity authentication method based on cloud-edge-terminal collaboration, characterized in that, Including the following steps: Obtain multiple types of identity data; After preprocessing various identity data by the edge computing node, perform data analysis to obtain a primary authentication result; The edge computing node judges the communication status of the cloud server, and when the communication is normal, sends the primary authentication result and identity data to the cloud server; The cloud server performs a product weighting operation on multiple types of identity feature data and the primary authentication result to obtain fused identity feature data. According to two groups of data, namely the data to be authenticated and the data stored after identity registration, obtain the corresponding two groups of identity feature data, perform a comparison, and obtain a secondary authentication result based on the comparison result, and feedback the secondary authentication result to the edge computing node.

2. The multimodal identity authentication method based on cloud-edge-terminal collaboration according to claim 1, wherein, The identity data includes at least two of face, handwriting, fingerprint, finger vein, palmprint, palm vein, voiceprint, and digital certificate.

3. The multimodal identity authentication method based on cloud-edge-terminal collaboration according to claim 1, characterized in that, The process of the cloud server performing a product weighting operation on multiple types of identity feature data and the primary authentication result includes: Normalize the identity feature data of different modalities separately; Use a multi-modal fusion model to perform weighting by multiplying the comparison scores obtained from the primary authentication result with the normalized identity feature data; The multi-modal fusion model processes the weighted identity feature data to obtain fused identity feature data.

4. The multimodal identity authentication method based on cloud-edge-terminal collaboration according to claim 3, wherein, The process of normalizing the identity feature data of different modalities separately includes: The normalization operation expression of the identity feature data of the nth modality is: Among them, represents the minimum value operation, represents the maximum value operation, and n is a positive integer.

5. The multi-modal identity authentication method based on cloud-edge-end collaboration according to claim 1, wherein The process of obtaining the corresponding two groups of identity feature data, performing a comparison, and obtaining a secondary authentication result based on the comparison result includes: According to two groups of data, namely the data to be authenticated and the data stored after identity registration, obtain the corresponding two groups of identity feature data, compare two feature data in the two groups of biometric data using cosine similarity to obtain a comparison score; obtain a secondary authentication result based on the comparison score and a set comparison threshold.

6. The multimodal identity authentication method based on cloud-edge-terminal collaboration according to claim 3, wherein, The multi-modal fusion model includes a product weighting operation of multi-modal identity features and the primary authentication result, a one-dimensional convolutional layer, multiple Mamba modules, and a fully connected layer connected in sequence.

7. The multimodal identity authentication method based on cloud-edge-end collaboration according to claim 6, characterized in that, The product weighting operation is: Among them, represents the weighted identity feature; represents the comparison score in the one-time authentication result.

8. The multimodal identity authentication method based on cloud-edge-terminal collaboration according to claim 7, characterized in that, The Mamba module includes two branches. The first branch includes a one-dimensional convolutional module, a tanh activation function, a state space model module, and a one-dimensional convolutional module connected in sequence; the second branch includes a one-dimensional convolutional module and a tanh activation function connected in sequence. The input and output dimensions of the state space model module are both [N, C], and its expression is: Among them, is the feature dimension in the identity feature; is the hidden state of the identity feature; are both state matrices; is the input identity feature.

9. The multimodal identity authentication method based on cloud-edge-end collaboration according to claim 7, characterized in that, The multi-modal fusion model uses the Softmax function to guide the multi-modal fusion model to learn relevant features, and the expression is: Among them, represents the fused identity feature of the i-th sample.

10. A multi-modal identity authentication system based on cloud-edge-terminal collaboration, characterized in that, Including: An identity acquisition device for obtaining multiple types of identity data; An edge computing node, connected to the identity acquisition device and the cloud server, for preprocessing various identity data and then performing data analysis to obtain a primary authentication result; Judge the communication status of the cloud server, and when the communication is normal, send the primary authentication result and identity data to the cloud server; A cloud server is used to perform a product weighting operation on multiple identity feature data and a primary authentication result to obtain fused identity feature data. According to two groups of data, namely the data to be authenticated and the data stored after identity registration, two corresponding groups of identity feature data are obtained and compared. A secondary authentication result is obtained based on the comparison result, and the secondary authentication result is fed back to the edge computing node.

Images