Homomorphic encryption method for palm print data and identification network thereof
By introducing the SE attention mechanism and BFN layer in the ShuffleNet v2 network, and using generalized dual random matrix encryption technology, homomorphic encryption of the palm print recognition system is solved, and the security risks brought about by plain text storage and deep learning model attacks are solved. , achieving efficient privacy protection and security improvement.
Patent Information
- Application Number
- CN202411336297.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2024-12-27
AI Technical Summary
In the existing palm pattern recognition system, the palm pattern data stored in plain text poses security risks and is vulnerable to hacker attacks and data leakage. Attacks from deep learning models may lead to reduced system accuracy and reliability.
The ShuffleNet v2 network is used as the basis, and the SE attention mechanism is added, the BN layer is replaced with the BFN layer, and the generalized dual random matrix encryption technology is combined to homomorphically encrypt the palm print data and identification network to reduce the risk of data and model parameters exposure.
Without affecting the recognition accuracy, the exposure risk of palm print data and model parameters is significantly reduced, the confidentiality and security of the system is improved, the ability to resist attacks is enhanced, and the reliability and security of the palm print recognition system is ensured. .
Smart Images
Figure CN119206801A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of network security and computer vision, and in particular to a homomorphic encryption method for palmprint data and its recognition network. Background Art
[0002] As an important branch of the field of biometrics, palmprint recognition technology has flourished in recent years. Its uniqueness lies in the fact that palmprints contain rich texture information and have a high degree of individual feature differences, which can greatly improve the convenience and accuracy of recognition. Secondly, palmprint information can be collected contactlessly and performs well in environments with high hygiene requirements. In addition, palmprint recognition is highly private and difficult to forge or tamper with, so it is highly favored in terms of security.
[0003] At present, palmprint recognition systems usually store the user's palmprint data in the database in plain text. This storage method has serious security risks. First, plain text storage makes palmprint data vulnerable to hacker attacks or data leakage. Once unauthorized access occurs, the user's privacy will be greatly violated, which may lead to risks such as identity theft and financial fraud. Second, plain text storage increases the possibility of data tampering or forgery, making the system vulnerable to fraudulent attacks, affecting the accuracy and reliability of recognition.
[0004] In addition, as research progresses, attacks against deep learning models continue to emerge, and attackers may be able to master the internal mechanisms and algorithms of the palmprint recognition system, gain a deep understanding of the system's weaknesses and vulnerabilities, and then conduct targeted attacks, for example, by constructing adversarial samples or malicious inputs to bypass the recognition system's verification mechanism. Secondly, parameter leakage may cause the palmprint recognition system model to be stolen or tampered with, resulting in reduced accuracy and reliability of the system, affecting the user's normal use and experience. Summary of the invention
[0005] In view of the above technical problems, the present invention provides a homomorphic encryption method for palmprint data and its recognition network. The homomorphic encryption method for palmprint data and its recognition network encrypts the original image by a specific image key before storing the palmprint data, and uses the network key to encrypt the palmprint recognition network used by the system layer by layer, which greatly reduces the risk of palmprint data and model parameter exposure without losing recognition accuracy.
[0006] At the same time, considering that the recognition network needs to be embedded in mobile devices, the ShuffleNet network is selected as the basic network and improved to achieve faster recognition speed, smaller parameter size and higher recognition accuracy.
[0007] To this end, the technical solution of the present invention is a homomorphic encryption method for palmprint data and its recognition network, comprising the following steps:
[0008] Step (1): Taking ShuffleNet v2 as the basic network, adding SE attention mechanism and replacing the BN layer in the network with the BFN layer, thus designing the palmprint recognition network ShuffleNet_SE;
[0009] Step (2): Use the palm region of interest extracted from the specified palmprint recognition dataset as the training dataset, train the palmprint recognition network ShuffleNet_SE, and use the Adam optimizer to execute the gradient descent strategy;
[0010] Step (3): Encrypt the palmprint image and recognition network. The encryption operation uses a generalized double random matrix as the basis of the encryption function. For the palmprint recognition network N(x), there are multiple linear layers, and each layer can be simplified to the product form with the weight matrix, as follows:
[0011] N(x;W)=∏ k W k x (1)
[0012] Among them, k represents the number of network layers, W k represents the weight matrix of the k-th layer network, and x represents the input value;
[0013] Given a generalized doubly random matrix A, for each linear layer, the weight matrix W k Can be Therefore, formula (1) can be converted to:
[0014]
[0015] Among them, A0 is the image key, A i is the network key of layer i;
[0016] There is a generalized sparse matrix A in the nonlinear activation function g in the network. The final result of exchanging the calculation order with function g is:
[0017] Ag((A -1 x)) = g(AA -1 x)=g(x) (3)
[0018] The matrix set satisfying formula (3) is denoted as A * Then, the convolutional network composed of linear layers and nonlinear activation functions is encrypted to obtain the encrypted network, N i represents the i-th layer of the palmprint recognition network N(x), and the i-th layer network key A i∈A * By from A * Randomly obtain, the encryption formula is:
[0019]
[0020] Among them, the network key A i Choose a sparse matrix and limit the density of the matrix through the privacy parameter α, that is, A i The number of non-zero elements in does not exceed the privacy parameter α. The value of the privacy parameter α can be specified as any integer. When the correct network key and image key are used, the size of the privacy parameter α has no effect on the network recognition accuracy. When the network key or image key used does not match, as α increases, the confidentiality and security of the system also increase.
[0021] Preferably, step (1) comprises the following steps:
[0022] Step (1-1): Input the specified palmprint image, which goes through five stages, among which the second, third and fourth stages are composed of downsampling / upsampling modules and multiple ordinary modules;
[0023] Step (1-2): split the channel of the feature matrix of the input palmprint image into two branches through multiple common modules, and process each branch separately. This process corresponds to the Channel Split operation;
[0024] In one of the branches, the features of the palmprint image are compressed and extracted through the bottleneck structure. After the extraction is completed, the output of this branch is spliced with the feature matrix of another branch. The spliced feature matrix is remixed through the Channel Shuffle operation to ensure sufficient interaction of information between different channels.
[0025] The bottleneck structure includes: channel compression layer, deep convolution layer, channel recovery layer and a SE module;
[0026] Step (1-3): The downsampling / upgrading module includes two branches, namely, a downsampling module and an upgrading module. The downsampling module branch and the upgrading module branch respectively extract features from the input feature image, and then perform splicing and ChannelShuffle operations;
[0027] Step (1-4): Based on the original bottleneck structure, the second BN operation is replaced by the BFN operation, thereby replacing the BN layer in the network with the BFN layer, blocking the cumulative effect of the estimation offset and alleviating the degradation of network performance;
[0028] The channel attention mechanism structure in the network includes Squeeze and Excitation. The Squeeze operation is used to compress spatial information, compressing the original C*H*W dimensional information into C*1*1.
[0029] The Excitation operation uses two fully connected layers. One of the fully connected layers reduces the dimension of the C*1*1 feature map to C / R*1*1 by using the ReLU activation function. The other fully connected layer remaps the feature back to C*1*1 without using the ReLU activation function. Then, the weight coefficient of each channel is obtained through sigmoid, and finally the weight coefficient is multiplied by the original feature to complete the calculation.
[0030] Preferably, in step (1-2), the Channel Shuffle operation includes the following steps:
[0031] Step (1-2-1): Divide the channels of the feature image into several groups according to certain rules;
[0032] Step (1-2-2): rearrange or shuffle the channels in each group so that the originally adjacent channels are farther apart, thereby achieving information exchange and integration between channels;
[0033] Step (1-2-3): The rearranged or shuffled channels are recombined into a new feature map, wherein the new feature map contains information from different channels, and the rearranged or shuffled feature image has better representation ability.
[0034] Preferably, the channel compression layer and the channel restoration layer are both 1x1 group convolutions, and the depth convolution layer is a 3x3 group depthwise separable convolution.
[0035] Preferably, step (1-3) comprises the following steps:
[0036] Step (1-3-1): In the downsampling module branch, the feature map is first processed through the deep convolution layer to effectively reduce the spatial dimension and retain the channel dimension, and then the number of channels is adjusted through the channel compression layer to control the amount of calculation;
[0037] Step (1-3-2): In the dimension-increasing module branch, the number of channels is first adjusted through the channel compression layer, and the nonlinear expression ability is improved through the ReLU activation function. Then, the feature expression is enhanced while keeping the dimension of the input feature map unchanged through the deep convolution layer, and then the dimension is adjusted through a channel recovery layer. Finally, the ReLU activation function and SE module are used to further improve the expression ability of the model.
[0038] Step (1-3-3): At the end of the module, the extraction results of the two branches are spliced, and the channels are rearranged through the ChannelShuffle operation to ensure that the features between the channels are fully mixed.
[0039] Preferably, step (3) comprises the following steps:
[0040] Step (3-1): Determine the privacy parameter α according to the specific requirements of the system;
[0041] Step (3-2): Encrypt the linear layer of the attention mechanism of the palmprint recognition network ShuffleNet_SE. According to the privacy parameter α and the parameter scale of the first linear layer, calculate all generalized sparse matrices A^* that satisfy formula (3), where the function g is the ReLU activation function. Randomly select A_0 from A^* and multiply it with the original image as the image key to complete the encryption operation of the palmprint image.
[0042] Step (3-3): For each linear layer in the network, find its corresponding A^* according to the steps in step (3-2) and encrypt it according to the algorithm shown in formula (4).
[0043] The beneficial effect of the present invention is that the homomorphic encryption method integrates the SE attention mechanism on the basis of the original architecture of ShuffleNet v2 and replaces the BN layer with the BFN layer, thereby designing a ShuffleNet_SE network, which can effectively reduce the consumption of computing resources while maintaining recognition accuracy and adapt to resource-constrained device environments;
[0044] At the same time, this method proposes an encryption method for palmprint images and recognition networks based on generalized double random matrices, which can significantly reduce the risk of palmprint data and model parameter exposure. In this process, the flexible adjustment of the privacy parameter α further enhances the confidentiality and security of the system.
[0045] Through the system design of this homomorphic encryption method, the privacy and security of palmprint data can be effectively protected without affecting the recognition accuracy, and the network's anti-attack capability is improved, ensuring the reliability and security of the palmprint recognition system in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of the homomorphic encryption process and the network identification process in the present invention;
[0047] Figure 2 It is a structural block diagram of the palmprint recognition network in the present invention;
[0048] Figure 3It is a schematic diagram of the Channel Shuffle operation in the present invention;
[0049] Figure 4 It is a schematic diagram of the attention mechanism structure of the palmprint recognition network in the present invention. DETAILED DESCRIPTION
[0050] The present invention will be further described below in conjunction with embodiments.
[0051] pass Figure 1-Figure 4 It can be seen that the homomorphic encryption method of the palmprint data and its recognition network includes the following steps:
[0052] Step (1): Taking ShuffleNet v2 as the base network, adding SE attention mechanism, and replacing the BN layer in the network with the BFN layer, thus designing the palmprint recognition network ShuffleNet_SE.
[0053] Step (2): Use the palm region of interest extracted from the specified palmprint recognition dataset as the training dataset, train the palmprint recognition network ShuffleNet_SE, and use the Adam optimizer to execute the gradient descent strategy.
[0054] Step (3): Encrypt the palmprint image and recognition network. The encryption operation uses a generalized double random matrix as the basis of the encryption function. For the palmprint recognition network N(x), there are multiple linear layers, and each layer can be simplified to the product form with the weight matrix, as follows:
[0055] N(x;W)=∏ k W k x (1)
[0056] Among them, k represents the number of network layers, W k represents the weight matrix of the k-th layer network, and x represents the input value.
[0057] Given a generalized doubly random matrix A, for each linear layer, the weight matrix W k Can be Therefore, formula (1) can be converted to:
[0058]
[0059] Among them, A0 is the image key, A i is the network key of layer i.
[0060] There is a generalized sparse matrix A in the nonlinear activation function g in the network. The final result of exchanging the calculation order with function g is:
[0061] Ag((A -1x)) = g(AA -1 x)=g(x) (3)
[0062] The matrix set satisfying formula (3) is denoted as A * Then, the convolutional network composed of linear layers and nonlinear activation functions is encrypted to obtain the encrypted network, N i represents the i-th layer of the palmprint recognition network N(x), and the i-th layer network key A i ∈A * By from A * Randomly obtain, the encryption formula is:
[0063]
[0064] Among them, the network key A i Choose a sparse matrix and limit the density of the matrix through the privacy parameter α, that is, A i The number of non-zero elements in does not exceed the privacy parameter α. The value of the privacy parameter α can be specified as any integer. When the correct network key and image key are used, the size of the privacy parameter α has no effect on the network recognition accuracy. When the network key or image key used does not match, as α increases, the confidentiality and security of the system also increase.
[0065] Step (1) includes the following steps:
[0066] Step (1-1): Input the specified palmprint image, which goes through five stages, among which the second, third and fourth stages are composed of downsampling / upsampling modules and multiple ordinary modules.
[0067] Step (1-2): The channel of the feature matrix of the input palmprint image is split into two branches through multiple common modules, and each branch is processed separately. This process corresponds to the Channel Split operation.
[0068] This operation not only reduces the computational cost, but also enables the model to effectively process complex features through subsequent channel reorganization, thereby providing efficient and accurate image classification or recognition capabilities on resource-constrained devices.
[0069] In one of the branches, the features of the palmprint image are compressed and extracted through the bottleneck structure. After the extraction, the output of this branch is spliced with the feature matrix of another branch. The spliced feature matrix is remixed through the Channel Shuffle operation to ensure sufficient interaction of information between different channels.
[0070] The bottleneck structure includes: channel compression layer, deep convolution layer, channel recovery layer and a SE module.
[0071] Step (1-3): The downsampling / dimensionality upscaling module includes two branches: a downsampling module and a dimensionality upscaling module. The downsampling module branch and the dimensionality upscaling module branch respectively extract features from the input feature image, and then perform splicing and ChannelShuffle operations.
[0072] Step (1-4): Based on the original bottleneck structure, the second BN operation is replaced by a BFN operation, thereby replacing the BN layer in the network with a BFN layer, blocking the cumulative effect of the estimation offset and alleviating the degradation of network performance.
[0073] The channel attention mechanism structure in the network includes Squeeze and Excitation. The Squeeze operation is used to compress spatial information and compress the original C*H*W dimensional information into C*1*1.
[0074] The Excitation operation uses two fully connected layers. One of the fully connected layers reduces the dimension of the C*1*1 feature map to C / R*1*1 by using the ReLU activation function. The other fully connected layer remaps the feature back to C*1*1 without using the ReLU activation function. Then, the weight coefficient of each channel is obtained through sigmoid, and finally the weight coefficient is multiplied by the original feature to complete the calculation.
[0075] Through this calculation method, the model can pay more attention to important channel features and suppress minor channel features, thereby generating an optimized new feature map, which is more selective and discriminative in information expression, improving the network's representation ability and the final recognition effect.
[0076] In step (1-2), the Channel Shuffle operation includes the following steps:
[0077] Step (1-2-1): Divide the channels of the feature image into several groups according to certain rules.
[0078] Step (1-2-2): Rearrange or shuffle the channels in each group so that the originally adjacent channels are farther apart, thereby achieving information exchange and integration between channels.
[0079] Step (1-2-3): The rearranged or shuffled channels are recombined into a new feature map, wherein the new feature map contains information from different channels, and the rearranged or shuffled feature image has better representation ability.
[0080] The channel compression layer and channel restoration layer are both 1x1 group convolutions, and the depth convolution layer is a 3x3 group depthwise separable convolution.
[0081] Step (1-3) includes the following steps:
[0082] Step (1-3-1): In the downsampling module branch, the feature map is first processed through the deep convolution layer to effectively reduce the spatial dimension and retain the channel dimension, and then the number of channels is adjusted through the channel compression layer to control the amount of calculation.
[0083] Step (1-3-2): In the dimension-increasing module branch, the number of channels is first adjusted through the channel compression layer, and the nonlinear expression ability is improved through the ReLU activation function. Then, the feature expression is enhanced while keeping the dimension of the input feature map unchanged through the deep convolution layer, and then the dimension is adjusted through a channel recovery layer. Finally, the ReLU activation function and SE module are used to further improve the expression ability of the model.
[0084] Step (1-3-3): At the end of the module, the extraction results of the two branches are spliced, and the channels are rearranged through the ChannelShuffle operation to ensure that the features between the channels are fully mixed.
[0085] Step (3) includes the following steps:
[0086] Step (3-1): Determine the privacy parameter α according to the specific needs of the system. When the value of α is set to 8, a relatively ideal encryption effect can be achieved. When the encryption level is higher, the value of α can be appropriately increased on this basis.
[0087] Step (3-2): Encrypt the linear layer of the attention mechanism of the palmprint recognition network ShuffleNet_SE. According to the privacy parameter α and the parameter scale of the first linear layer, calculate all generalized sparse matrices A^* that satisfy formula (3), where the function g is the ReLU activation function. Randomly select A_0 from A^* and multiply it with the original image as the image key to complete the encryption operation of the palmprint image.
[0088] Step (3-3): For each linear layer in the network, find its corresponding A^* according to the steps in step (3-2) and encrypt it according to the algorithm shown in formula (4).
[0089] The homomorphic encryption method of palmprint data and its recognition network integrates the SE attention mechanism on the basis of the original ShuffleNet v2 architecture and replaces the BN layer with the BFN layer to design the ShuffleNet_SE network. It can effectively reduce the consumption of computing resources while maintaining recognition accuracy and adapt to resource-constrained device environments.
[0090] At the same time, this method proposes an encryption method for palmprint images and recognition networks based on generalized double random matrices, which can greatly reduce the risk of palmprint data and model parameters being exposed. In this process, the flexible adjustment of the privacy parameter α further enhances the confidentiality and security of the system.
[0091] In short, through the system design of this encryption method, the privacy and security of palmprint data can be effectively protected without affecting the recognition accuracy, and the network's anti-attack capability is improved, ensuring the reliability and security of the palmprint recognition system in practical applications.
[0092] However, what is described above is only a specific embodiment of the present invention and should not be used to limit the scope of implementation of the present invention. Therefore, the replacement of equivalent components, or equivalent changes and modifications made according to the scope of protection of the present invention should still fall within the scope covered by the claims of the present invention.
Claims
1. A homomorphic encryption method for palmprint data and its recognition network, characterized by: The steps include: Step (1): Taking ShuffleNet v2 as the basic network, adding SE attention mechanism and replacing the BN layer in the network with the BFN layer, thus designing the palmprint recognition network ShuffleNet_SE; Step (2): Use the palm region of interest extracted from the specified palmprint recognition dataset as the training dataset, train the palmprint recognition network ShuffleNet_SE, and use the Adam optimizer to execute the gradient descent strategy; Step (3): Encrypt the palmprint image and recognition network. The encryption operation uses a generalized double random matrix as the basis of the encryption function. For the palmprint recognition network N(x), there are multiple linear layers, and each layer can be simplified to the product form with the weight matrix, as follows: N(x;W)=∏ k W k x (1) Among them, k represents the number of network layers, W k represents the weight matrix of the k-th layer network, and x represents the input value; Given a generalized doubly random matrix A, for each linear layer, the weight matrix W k Can be Therefore, formula (1) can be converted to: Among them, A0 is the image key, A i is the network key of layer i; There is a generalized sparse matrix A in the nonlinear activation function g in the network. The final result of exchanging the calculation order with function g is: Ag((A -1 x))=g(AA -1 x)=g(x) (3) The matrix set satisfying formula (3) is denoted as A * Then, the convolutional network composed of linear layers and nonlinear activation functions is encrypted to obtain the encrypted network, N i represents the i-th layer of the palmprint recognition network N(x), and the i-th layer network key A i ∈A * By from A * Randomly obtain, the encryption formula is: Among them, the network key A i Choose a sparse matrix and limit the density of the matrix through the privacy parameter α, that is, A i The number of non-zero elements in does not exceed the privacy parameter α. The value of the privacy parameter α can be specified as any integer. When the correct network key and image key are used, the size of the privacy parameter α has no effect on the network recognition accuracy. When the network key or image key used does not match, as α increases, the confidentiality and security of the system also increase.
2. The homomorphic encryption method for palmprint data and its recognition network according to claim 1 is characterized in that: The step (1) comprises the following steps: Step (1-1): Input the specified palmprint image, which goes through five stages, among which the second, third and fourth stages are composed of downsampling / upsampling modules and multiple ordinary modules; Step (1-2): split the channel of the feature matrix of the input palmprint image into two branches through multiple common modules, and process each branch separately. This process corresponds to the Channel Split operation; In one of the branches, the features of the palmprint image are compressed and extracted through the bottleneck structure. After the extraction is completed, the output of this branch is spliced with the feature matrix of another branch. The spliced feature matrix is remixed through the ChannelShuffle operation to ensure sufficient interaction of information between different channels. The bottleneck structure includes: channel compression layer, deep convolution layer, channel recovery layer and a SE module; Step (1-3): The downsampling / upgrading module includes two branches, namely, a downsampling module and an upgrading module. The downsampling module branch and the upgrading module branch respectively extract features from the input feature image, and then perform splicing and Channel Shuffle operations; Step (1-4): Based on the original bottleneck structure, the second BN operation is replaced by the BFN operation, thereby replacing the BN layer in the network with the BFN layer, blocking the cumulative effect of the estimation offset and alleviating the degradation of network performance; The channel attention mechanism structure in the network includes Squeeze and Excitation. The Squeeze operation is used to compress spatial information, compressing the original C*H*W dimensional information into C*1*1. The Excitation operation uses two fully connected layers. One of the fully connected layers reduces the dimension of the C*1*1 feature map to C / R*1*1 by using the ReLU activation function. The other fully connected layer remaps the feature back to C*1*1 without using the ReLU activation function. Then, the weight coefficient of each channel is obtained through sigmoid, and finally the weight coefficient is multiplied by the original feature to complete the calculation.
3. The homomorphic encryption method for palmprint data and its recognition network according to claim 2 is characterized in that: In the step (1-2), the Channel Shuffle operation includes the following steps: Step (1-2-1): Divide the channels of the feature image into several groups according to certain rules; Step (1-2-2): rearrange or shuffle the channels in each group so that the originally adjacent channels are far apart, thereby achieving information exchange and integration between channels; Step (1-2-3): The rearranged or shuffled channels are recombined into a new feature map, wherein the new feature map contains information from different channels, and the rearranged or shuffled feature image has better representation ability.
4. The homomorphic encryption method for palmprint data and its recognition network according to claim 2 is characterized in that: The channel compression layer and the channel recovery layer are both 1x1 group convolutions, and the depth convolution layer is a 3x3 group depthwise separable convolution.
5. The homomorphic encryption method for palmprint data and its recognition network according to claim 4 is characterized in that: The steps (1-3) include the following steps: Step (1-3-1): In the downsampling module branch, the feature map is first processed through the deep convolution layer to effectively reduce the spatial dimension and retain the channel dimension, and then the number of channels is adjusted through the channel compression layer to control the amount of calculation; Step (1-3-2): In the dimension-increasing module branch, the number of channels is first adjusted through the channel compression layer, and the nonlinear expression ability is improved through the ReLU activation function. Then, the feature expression is enhanced while keeping the dimension of the input feature map unchanged through the deep convolution layer, and then the dimension is adjusted through a channel recovery layer. Finally, the ReLU activation function and SE module are used to further improve the expression ability of the model. Step (1-3-3): At the end of the module, the extraction results of the two branches are spliced, and the channels are rearranged through the ChannelShuffle operation to ensure that the features between the channels are fully mixed.
6. The homomorphic encryption method for palmprint data and its recognition network according to claim 1 is characterized in that: The step (3) comprises the following steps: Step (3-1): Determine the privacy parameter α according to the specific requirements of the system; Step (3-2): Encrypt the linear layer of the attention mechanism of the palmprint recognition network ShuffleNet_SE. According to the privacy parameter α and the parameter scale of the first linear layer, calculate all generalized sparse matrices A^* that satisfy formula (3), where the function g is the ReLU activation function. Randomly select A_0 from A^* and multiply it with the original image as the image key to complete the encryption operation of the palmprint image. Step (3-3): For each linear layer in the network, find its corresponding A^* according to the steps in step (3-2) and encrypt it according to the algorithm shown in formula (4).