An Information Hiding Method for Deep Neural Networks

Abstract

The present invention relates to an information hiding method for deep neural networks, belonging to the field of information security. The method includes: an information hiding stage, a triggering stage, and a hidden information extraction stage. The information hiding stage includes: preprocessing hidden parameter bits, pseudo-randomly selecting embedding layers, encrypting information, and embedding ciphertext; the triggering stage includes: the communication parties sharing relevant parameters, obtaining valid inputs, obtaining target categories, and verifying triggering targets; the hidden information extraction stage includes: determining embedding positions according to triggering targets, obtaining valid parameter bits, extracting information, and decrypting information. The present invention solves the problem that the deep neural network information hiding method that directly embeds neural network parameters cannot resist pruning attacks; by pseudo-randomly selecting to determine the embedding position of hidden information and using the triggering target as a key to encrypt the hidden information, it can, to a certain extent, resist white-box attacks after the attacker knows the hidden information method, improving the security of the method.

Description

Technical Field

[0001] The present invention belongs to the field of information security and relates to an information hiding method for deep neural networks. Background Art

[0002] As a main method for protecting information security, information hiding uses redundant information imperceptible to humans to hide secret information in a cover carrier to achieve secure communication. This can not only ensure the security of the secret information itself, but also ensure the security of the transmission of the secret information. However, existing information hiding methods have problems of distortion in both the obtained stego carrier and the quality of the extracted secret information. Moreover, the change in statistical characteristics after the hidden information is embedded in the carrier also affects the security. To solve the above problems, researchers have combined deep neural networks, which have achieved breakthrough results in various fields, with steganography, providing new ideas for information hiding. Deep neural network information hiding algorithms are generally divided into two categories: deep neural network information hiding based on output and based on internal mechanisms. The first type of method usually modulates the output of the neural network through an embedding network to embed the hidden information, and then recovers the secret information from the output result through an extraction network. Although the deep neural network information hiding method based on output improves the steganography quality and certain security, the hiding capacity of this type of method is still limited by the size of the output carrier, and jointly training the two networks requires a large amount of computing resources. The second type of method usually encodes the hidden information and directly embeds it into the neural network parameters or structure without the need to additionally train an extraction network, greatly reducing resource consumption. However, after embedding the hidden information into the neural network model, the parameters involving the secret information cannot be changed. Therefore, once an attacker performs a pruning attack through the model parameters, the secret information hidden in the deep neural network model will be directly damaged. Generally, existing deep neural network information hiding methods cannot achieve a good balance in terms of embedding capacity, robustness, and security. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide an information hiding method for deep neural networks, solve the problem that the deep neural network information hiding method of directly embedding neural network parameters cannot resist pruning attacks; propose a scheme for secondary embedding and hiding of information corresponding to unstable neurons, solve the problem that hidden information is not error-tolerant, and ensure the extraction integrity rate of hidden information; use pseudo-random selection to determine the embedding position of hidden information and use the trigger target as a key to encrypt the hidden information, which can, to a certain extent, resist white-box attacks after an attacker knows the hidden information method, and improve the security of the method.

[0004] To achieve the above purpose, the present invention provides the following technical solutions:

[0005] An information hiding method for deep neural networks, the method comprising the following steps:

[0006] Step a, pseudo-randomly select the embedding layer of the deep neural network model, preprocess the model parameters, and design a hidden information key;

[0007] Step b, encrypt the hidden information according to the hidden information key obtained in step a, store the hidden information in the parameters of the embedding layer of the deep neural network model obtained in step a, and at the same time store the secondary embedding length of the hidden information in the bias of the model embedding layer;

[0008] Step c, the two parties of the hidden information communication share relevant parameters, obtain valid inputs, target categories, and verify trigger targets;

[0009] Step d, determine the digit positions of the effective parameters obtained from the layer of the deep neural network model where the hidden information is located according to the trigger target obtained in step c;

[0010] Step e, determine the unselected embedding layer according to the pseudo-random sequence, and determine the secondary embedding length of the hidden information according to the value in the current layer bias; then, jointly calculate through the pseudo-random sequence and the secondary embedding length to obtain the layer order when extracting the hidden information and the starting position when extracting the hidden information from the corresponding layer; finally, decrypt the extracted secret information with the key. Optionally, in step a, a pseudo-random sequence R is generated according to a pseudo-random number generator, and its expression is as follows:

[0011] R = {R1, R2,..., R n} (1)

[0012] After data conversion according to R, a key K is generated, and its expression is as follows:

[0013] K = {K1, K2,... K n} (2)

[0014] In the formula, n is the number of layers of the deep neural network.

[0015] Optionally, in step b, the hidden information is divided into multiple bit segments M of 16 bits each after data conversion, and its expression is as follows:

[0016] M = {M1, M2,..., M m} (3)

[0017] According to the pseudo-random sequence R, select and determine the hidden information embedding layer; after determining the embedding layer, obtain the neurons therein and determine whether they are important neurons. If they are important neurons, encrypt the hidden information bit segments with the key according to formula (1) to obtain the encrypted text segments, and its mathematical representation is as follows:

[0018]

[0019] where ⊕ is bitwise exclusive OR, 1 ≤ i ≤ m, represents a 16-bit binary encryption key segment, represents the encrypted ciphertext segment;

[0020] Modify the parameters in the neuron according to the substitution function; calculate the new parameter values and determine whether it is an unstable neuron. If it is an unstable neuron, embed the hidden information into the position of the layer to be embedded in the next layer for the second time to ensure correct recovery in case of extraction error. The mathematical expression is as follows:

[0021] G(·):(W, S) → W new (5)

[0022] In the formula, W is the neuron, S is the encrypted information, and W new is the new parameter value after embedding the information; store the second embedding length of the hidden information in this layer into the bias, which is used to determine the starting position of each layer when extracting the hidden information.

[0023] Optionally, in step c, during the running of the model, if the model output gets a specific target category three times in a row, trigger the extraction process to extract the embedded hidden information; otherwise, the neural network model will only perform its normal function.

[0024] Optionally, in step d, generate a pseudo-random sequence R′ with the triggered target category as the initial seed, where R = R′, determine the unselected embedding layer according to the pseudo-random sequence R′, and obtain the second embedding length in the bias.

[0025] Optionally, in step e, obtain the important neurons in the embedding layer and determine whether they are unstable neurons. If they are unstable neurons, discard the current hidden information and extract the secret information in the effective embedding layer of the next layer according to the second embedding length; otherwise, directly extract the hidden information in the current neuron parameters; finally, decrypt the secret information S with the key K to obtain the hidden information M. The mathematical expression of the extraction function is as follows:

[0026] f(·):W → S (6)

[0027] In the formula, W is the neuron parameter, convert the parameter into 32-bit binary, and extract the last 16 bits W2[16:] to obtain the secret information S.

[0028] The beneficial effects of the present invention are as follows:

[0029] Considering that unstable neurons can cause errors in extracting hidden information, a method of secondary embedding is proposed, which solves the problem of the intolerance of hidden information to errors and ensures the integrity rate of hidden information extraction. The position of hidden information embedding is determined by pseudo-random selection, and the trigger target is used as the key to encrypt the hidden information, which can resist the white-box attack by the attacker after knowing the method of hidden information to a certain extent and improve the security of the method. Then, according to the key designed in the initial stage, the extraction of hidden information is realized.

[0030] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. Brief Description of the Drawings

[0031] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0032] Figure 1 It is a flowchart of the hidden information embedding process.

[0033] Figure 2 It is a flowchart of the hidden information triggering process.

[0034] Figure 3 It is a flowchart of the hidden information extraction process. Detailed Embodiments

[0035] The following illustrates the embodiments of the present invention through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0036] Among them, the drawings are only used for exemplary illustration, showing only schematic diagrams, not physical diagrams, and cannot be understood as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0037] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the accompanying drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0038] An information hiding method for the robustness, fault tolerance and security of a deep neural network, which uses pre-trained models of different sizes on a public repository for experiments. The goal of this experiment is to demonstrate the generality of the method. Figure 1 The experimental flowchart of information hiding in a neural network is given. It includes the following steps:

[0039] Step a, pseudo-randomly select the embedding layer of the deep neural network model, preprocess the model parameters, and design the key for hiding information.

[0040] Step b, encrypt the hidden information according to the key for hiding information obtained in step a, store the hidden information into the parameters of the embedding layer of the deep neural network model obtained in step a, and at the same time store the secondary embedding length of the hidden information into the bias of the model embedding layer.

[0041] Step c, the two parties communicating the hidden information share relevant parameters, obtain the valid input, the target category, and verify the trigger target.

[0042] Step d, determine the digit positions of the valid parameters of the layer where the hidden information is located in the deep neural network model according to the trigger target obtained in step c.

[0043] Step e, determine the unselected embedding layer according to the pseudo-random sequence, and determine the secondary embedding length of the hidden information according to the value in the current layer bias; then, jointly calculate through the pseudo-random sequence and the secondary embedding length to obtain the layer order when extracting the hidden information and the starting position when extracting the hidden information from the corresponding layer; finally, decrypt the extracted secret information with the key. In step a, according to the pseudo-random number generator, a pseudo-random sequence R is generated, and its expression is as follows:

[0044] R = {R1, R2,..., R n} (1)

[0045] After data conversion according to R, a key K is generated, and its expression is as follows:

[0046] K = {K1, K2,... Kn} (2)

[0047] Where n is the number of layers of the deep neural network.

[0048] In the step b, after data conversion of the hidden information, it is divided into multiple bit segments M of 16 bits each group, and its expression is as follows:

[0049] M = {M1, M2,..., M m} (3)

[0050] Determine the hidden information embedding layer according to the pseudo-random sequence R. After determining the embedding layer, obtain the neurons therein and judge whether they are important neurons. If they are important neurons, encrypt the hidden information bit segments with the key according to formula (1) to obtain the encrypted text segment, and its mathematical representation is as follows:

[0051]

[0052] Where ⊕ is the bitwise exclusive OR, 1 ≤ i ≤ m, represents a 16-bit binary encryption key segment, represents the encrypted text segment after encryption.

[0053] Modify the parameters in the neurons according to the substitution function. Calculate the new parameter values and judge whether they are unstable neurons. If they are unstable neurons, embed the hidden information into the position of the next layer to be embedded layer twice to ensure correct recovery when extraction fails. Its mathematical expression is as follows:

[0054] G(·): (W, S) → W new (5)

[0055] Where W is the neuron, S is the encrypted information, and W new is the new parameter value after embedding the information. Store the twice embedding length of the hidden information in this layer into the bias, which is used to determine the starting position of each layer when extracting the hidden information.

[0056] In the step c, during the running process of the model, if the model outputs the specific target category three times continuously, trigger the extraction process to extract the embedded hidden information; otherwise, the neural network model will only perform its normal function.

[0057] In the step d, generate a pseudo-random sequence R′ with the triggered target category as the initial seed, where R = R′, determine the unselected embedding layer according to the pseudo-random sequence R′, and obtain the twice embedding length in the bias.

[0058] In step e, important neurons in the embedding layer are obtained and it is determined whether they are unstable neurons. If they are unstable neurons, the current hidden information is discarded and the secret information in the next-layer effective embedding layer is extracted according to the secondary embedding length; otherwise, the hidden information in the current neuron parameters is directly extracted. Finally, the secret information S is decrypted with the key K to obtain the hidden information M. The mathematical expression of the extraction function is as follows:

[0059] f(·):W→S (6)

[0060] In the formula, W is the neuron parameter. The parameter is converted into 32-bit binary, and the last 16 bits W2[16:] are extracted to obtain the secret information S.

[0061] Figure 2 is the process flow chart of the hidden information triggering process. Figure 3 is the process flow chart of the hidden information extraction process.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An information hiding method for deep neural networks, characterized in that: The method includes the following steps: Step a: Pseudo-randomly select the embedding layer of the deep neural network model, preprocess the model parameters, and design a hidden information key; Step b: Encrypt the hidden information according to the hidden information key obtained in step a, store the hidden information in the parameters of the embedding layer of the deep neural network model obtained in step a, and at the same time store the hidden information secondary embedding length in the model embedding layer bias; Step c: The two parties of the hidden information communication share relevant parameters, obtain valid inputs, target categories, and verify the trigger target; Step d: Determine the digit positions of the valid parameters in the deep neural network model layer where the hidden information is located according to the trigger target obtained in step c; Step e: Determine the unselected embedding layer according to the pseudo-random sequence, and determine the hidden information secondary embedding length according to the value in the current layer bias; then, jointly calculate through the pseudo-random sequence and the secondary embedding length to obtain the layer order when extracting the hidden information and the starting position of extracting the hidden information in the corresponding layer; finally, decrypt the extracted secret information with the key; In step e, obtain the important neurons in the embedding layer and determine whether they are unstable neurons. If they are unstable neurons, discard the current hidden information and extract the secret information in the next valid embedding layer according to the secondary embedding length; otherwise, directly extract the hidden information in the current neuron parameters; finally, decrypt the secret information S with the key K to obtain the hidden information M; the mathematical expression of the extraction function is as follows: f(·):W→S (6) In the formula, W is the neuron parameter, convert the parameter into 32-bit binary, and extract the last 16 bits W2[16:] to obtain the secret information S.

2. The information hiding method for a deep neural network according to claim 1, wherein: In step a, generate a pseudo-random sequence R according to the pseudo-random number generator, and its expression is as follows: R = {R1, R2,..., R n} (1) Generate a key K after data conversion according to R, and its expression is as follows: K = {K1, K2,..., K n} (2) In the formula, n is the number of layers of the deep neural network.

3. The information hiding method for a deep neural network according to claim 2, characterized in that: In step b, after data conversion of the hidden information, it is divided into multiple bit segments M of 16 bits each group, and its expression is as follows: M = {M1, M2,..., M m} (3) Select and determine the hidden information embedding layer according to the pseudo-random sequence R; after determining the embedding layer, obtain the neurons in it and determine whether they are important neurons. If they are important neurons, encrypt the hidden information bit segment with the key according to formula (1), and its mathematical representation is as follows: where ⊕ is bitwise exclusive OR, 1 ≤ i ≤ m, K i j represents a 16-bit binary encryption key segment, S i j represents the encrypted ciphertext segment; Modify the parameters in the neurons according to the encryption information by the replacement function; calculate the new parameter value, and determine whether it is an unstable neuron. If it is an unstable neuron, embed the hidden information secondary into the position of the next layer to be embedded layer to ensure correct recovery when extraction fails; its mathematical expression is as follows: G(·): (W, S) → W new (5) Wherein, W is a neuron, S is encrypted information, and W new is the new parameter value after embedding information; the secondary embedding length of the hidden information in this layer is stored in the bias, which is used to determine the starting position of each layer when extracting the hidden information.

4. The information hiding method for a deep neural network according to claim 3, characterized in that: In step c, during the operation of the model, if the model output gets a specific target category three times in a row, trigger the extraction process to extract the embedded hidden information; Otherwise, the neural network model will only perform the normal function of the model.

5. The information hiding method for a deep neural network according to claim 4, wherein: In step d, generate a pseudo-random sequence R′ with the trigger target category as the initial seed, where R = R′, determine the unselected embedding layer according to the pseudo-random sequence R′, and obtain the secondary embedding length in the bias.

Images