A Data Adaptive Hiding Method and System Based on Large Language Model

Abstract

This invention relates to the field of information hiding technology, and particularly to a data adaptive hiding method and system based on large language models. For LLM parameter files, it utilizes key parameters for covert embedding and evaluates parameter importance based on ablation and quantization scaling factors to identify parameters with minimal impact on LLM performance as key embedding locations. Then, based on parameter importance, a steganography strategy is formulated, implementing half-precision LSB steganography and quantized LSB steganography for both non-quantized and quantized scenarios. This ensures that regardless of whether the user quantizes and compresses the model, large-capacity data information can be covertly embedded in the LLM while maintaining data integrity. This invention solves the problems of low data transmission efficiency and unsatisfactory data transmission covertness in existing open-source large language models, improving the effectiveness of data covert transmission.

Description

Technical Field

[0001] This invention relates to the field of information hiding technology, and in particular to a data adaptive hiding method and system based on a large language model. Background Technology

[0002] Information hiding technology ensures the confidentiality of information transmission, focusing not only on the content protection of secret information but also on enhancing the concealment of its transmission. It is a powerful means of protecting data assets in complex network environments. Large Language Models (LLMs) have rapidly become one of the most transformative technologies in the field of Artificial Intelligence (AI). Supported by deep learning and massive datasets, these foundational models can generate text, translate languages, summarize content, write code, and automatically perform numerous language tasks, leading to their widespread application across various fields. Compared to proprietary LLMs like OpenAI's ChatGPT, open-source LLMs offer the following advantages: First, they have transparent code and weights, allowing researchers to study how the model works without trade secrets hindering progress; second, users do not need to spend significant resources on pre-training, but can customize the model by fine-tuning it using proprietary data according to task requirements; third, they are less costly in the long run, avoiding expensive recurring licensing fees from vendors like OpenAI, only requiring payment for cloud or on-premises infrastructure; and finally, they offer a degree of privacy protection by preventing data uploads to vendors. Based on these considerations, more and more companies are deploying open-source LLMs to provide intelligent services for their work. Compared with traditional neural networks, LLMs have complex network structures and internal operating mechanisms with a large number of parameters. If the model steganography and triggering methods used for traditional neural network models are directly applied to LLMs, not only will they fail to effectively steganize and trigger, but they will also face the problem of low embedding efficiency. Summary of the Invention

[0003] To address this, the present invention provides a data adaptive hiding method and system based on a large language model, which solves the problems of low data hiding efficiency and unsatisfactory data hiding effect of existing open-source large language models.

[0004] According to the design scheme provided by this invention, on the one hand, a data adaptive hiding method based on a large language model is provided, comprising:

[0005] Based on the degree of influence of each parameter in the large language model on the model performance, the parameters of the large language model are grouped, and the target parameter positions for data cryptic embedding are selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter types, ordinary parameter types, and important parameter types.

[0006] Generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter location.

[0007] As a data adaptive hiding method based on a large language model in this invention, the parameters of the large language model are further grouped according to the degree of influence of each parameter on the model performance, including:

[0008] Obtain the optimal parameters for the large language model, and record the performance of the large language model under the optimal parameters as the maximum value;

[0009] Based on the number of parameters in each layer of the large language model structure, neuron layers with a number of parameters greater than a specified value are selected as parameter replacement regions.

[0010] We replace the neuron parameters in the optimal parameters of a large language model with random binary bits, and then obtain the performance of the large language model after replacing the neuron parameters.

[0011] The importance of the corresponding neuron parameters is measured by the absolute value of the performance loss of the large language model before and after the replacement, and the neuron parameters in the parameter replacement region are grouped according to the importance of the neuron parameters.

[0012] As a data adaptive hiding method based on a large language model in this invention, further, selecting the target parameter position for data cryptic embedding from the grouped model parameters includes:

[0013] Initialize the embedding bit of non-critical parameters, ordinary parameters and important parameters of the large language model, and concatenate the weight matrices of the non-critical parameters, ordinary parameters and important parameters to obtain the parameter weight matrix of the large language model. The parameter weight matrix is ​​used to store the connection weights between neurons of the large language model for transmitting information.

[0014] Based on the number of embedding bits for non-critical parameters, ordinary parameters, and important parameters of the large language model, corresponding mask matrices are created for minimum effective bit embedding.

[0015] As a data adaptive hiding method based on a large language model in this invention, further, the binary data is covertly embedded into the target parameter position, including:

[0016] Determine the grouping type of model parameters corresponding to each element in the parameter weight matrix of a large language model;

[0017] If the matrix element is a non-critical parameter, then extract the binary data element according to the number of bits embedded in the non-critical parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the non-critical parameter to update the current matrix element.

[0018] If the matrix elements are ordinary parameters, then extract the binary data elements according to the number of bits embedded in the ordinary parameters and assign them to the first variable. Assign the remaining binary data elements that have not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the ordinary parameters to update the current matrix elements.

[0019] If the matrix element is an important parameter, then determine whether the length of the binary data is less than the number of bits embedded in the important parameter. If it is less, then convert the binary data to the specified data type. If it is not less, then extract the binary data element according to the number of bits embedded in the important parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the important parameter to update the current matrix element.

[0020] Furthermore, this invention also provides a data adaptive hiding system based on a large language model, comprising: an embedding position acquisition module and a steganographic data embedding module, wherein,

[0021] The embedding location acquisition module is used to group the parameters of the large language model according to the degree of influence of each parameter on the model performance, and select the target parameter position for data cryptic embedding from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type and important parameter type.

[0022] The steganography embedding module is used to generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0023] Furthermore, this invention also provides a data adaptive hiding method based on a large language model, comprising:

[0024] The initial large language model parameters are uniformly linearly quantized using a quantization scaling factor to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters.

[0025] Based on the degree of influence of each parameter in the quantization model on the model performance, the parameters of the quantization model are grouped, and the target parameter positions for data hidden embedding are selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter types, ordinary parameter types, and important parameter types.

[0026] Generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position;

[0027] The model embedding binary data is dequantized, and the embedded target parameter location region is replaced with the parameter region corresponding to the initial large language model.

[0028] As a data adaptive hiding method based on a large language model in this invention, the parameters of the quantization model are further grouped according to the degree of influence of each parameter on the model performance, including:

[0029] Obtain the optimal parameters of the quantization model and record the model performance under the optimal parameters as the maximum value;

[0030] Based on the quantization scaling factor of neurons in the quantization model, neurons with a quantization scaling factor greater than a specified value are selected as parameter replacement regions.

[0031] By using random binary bits to replace the corresponding neuron parameters in the optimal parameter replacement region of the quantized model parameters, the model performance after the neuron parameter replacement is obtained.

[0032] The importance of the corresponding neuron parameters is measured by the absolute value of the model performance loss before and after the replacement, and the neuron parameters in the parameter replacement region are grouped according to the importance of the neuron parameters.

[0033] Furthermore, this invention provides a data adaptive hiding system based on a large language model, comprising: a model parameter quantization module, an embedding position acquisition module, a steganographic data embedding module, and a model parameter dequantization module, wherein...

[0034] The model parameter quantization module is used to uniformly and linearly quantize the initial large language model parameters using a quantization scaling factor to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters.

[0035] The embedding location acquisition module is used to group the parameters of the quantization model according to the degree of influence of each parameter on the model performance, and select the target parameter location for data cryptic embedding from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type and important parameter type.

[0036] The steganography data embedding module is used to generate the integrity check code of the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0037] The model parameter dequantization module is used to dequantize the model with embedded binary data and replace the embedded target parameter location region with the parameter region corresponding to the initial large language model.

[0038] The beneficial effects of this invention are:

[0039] This invention uses key parameter localization in hidden embedding to reveal the importance of different parameters to LLM performance. Based on an adaptive steganography strategy, it ensures the integrity of embedded data in both non-quantized and quantized scenarios, while also concealing large amounts of data. Triggers hidden in the open-source LLM supply chain can be used to parse and retrieve the hidden data. Furthermore, in experimental data, using chatglm3-6b with 11.63GB of parameters as the embedding object, the data hiding process was implemented, and the effect of adaptive steganography was analyzed. Under non-quantized and int8 quantized model import conditions, it can embed more than 5.81GB and 2.18GB of data respectively with less than 1% performance sacrifice. The experimental data demonstrates that this solution can conceal large amounts of data within LLMs while ensuring data integrity. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the Large Language Model (LLM) inference process and parameter structure in the embodiment.

[0041] Figure 2 This is a schematic diagram of the neuron steganography process in the embodiment;

[0042] Figure 3 This is a schematic diagram of the data hiding algorithm principle framework in the embodiment;

[0043] Figure 4 This is a schematic diagram of the adaptive data hiding process for non-quantitative scenarios in the embodiment;

[0044] Figure 5 This is a schematic diagram of the adaptive hiding process for quantized scene data in the embodiment;

[0045] Figure 6 This is a schematic diagram of the important neuron localization process in the embodiment;

[0046] Figure 7 This is a schematic diagram of the quantization steganography process in the embodiment;

[0047] Figure 8 This is a schematic diagram of an ablation experiment based on random bit substitution in the embodiment;

[0048] Figure 9 This is a schematic diagram illustrating the distribution of model parameter importance in the embodiment;

[0049] Figure 10 This is a schematic diagram illustrating the neuron importance analysis in the example;

[0050] Figure 11 This is a schematic diagram of the half-precision LSB steganalysis results in the embodiment;

[0051] Figure 12 This is a schematic diagram of the quantized LSB steganalysis results in the example. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0053] Currently, open-source Large Language Models (LLMs) have achieved great success in various fields. The complex network structure and billions of parameters of large language models make open-source LLM files convenient for the propagation of covert data, which is not easily detected by existing detection techniques.

[0054] There are many types of open-source LLMs, and while their structures differ, the reasoning processes are generally the same. This analysis primarily focuses on chatglm3-6b. Figure 1 As shown, the inference process of chatglm3-6b is analyzed from the perspective of input flow. The specific process includes: (1) After preprocessing such as word segmentation and encoding, the input is mapped to feature vectors by word embedding. (2) Entering GLMBlock, it first goes through the input normalization layer. Entering the attention module, the input data is QKV mapped, RotaryPositional Embedding, scaled dot product attention, and linear layer. After leaving the attention module, Dropout, residual connection, and post-normalization are performed in sequence. Entering the MLP module, it goes through two linear layers, and needs to go through the SwiGLU activation function in between. After leaving the MLP, it goes through Dropout and residual connection. Under the control of the for loop, this output is used as the input of the next round. Under the control of the loop controller, it loops 28 times in total, and the parameters of each block are different. (3) After 28 rounds of GLMBlock loop, the attention scores of the last block output will be used for output processing. The function of output processing is to convert attention into output ID. (4) Under the control of the outermost loop controller, as long as the meaning represented by this ID is not... <eos>Then it will start from WordEmbedding again, and generate the next token output through all processes. (5) Finally, convert all output IDs into text output.

[0055] A neural network is a computational model that mimics the working principle of the human brain. It consists of a large number of interconnected nodes (or "neurons"). Information is transmitted between neurons through connections, and each connection has corresponding weights. These weights are stored in a matrix called the weight matrix.

[0056] Specifically, suppose we have a neural network with an input layer of n neurons and a hidden layer of m neurons. The weight matrix from the input layer to the hidden layer will be an m×n matrix, where each element wij represents the connection weight from the j-th neuron in the input layer to the i-th neuron in the hidden layer. wj represents the connection between the j-th neuron and subsequent neurons; therefore, wj can be represented as the weight parameter of the j-th neuron. A neuron is essentially represented as a one-dimensional weight parameter.

[0057] The data types of weight parameters are divided into floating-point and integer types. Different floating-point data types include single-precision (float32), half-precision (float16, bfloat16), etc. Floating-point numbers are divided into three parts in a computer: sign, exponent, and mantissa. For example... Figure 2 As shown, taking float16 as an example, it occupies 16 bits in memory. These 16 bits are divided into three parts: 1 / 5 / 10 represent the sign / exponent / mantissa. The formula for calculating float16 data is as follows:

[0058]

[0059] Integer weight parameters are often the quantized result of floating-point data type weight parameters. Taking `int8` as an example, the length of an `int8` integer is 8 bits, including one sign bit and 7 value bits, representing a range of -128 to 127. The calculation method for `int8` data is as follows:

[0060] (-1) sign *2 n

[0061] Whether using integer or floating-point neurons, to hide data, modifying the bits within the least significant bit (LSB) is a prudent approach. Changing the higher bits can significantly alter the numerical value, potentially impacting model performance. The goal is to embed information without the user noticing. Figure 2 As shown, neuron LSB steganography involves embedding a binary string within the least significant bits of the model weights.

[0062] like Figure 3 The adaptive data hiding framework based on a large language model (LLM) allows data embedders to secretly embed a large amount of data information into the parameters of open-source LLMs with almost no impact on model performance. This involves four steps: data preparation, model training, model release, and user deployment. Users create LLMs using steganography before releasing the model to the open-source community. During user deployment, after the user downloads and imports the model, triggers in the supply chain activate the hidden information embedded in the LLM. To achieve the goal, the key parameters for hidden embedding in the LLM must first be located. Then, adaptive steganography is used to formulate a steganography strategy based on the location of these key parameters, implementing hidden data embedding for both non-quantized and quantized scenarios. The data hiding framework consists of: key parameter location, adaptive steganography, and trigger execution.

[0063] The key parameter localization involves prioritizing parameters with minimal impact on LLM performance for embedding. This provides support for developing adaptive steganography embedding strategies, minimizing performance loss and preventing user detection. To locate these key parameters, ablation techniques based on random bit substitution are used to evaluate the location of critical layers in the LLM. Specifically, for quantized neurons, their criticality is related to the quantization factor, and the magnitude of the quantization factor can serve as a basis for judging critical neurons.

[0064] Adaptive steganography allows for the covert embedding of data information within an LLM (Limited Model Learning) when it's released in the open-source community. By implementing an embedding strategy, it prioritizes embedding less critical parameters, minimizing embedding in critical parameters. Compared to traditional methods without embedding strategies, this approach effectively reduces model performance degradation caused by embedding. For both non-quantized and quantized scenarios, half-precision LSB steganography and quantized LSB steganography are used to ensure the integrity of the embedded data in both commonly used model import scenarios.

[0065] Specifically, for non-quantitative scenarios, see the embodiments of the present invention. Figure 4 As shown, this paper presents a data adaptive hiding method based on a large language model, which can secretly embed a large amount of data information into the parameters of open-source LLMs with almost no impact on model performance, including:

[0066] S101. Based on the degree of influence of each parameter in the large language model on the model performance, the parameters of the large language model are grouped, and the target parameter positions for data cryptic embedding are selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter types, ordinary parameter types, and important parameter types.

[0067] Specifically, the optimal parameters of the large language model can be obtained first, and the performance of the large language model under the optimal parameters can be recorded as the maximum value. Then, based on the size of the number of parameters in each layer of the large language model structure, the neuron layer with a parameter size greater than a specified value is selected as the parameter replacement region. Random binary bits are used to replace the neuron parameters in the optimal parameters of the parameter replacement region of the large language model, and the performance of the large language model after the neuron parameter replacement is obtained. The absolute value of the performance loss of the large language model before and after replacement is used to measure the importance of the corresponding neuron parameter, so that the neuron parameters in the parameter replacement region can be grouped according to the importance of the neuron parameters.

[0068] The selection of target parameter locations for hidden data embedding from the grouped model parameters can be designed to include:

[0069] Initialize the embedding bit of non-critical parameters, ordinary parameters and important parameters of the large language model, and concatenate the weight matrices of the non-critical parameters, ordinary parameters and important parameters to obtain the parameter weight matrix of the large language model. The parameter weight matrix is ​​used to store the connection weights between neurons of the large language model for transmitting information.

[0070] Based on the number of embedding bits for non-critical parameters, ordinary parameters, and important parameters of the large language model, corresponding mask matrices are created for minimum effective bit embedding.

[0071] Specific parameter regions within an LLM (Limited Model Language) significantly impact the model's language capabilities. Removing these specific parameter regions (zero-out) drastically reduces the LLM's language capabilities, while removing other parameter regions has almost no effect. Similarly, in scenarios involving embedded malicious payloads, specific parameter regions within the LLM also significantly influence model performance.

[0072] Specifically, given As parameters of neurons in the model, each θ j Let ∈R represent the parameters of the j-th hidden neuron. Assume that the parameters θ are the optimal parameters for the LLM, meaning the LLM performs best at this point, denoted as P. max When a single neuron θ j When subjected to an embedding attack, its value will change to r. j For the n neuron parameters in θ that are subjected to embedding attacks, let I denote the importance of these parameters. n (θ)∈R d I n (θ) is measured by the absolute value of the model performance loss after bit substitution of this part of the neurons, as shown in the following formula:

[0073]

[0074] In real-world scenarios, users often encrypt incoming data to eliminate the characteristics of malicious content. The encrypted content appears to other users as a completely random stream of bits. Therefore, each bit substitution uses random binary bits. In this case, the random variables r1, r2, ..., r... n They satisfy independent and identically distributed (i.e., i.i.d.). According to the law of large numbers, as n increases, r1, r2, ..., r n The average value will approach a constant, which can be expressed by the formula:

[0075]

[0076] When the number of embedded neurons n is large enough, the importance I n The formula for calculating (θ) can be expressed as follows:

[0077]

[0078] At this point, each random replacement can be equivalent to a fixed-value replacement, i.e., r j =μ,I n The value of (θ) is independent of the content θ is replaced, but only depends on the position i of the replaced parameter and the number n. Therefore, by replacing the neurons at n different parameter positions j, the importance I of these neurons can be obtained by calculating the model performance loss Lost. n (θ). To make the random variables r1, r2, ..., r n To satisfy the law of large numbers, n should be large enough to ensure the stability and validity of the experimental results. When studying the importance of parameters, the layer with a relatively large number of parameters is replaced as a parameter region to determine the key layer in the LLM.

[0079] Once the criticality of certain parameter regions in the model is determined, different steganography strategies need to be set for regions of different criticalities to minimize the model performance loss caused by steganography. A simple steganography strategy is to not embed some highly important parameters. However, LLM has strong robustness and generalization ability. Embedding a small number of least significant bit (LSB) steganography bits for highly important parameters will not affect model performance, and embedding a large number of least significant bit (LSB) steganography bits for extremely low-importance parameters (the number of LSB bits cannot be equal to the number of numerical bits) will also not affect model performance.

[0080] The given steganography strategy has the following format:

[0081]

[0082] Among them, parameter∈{θ,τ},N LSB ∈[0,8]if quantize else N LSB ∈[0,16]. For parameters, they are divided into three levels according to their importance: critical, ordinary, and non-critical. For non-quantized scenarios, parameters are measured in layers. For quantized scenarios, parameters are measured in layers and neurons. Therefore, to maximize the amount of steganography while minimizing performance loss, the following general rules should be followed:

[0083]

[0084] S102. Generate the integrity check code of the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0085] The method of secretly embedding binary data into the target parameter location can be designed to include:

[0086] Determine the grouping type of model parameters corresponding to each element in the parameter weight matrix of a large language model;

[0087] If the matrix element is a non-critical parameter, then extract the binary data element according to the number of bits embedded in the non-critical parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the non-critical parameter to update the current matrix element.

[0088] If the matrix elements are ordinary parameters, then extract the binary data elements according to the number of bits embedded in the ordinary parameters and assign them to the first variable. Assign the remaining binary data elements that have not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the ordinary parameters to update the current matrix elements.

[0089] If the matrix element is an important parameter, then determine whether the length of the binary data is less than the number of bits embedded in the important parameter. If it is less, then convert the binary data to the specified data type. If it is not less, then extract the binary data element according to the number of bits embedded in the important parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the important parameter to update the current matrix element.

[0090] When publishers release open-source LLMs, the data types of their parameters are generally float16 or bfloat16 half-precision types. If users do not import and deploy the model using model quantization, the parameter types will still be half-precision types when the LLM is entered into the user's host. By combining neural network LSB steganography and implementing half-precision LSB steganography for half-precision type LLMs, large amounts of data can be hiddenly embedded in the LLM in non-quantization scenarios.

[0091] The half-precision LSB steganography process is shown in Algorithm 1. First, according to the adaptive strategy, the parameter weight matrix W and the corresponding embedding bit positions N1, N2, and N3 are initialized based on the classification of non-critical, ordinary, and important parameters. Second, based on the embedding bit positions N1, N2, and N3, mask matrices MASK1, MASK2, and MASK3 are created for subsequent LSB-based embedding. Third, the SHA256 integrity check code of the payload is calculated, and the payload and check code are packaged into an embedded binary stream s. Simultaneously, it is checked whether s can be completely embedded into W under this strategy. Fourth, s is embedded according to the strategy. Finally, the parameter weight matrix W' of the embedded s is returned.

[0092]

[0093]

[0094]

[0095] Furthermore, based on the aforementioned adaptive hiding method for non-quantized scenarios, this embodiment of the invention also provides a data adaptive hiding system based on a large language model, comprising: an embedding position acquisition module and a steganographic data embedding module, wherein,

[0096] The embedding location acquisition module is used to group the parameters of the large language model according to the degree of influence of each parameter on the model performance, and select the target parameter position for data cryptic embedding from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type and important parameter type.

[0097] The steganography embedding module is used to generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0098] For quantification scenarios, see Figure 5 As shown, the data adaptive hiding method based on a large language model in this embodiment of the invention can be designed to include:

[0099] S101. Use a quantization scaling factor to uniformly linearly quantize the parameters of the initial large language model to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters.

[0100] S102. Based on the degree of influence of each parameter in the quantization model on the model performance, the parameters of the quantization model are grouped, and the target parameter positions for data hidden embedding are selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter types, ordinary parameter types, and important parameter types.

[0101] S103. Generate the integrity check code of the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0102] S104. Dequantize the model with embedded binary data and replace the embedded target parameter location region with the parameter region corresponding to the initial large language model.

[0103] The parameters of the quantization model can be grouped according to their influence on the model performance, and may include:

[0104] Obtain the optimal parameters of the quantization model and record the model performance under the optimal parameters as the maximum value;

[0105] Based on the quantization scaling factor of neurons in the quantization model, neurons with a quantization scaling factor greater than a specified value are selected as parameter replacement regions.

[0106] By using random binary bits to replace the corresponding neuron parameters in the optimal parameter replacement region of the quantized model parameters, the model performance after the neuron parameter replacement is obtained.

[0107] The importance of the corresponding neuron parameters is measured by the absolute value of the model performance loss before and after the replacement, and the neuron parameters in the parameter replacement region are grouped according to the importance of the neuron parameters.

[0108] For LLM in quantization scenarios, the importance of neurons in uniform linear quantization is related to the quantization scaling factor; the larger the scaling factor, the more critical the neuron. The scaling factor determines the spacing between quantization levels, i.e., the ratio between the quantized value and the original value. Under certain assumptions, parameter θ is the optimal parameter for LLM, at which point the LLM performance reaches its maximum. Given... To quantize the scaling factor of the neuron, the quantized parameters of the neuron are as follows:

[0109]

[0110] The parameters obtained by dequantization are as follows:

[0111]

[0112] Where Q is the uniform linear quantization algorithm. Q ′ For the corresponding inverse quantization algorithm, Q ′ S (τ)=Z(τ+Z).

[0113] Assuming that after quantization, τ is also the optimal parameter for LLM, then the model has performance. Logically, changes in model performance are related to the numerical differences before and after parameter modifications; the greater the numerical difference, the greater the performance loss. In quantization scenarios, random bit substitution of τ yields... Compared to τ, a numerical change has occurred, namely The parameters after dequantization are now: With θ ′ In comparison, there will be numerical changes, that is The numerical difference is In other words, numerically, besides the γ caused by bit substitution, it is also determined by the scaling factor S. The larger S is, the greater the numerical variation and the greater the impact on model performance. Utilizing... Figure 6 Let's use an example to further illustrate this. Suppose we have a float16 type θ. α =[200,400,800] and θ β = [0.2, 0.4, 0.8], their quantization and dequantization scales are s respectively. α =100 and s β =0.1. Both quantization results are [2,4,8]. If embedded, the value changes to [3,5,9]. After dequantization, With θ α The numerical difference is 100; while With θ β The numerical difference is only 0.1. Therefore, it can be considered that in quantization scenarios, for neurons with uniform linear quantization, the larger the scaling factor, the greater the importance and the more critical the neuron.

[0114] Quantization is not only a common way for users to import models, enabling lightweight deployment of LLMs, but it can also compromise the integrity of embedded data. Furthermore, the size of a quantized model is significantly smaller than that of a non-quantized model, resulting in noticeable performance overhead when embedding large payloads. Therefore, designing quantized steganography requires consideration of two aspects: how to ensure the integrity of embedded data, and how to reduce the performance overhead caused by embedding.

[0115] If a user deploys a model using model quantization, the data embedded solely through half-precision LSB steganography will have its integrity compromised, rendering the attack ineffective. Therefore, in this embodiment, the quantized model is first embedded, then dequantized to obtain the LLM with embedded data. Using this approach to design quantization steganography effectively ensures the integrity of the embedded data. Specifically, when modifying the quantized int parameter τ, It's still an int type. Now we'll dequantize it. get At this point, we quantify it, and the calculation method is as follows:

[0116]

[0117] After dequantization and requantization, the numerical value remains unchanged. Therefore, the integrity of the data embedded during quantization is not compromised after dequantization and quantization operations, thus enabling the hidden embedding of data information in quantization scenarios.

[0118] To explore how to reduce the performance overhead caused by embedding, this embodiment analyzes the performance changes of the model during the embedding process from the perspective of quantization steganography. For example... Figure 7 As shown, due to the Int operation in the uniform linear quantization algorithm, the decimal part of the float type data is ignored when converting float type data to int type data, thus resulting in a loss in the overall model performance. q During the embedding phase, numerical variations lead to model loss. e During the dequantization stage, there is no loss of numerical accuracy, and the model does not incur any performance loss. The final performance loss of the model is the Loss. all =Loss q +Loss e Loss q The quantization algorithm determines the loss, which is beyond our control. In this embodiment, all we can do is allow the loss to... e To minimize this, during the embedding stage, we consider using the same method as Half-precision LSB steganography, modifying the input parameter type to int. Furthermore, due to the limited length of the embedded content, sometimes not all quantized parameters are embedded. Parameters not embedded suffer precision loss due to quantization and dequantization operations. Therefore, we only need to replace the embedded parameter regions in the original model, maintaining the original precision of the values ​​in the unembedded regions, thereby minimizing the model's precision loss.

[0119] The quantization LSB steganography process is shown in Algorithm 2. The first step is to quantize M to obtain M'. The second step is the same as the half-precision LSB steganography process, except for the input parameter type. This determines that during the embedding masking operation in Quantization LSBsteganography, the binary bit string q is converted to int type instead of float type. The third step is to dequantize M', which contains the embedded payload. Finally, the embedded parameter region in M' is replaced with the original model M, and M is returned as the malicious model M'.

[0120]

[0121] Furthermore, regarding the hiding methods in the aforementioned quantization scenarios, this embodiment of the invention also provides a data adaptive hiding system based on a large language model, comprising: a model parameter quantization module, an embedding position acquisition module, a steganographic data embedding module, and a model parameter dequantization module, wherein...

[0122] The model parameter quantization module is used to uniformly and linearly quantize the initial large language model parameters using a quantization scaling factor to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters.

[0123] The embedding location acquisition module is used to group the parameters of the quantization model according to the degree of influence of each parameter on the model performance, and select the target parameter location for data cryptic embedding from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type and important parameter type.

[0124] The steganography data embedding module is used to generate the integrity check code of the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

[0125] The model parameter dequantization module is used to dequantize the model with embedded binary data and replace the embedded target parameter location region with the parameter region corresponding to the initial large language model.

[0126] For LLM parameter files, key parameters are located using hidden embedding and parameter importance is evaluated based on ablation and quantization scaling factors to identify parameters with minimal impact on LLM performance as key embedding locations. Then, a steganography strategy is formulated based on parameter importance, implementing half-precision LSB steganography and quantized LSB steganography for non-quantized and quantized scenarios respectively. This ensures that large-capacity data information can be hiddenly embedded in the LLM regardless of whether the user quantizes and compresses the model, while maintaining the integrity of the data information.

[0127] For large language model steganography data, a trigger-based parsing strategy can be designed to parse the hidden data information in adaptive steganography. After the trigger is activated, it will obtain all the parameter weights of the LLM. Then, the trigger will extract and execute the embedded data information according to the rules of the steganography strategy. The process of the trigger extracting and parsing the embedded data information is shown in Algorithm 3. The first step is to initialize the parameter weight matrix W and the corresponding embedding bit positions N1, N2, N3 according to the adaptive strategy, classifying parameters as non-critical, ordinary, and important. The second step is to extract the embedded binary string from W based on the length L of the embedded binary string and the embedding bit positions N1, N2, N3. The third step is to separate the checksum and payload, verify the integrity of the embedded data, and decode it.

[0128]

[0129]

[0130] To verify the effectiveness of this solution, the following explanation is provided based on experimental data:

[0131] In the experiment, the chatglm3-6b model parameter file was used. The file format was safetensors, with a total of 6.24 billion parameters and a size of 11.63GB. The PyTorch framework (PyTorch version 2.3.0 + cu121) was used to manipulate the parameter weights. The quantization functions provided by the chatglm3-6b official documentation were used for model quantization, and a corresponding dequantization function was written to perform dequantization. The operating system environment for the experiment was Ubuntu 22.04.4LTS (GNU / Linux 5.15.0-106-generic x86_64). The CPU used included 64 x86_64 architecture AMD EPYC 7282 16-core Processors, with a total memory size of 125.7GB. The GPU included two NVIDIA RTX A6000s (40GB VRAM), one NVIDIA A100-PCIE-40GB (40GB VRAM), and CUDA version 12.2. The performance of the model was evaluated using the MMLU (Massive Multitask Language Understanding) dataset. This dataset consists of multiple-choice questions with four options each, covering 57 subjects including STEM (Science, Technology, Engineering, and Mathematics), humanities, and social sciences. It aims to measure the model's generalization and knowledge transfer capabilities in zero-shot and few-shot scenarios. The performance metric is Performance = C / N, where C represents the number of correctly answered questions and N represents the total number of questions. Embedding capacity is measured by the size of the embedded data. Specifically, the embedded data used in the experiments was a randomly generated stream of binary bits to simulate embedded data in real-world scenarios.

[0132] 1. Layer Importance Analysis

[0133] The aim is to identify the importance of different parameters in the model to determine the key parameters. Experiments are first conducted using layers as the smallest steganographic unit. Chatglm3-6b has a total of 200 parameter layers. For single-dimensional parameter layers, such as normalization layers and bias layers, the amount of content that can be embedded within acceptable performance variations is limited due to the small number of parameters. Therefore, when searching for key layers, only 4*28+2=114 layers are considered. Here, 4*28 represents the self_attention.query_key_value layer, self_attention.dense layer, mlp.dense_h_to_4h layer, and mlp.dense_4h_to_h layer among the 28 blocks contained in the core GLMTransformer.

[0134] For both non-quantized and quantized scenarios, ablation experiments based on random bit substitution were conducted on layers in 28 blocks to explore the relationship between the steganography capacity of each layer and the changes in model performance. It was found that almost all layers exhibit a similar threshold effect; model performance only significantly decreases after exceeding a certain threshold of LSB bits. Taking block0 as an example... Figure 8 The results show that the threshold for the self_attention.query_key_value layer and mlp.dense_h_to_4h layer is 11, and the threshold for the self_attention.dense layer and mlp.dense_4h_to_h layer is 12.

[0135] The reason for this is that steganography primarily affects model performance through changes in parameter values. Embedding parameters with a small LSB (Less than 12) bit depth results in only minor changes in value, which does not impact LLM performance. However, when the LSB bit depth reaches 11 or even 12, the exponent changes, causing more significant fluctuations in value, and the model's performance almost completely disappears.

[0136] To explore the importance of the overall model parameters in their distribution, we used block number as the x-axis, selected an LSB steganography bit depth of 11, and analyzed the performance changes of each layer, such as... Figure 9 As shown, it can be intuitively observed that embedding the parameters of the first two-thirds of the model results in a significantly higher performance loss than embedding the parameters of the last one-third. Therefore, it can be considered that the parameters of the first two-thirds of the model are more critical.

[0137] 2. Neuron Importance Analysis

[0138] For neurons with uniform linear quantization, a scaling factor is preserved during the quantization process. To verify the effectiveness of the method of determining neuron importance through the scaling factor, we took mlp.dense_h_to_4h layers, mlp.dense_4h_to_h layers, and self_attention.dense layers, and for each layer, we selected neurons with the largest and smallest scaling factors in the same proportion for comparative embedding analysis. We performed 5-8 bit LSB steging on both parts and compared the model performance after steging. The proportions were 10%, 30%, and 50%, respectively. The experimental results are as follows: Figure 10 As shown in the figure. Here, Major represents the large scaling factor, and Minor represents the small scaling factor.

[0139] exist Figure 10 It can be observed that for mlp.dense_h_to_4h layer, mlp.dense_4h_to_h layer, and self_attention.dense layer, under the same LSB bit depth, the model performance loss caused by steganography of neurons with small scaling factors is relatively small. In this way, it is proved that in these linear layers, the scaling factor size is an effective indicator for evaluating the importance of neurons.

[0140] 3. Half-precision LSB steganalysis

[0141] We verified that using adaptive steganography can effectively reduce the performance overhead caused by embedding. First, without using any strategy, we applied the same LSB steganography (LSB bit width) to all layers. The performance and maximum embedding size of the model under this condition were used as benchmarks. For example... Figure 11 As shown by the dashed line, when the embedding bit depth reaches 8, the performance impact of LLM is less than 1%, but the hidden data content can reach 5.81GB. However, when the embedding bit depth reaches 11, LLM almost loses all its capabilities.

[0142] Next, the steganography strategy is used: (1) all self_attention.query_key_value layers (block0 to block27) are taken as key layers, the last 1 / 3 (block18 to block27) of mlp.dense_4h_to_h layers are taken as non-key layers, and the remaining non-one-dimensional layers are ordinary layers; (2) ordinary layers are embedded with i bits, key layers are embedded with i-1 bits, and non-key layers are embedded with i+1 bits.

[0143] Under this strategy, compared to the baseline method that embeds i bits across all layers, the amount of embedded data is only 3.83 MB larger, which is negligible, and the embedding capacity of the two strategies can be considered the same. Figure 11 As shown, this embedding strategy significantly improves performance compared to the baseline method when embedding the same amount of data. This demonstrates that adaptive steganography, by embedding fewer important parameters and prioritizing the embedding of less important parameters, can effectively reduce the performance overhead of embedding while embedding as much data as possible, making the embedded information harder for users to discover.

[0144] 4. Quantitative LSB Steganalysis

[0145] In quantization scenarios, the parameter types of the model change, and the size decreases. Taking int8 quantization as an example, the total number of parameters after int8 quantization of chatglm3-6b is still 6.24 billion, and the size is 6.31GB. To verify the effect of using adaptive steganography in quantization scenarios, we first applied LSB steganography with the same LSB bit width to all parameters of the quantized model without using any strategy. The performance and maximum embedding size of the model in this case were used as benchmarks. As shown in Figure 12, indicated by the green dashed line, when the embedding bit width reaches 3, the performance impact of LLM is less than 1%, but the hidden malicious content can reach 2.18GB. When the embedding bit width reaches 5, LLM loses all its capabilities.

[0146] Next, building upon the layer strategy in 6.3.1, a finer-grained strategy was designed based on the quantification of neuron importance. For the non-critical layers (mlp.dense_4h_to_h layers in blocks 18 to 27), neurons with the largest scaling factor (50%) were selected as critical neurons, and neurons with the smallest scaling factor (50%) were selected as non-critical neurons. Critical neurons were embedded one bit less (i bits), while non-critical neurons were embedded one bit more (i+2 bits). For example... Figure 12 As shown in the figure, this adaptive steganography embedding strategy, compared with the baseline layer strategy using only 6.3.1, as indicated by the red dashed line in the figure, further improves the model performance and makes the embedded information more difficult for users to perceive.

[0147] Although steganography based on adaptive strategies can effectively reduce the performance loss caused by embedding, the finer the strategy, the more complex its logic and the more complex the strategy for parsing the trigger, which increases the size of the trigger code and makes the trigger easier to detect.

[0148] The experimental data above demonstrates that the proposed solution can embed large amounts of data (over 5.81GB and 2.18GB) in both non-quantized and quantized models, respectively, with a performance loss of less than 1%. Regardless of whether the user quantizes or compresses the model, the solution can conceal large amounts of data within the LLM while ensuring the integrity of the data, further proving the effectiveness of the proposed solution.

[0149] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0150] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0151] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0152] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0153] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.< / eos>

Claims

1. A data adaptive hiding method based on a large language model, characterized in that, Include: The optimal parameters of a large language model are obtained, and the performance of the large language model under the optimal parameters is recorded as the maximum value. Based on the number of parameters in each layer of the large language model structure, neuron layers with a number of parameters greater than a specified value are selected as parameter replacement regions. Random binary bits are used to replace the neuron parameters in the optimal parameters of the parameter replacement regions of the large language model, and the performance of the large language model after the neuron parameter replacement is obtained. The absolute value of the performance loss of the large language model before and after the replacement is used to measure the importance of the corresponding neuron parameter. Based on the importance of the neuron parameters, the neuron parameters in the parameter replacement regions are grouped, and the target parameter positions for data latent embedding are selected from the grouped model parameters. The model parameters are grouped into non-critical parameter types, ordinary parameter types, and important parameter types. Selecting the target parameter position for data cryptic embedding from the grouped model parameters specifically includes: initializing the embedding bit lengths of the non-critical, ordinary, and important parameters of the large language model; concatenating the weight matrices of the non-critical, ordinary, and important parameters to obtain the large language model parameter weight matrix, which stores the connection weights between neurons in the large language model used for information transmission; and creating corresponding mask matrices for least significant bit embedding based on the embedding bit lengths of the non-critical, ordinary, and important parameters of the large language model. Generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter location.

2. The data adaptive hiding method based on a large language model according to claim 1, characterized in that, The binary data is secretly embedded into the target parameter location, including: Determine the grouping type of model parameters corresponding to each element in the parameter weight matrix of a large language model; If the matrix element is a non-critical parameter, then extract the binary data element according to the number of bits embedded in the non-critical parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the non-critical parameter to update the current matrix element. If the matrix elements are ordinary parameters, then extract the binary data elements according to the number of bits embedded in the ordinary parameters and assign them to the first variable. Assign the remaining binary data elements that have not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the ordinary parameters to update the current matrix elements. If the matrix element is an important parameter, then determine whether the length of the binary data is less than the number of bits embedded in the important parameter. If it is less, then convert the binary data to the specified data type. If it is not less, then extract the binary data element according to the number of bits embedded in the important parameter and assign it to the first variable. Assign the remaining binary data element that has not been extracted to the second variable. Convert the first variable to the specified data type and perform bitwise operations on the converted first variable based on the mask matrix of the important parameter to update the current matrix element.

3. A data adaptive hiding system based on a large language model, characterized in that, It includes: an embedding location acquisition module and a steganography data embedding module, wherein, The embedding location acquisition module is used to obtain the optimal parameters of the large language model and record the performance of the large language model under the optimal parameters as the maximum value. Based on the parameter size of each layer in the large language model structure, the neuron layer with a parameter size greater than a specified value is selected as the parameter replacement region. Random binary bits are used to replace the neuron parameters in the optimal parameters of the parameter replacement region of the large language model. The performance of the large language model after the neuron parameter replacement is obtained. The absolute value of the performance loss of the large language model before and after the replacement is used to measure the importance of the corresponding neuron parameter. Based on the importance of the neuron parameters, the neuron parameters in the parameter replacement region are grouped, and the target for data hidden embedding is selected from the grouped model parameters. The parameter location, where the model parameters are grouped into non-critical parameter types, ordinary parameter types, and important parameter types, involves selecting the target parameter location for data cryptic embedding from the grouped model parameters. Specifically, this includes: initializing the embedding bit lengths of the non-critical, ordinary, and important parameters of the large language model; concatenating the weight matrices of these three parameters to obtain the large language model parameter weight matrix, which stores the connection weights used to transmit information between neurons in the large language model; and creating corresponding mask matrices for least significant bit embedding based on the embedding bit lengths of the non-critical, ordinary, and important parameters of the large language model. The steganography embedding module is used to generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position.

4. A data adaptive hiding method based on a large language model, characterized in that, Include: The initial large language model parameters are uniformly linearly quantized using a quantization scaling factor to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters. The optimal parameters of the quantization model are obtained, and the model performance under the optimal parameters is recorded as the maximum value. Based on the quantization scaling factor of the neurons in the quantization model, neurons with a quantization scaling factor greater than a specified value are selected as parameter replacement regions. Random binary bits are used to replace the corresponding neuron parameters in the optimal parameters of the parameter replacement regions of the quantization model. The model performance after the neuron parameter replacement is obtained, and the absolute value of the model performance loss before and after replacement is used to measure the importance of the corresponding neuron parameter. Based on the importance of the neuron parameters, the neuron parameters in the parameter replacement regions are grouped, and the target parameter position for data cryptic embedding is selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type, and important parameter type. The selection of the target parameter position for data cryptic embedding from the grouped model parameters specifically includes: initializing the embedding bit of non-critical parameters, ordinary parameters, and important parameters of the quantization model, and concatenating the weight matrices of non-critical parameters, ordinary parameters, and important parameters to obtain the quantization model parameter weight matrix. The parameter weight matrix is ​​used to store the connection weights used to transmit information between each neuron in the quantization model; and creating corresponding mask matrices for least significant bit embedding based on the embedding bit of non-critical parameters, ordinary parameters, and important parameters of the quantization model. Generate an integrity check code for the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position; The model embedding binary data is dequantized, and the embedded target parameter location region is replaced with the parameter region corresponding to the initial large language model.

5. A data adaptive hiding system based on a large language model, characterized in that, It includes: a model parameter quantization module, an embedding location acquisition module, a steganography data embedding module, and a model parameter dequantization module. The model parameter quantization module is used to uniformly and linearly quantize the initial large language model parameters using a quantization scaling factor to obtain a quantized model. The quantization scaling factor is set according to the upper and lower bounds of the true values ​​of the large language model parameters. The embedding location acquisition module is used to obtain the optimal parameters of the quantization model and record the model performance under the optimal parameters as the maximum value. Based on the quantization scaling factor of the neurons in the quantization model, neurons with a quantization scaling factor greater than a specified value are selected as parameter replacement regions. Random binary bits are used to replace the corresponding neuron parameters in the optimal parameters of the parameter replacement regions of the quantization model, and the model performance after the neuron parameter replacement is obtained. The absolute value of the model performance loss before and after replacement is used to measure the importance of the corresponding neuron parameter. Based on the importance of the neuron parameters, the neuron parameters in the parameter replacement regions are grouped, and the target parameter position for data cryptic embedding is selected from the grouped model parameters. The grouping types of model parameters include non-critical parameter type, ordinary parameter type, and important parameter type. The specific steps for selecting the target parameter position for data cryptic embedding from the grouped model parameters include: initializing the embedding bit of non-critical parameters, ordinary parameters, and important parameters of the quantization model, and concatenating the weight matrices of non-critical parameters, ordinary parameters, and important parameters to obtain the quantization model parameter weight matrix. The parameter weight matrix is ​​used to store the connection weights used to transmit information between each neuron of the quantization model; and creating corresponding mask matrices for least significant bit embedding based on the embedding bit of non-critical parameters, ordinary parameters, and important parameters of the quantization model. The steganography data embedding module is used to generate the integrity check code of the data to be steganized, package the data to be steganized and its integrity check code into binary data, and secretly embed the binary data into the target parameter position. The model parameter dequantization module is used to dequantize the model with embedded binary data and replace the embedded target parameter location region with the parameter region corresponding to the initial large language model.

6. An electronic device, characterized in that, include: At least one processor, and a memory coupled to said at least one processor; The memory stores a computer program that can be executed by the at least one processor to implement the method as described in any one of claims 1 to 2 and 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, enables the implementation of the method as described in any one of claims 1 to 2 and 4.

Images