K time series-based differential privacy model training method, medium and system
Patent Information
- Application Number
- CN202611097819.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-08-21
AI Technical Summary
但当训练步数很长时，许多次独立的随机扰动会逐渐积累，造成总体的偏差越来越大
[0035]根据本发明实施例的基于K时间序列的差分隐私模型训练方法，在当步数据获取之后，基于当步数据生成对应的本步系数向量，如此，通过本步系数向量，进而根据本步系数向量以有限记忆的线性方式生成每一步噪声；让“后来的噪声”适度抵消“前面的噪声”；可有效降低累计误差。同时，使用安全修正包络把相关性稳妥地折算进意义不的隐私成本，以使得合规账本依然可加、可审计、可复合；这意味着，虽然我们在时间改变了噪声的结构，但在账本上每一步仍然是一笔“可累加”的明细，而且这笔明细不会低估真实的隐私开销。这样，行业普遍采用的RDP记账与泊松子采样放大可以原封不动地沿用，隐私报告的生成与复核流程也不会被打破。
Smart Images
Figure CN122616638A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of differential privacy model technology, and in particular to a training method, medium and system for a differential privacy model based on K-time series. Background Technology
[0002] In recent years, large language models have rapidly gained popularity in highly sensitive industries such as healthcare, finance, and government. Unlike general-purpose models, business-specific models often require supervised fine-tuning using the institution's own data. For example, real-world case dialogues can be used to improve the quality of medical Q&A, while transaction and customer service records can be used to optimize financial risk control and services. This data naturally contains sensitive personal information, and any improper handling during training could potentially lead to the "reverse deduction" of individual information through the model's output, triggering compliance risks and legal liabilities. Therefore, the training and fine-tuning process itself must be auditable, quantifiable, and reproducibly demonstrable that "the impact on individuals is within a controlled range."
[0003] Differential privacy emerged as the most mainstream verifiable privacy framework in the field of machine learning. Its intuitive meaning is that the distribution of the algorithm's output is "almost identical" regardless of whether a specific individual appears in the training set. In other words, no observer can confidently determine whether someone's data has been used. Differential privacy also has two very important engineering features: First, it provides a uniform scale for the "privacy budget," allowing developers and compliance departments to manage it like a financial budget; second, it supports "additive accounting" for multiple processing and multi-step training, facilitating the breakdown, accumulation, auditing, and final reporting of the entire training process.
[0004] In related technologies, when applying differential privacy to model training, independent noise is often added to each training step. In this mode, the impact of a single noise event may be small. However, when the number of training steps is long, many independent random perturbations gradually accumulate, causing the overall bias to become increasingly larger. This leads to three intuitive consequences: the model has a harder time entering the effective convergence region in the early stages; more training epochs are often required to achieve the same accuracy; and, given a constant privacy budget, the final accuracy will be significantly lower than that of non-privacy training. Summary of the Invention
[0005] This invention aims to at least partially address one of the technical problems in related technologies. To this end, one objective of this invention is to propose a training method for differential privacy models based on K-time series, which can improve the training efficiency of differential privacy models and simultaneously increase the accuracy of the final differential privacy model.
[0006] In a first aspect, embodiments of the present invention propose a training method for a differential privacy model based on K-time series, comprising the following steps: sampling the current step data according to a preset sampling rate to obtain a corresponding sampling batch, calculating the average gradient corresponding to the sampling batch, and calculating the corresponding unit direction based on the average gradient; calculating the current step coefficient vector corresponding to the current step data based on preset weight values, wherein the current step coefficient vector includes elements corresponding to the historical K steps; performing element-wise absolute value taking and weighted summation on the current step coefficient vector to generate the current step reference population strength; calculating the corresponding variance amplification factor based on the current step coefficient vector, and determining the security correction envelope based on the variance amplification factor and the current step reference population strength; calculating the corresponding noise scale based on the security correction envelope and the ledger quota for each step, and determining the final vector noise based on the current step coefficient vector, the noise scale, and the unit direction; and updating the differential privacy model parameters based on the average gradient, the final vector noise, and the historical parameter vector.
[0007] According to an embodiment of the present invention, a training method for a differential privacy model based on K-time series is as follows: First, the current step data is sampled according to a preset sampling rate to obtain a corresponding sampling batch, and the average gradient corresponding to the sampling batch is calculated, as well as the corresponding unit direction is calculated based on the average gradient. Next, the current step coefficient vector corresponding to the current step data is calculated based on preset weight values, wherein the current step coefficient vector includes elements corresponding to the historical K steps. Then, the current step coefficient vector is weighted and summed after taking the absolute value of each element to generate the current step reference population strength. Next, the corresponding variance amplification factor is calculated based on the current step coefficient vector, and a security correction envelope is determined based on the variance amplification factor and the current step reference population strength. Then, the corresponding noise scale is calculated based on the security correction envelope and the ledger quota for each step, and the final vector noise is determined based on the current step coefficient vector, the noise scale, and the unit direction. Finally, the differential privacy model parameters are updated based on the average gradient, the final vector noise, and the historical parameter vectors. This improves the training efficiency of the differential privacy model and the accuracy of the final differential privacy model.
[0008] In some embodiments, calculating the average gradient corresponding to the sampling batch includes: calculating the gradient corresponding to each sample in the sampling batch and clipping the gradient corresponding to each sample; and averaging the clipped gradients to obtain the average gradient corresponding to the sampling batch.
[0009] In some embodiments, calculating the corresponding unit direction based on the average gradient includes: calculating the norm of the average gradient and determining whether the norm is greater than 0; if so, dividing the average gradient by the norm to obtain the unit direction; if not, treating the current step data as invalid data.
[0010] In some embodiments, the coefficient vector for this step is calculated using the following formula:
[0011]
[0012] in, Indicates the first The coefficient vector of this step. This represents the element corresponding to the first step in the history. Indicates the first in history The element corresponding to the step;
[0013]
[0014] in, Indicates the first in history The weight value corresponding to each step.
[0015] In some embodiments, the current step reference total strength, the variance amplification factor, the safety correction envelope, and the noise scale are calculated using the following formula:
[0016]
[0017] in, Indicates the overall strength as referenced at the current step. Indicates the first The non-negative weights of each reference component;
[0018]
[0019] in, Indicates the variance amplification factor;
[0020]
[0021] in, Indicates a safety correction envelope;
[0022]
[0023] in, The variance representing the noise scale. Indicates the current budget, Represents the gradient of a single sample The square of the upper bound of the norm;
[0024]
[0025] in, Indicates the noise level.
[0026] In some embodiments, the final vector noise is calculated using the following formula:
[0027]
[0028] in, Indicates intermediate parameters. Represents a random number. Indicates the first in history The element corresponding to the step, Indicates the reference width;
[0029]
[0030] in, This represents the final vector noise. Indicates the noise level. Indicates the direction of the unit.
[0031] In some embodiments, the method further includes: calculating the current step cost based on the security correction envelope and the noise scale, and amplifying the security correction envelope, the current step cost, and the noise scale based on an amplification function to generate the current step privacy cost and its cumulative ledger; and generating a final privacy report based on the current step privacy cost and the cumulative ledger.
[0032] In some embodiments, the method further includes: calculating directional similarity based on the unit direction corresponding to the current step data and the unit direction corresponding to the previous step data; determining whether the current step data is in an overly correlated state based on the directional similarity and the current step reference overall strength; if so, performing a degradation operation to degrade the noise scale.
[0033] Secondly, embodiments of the present invention propose a computer-readable storage medium storing a K-time series-based differential privacy model training program, which, when executed by a processor, implements the K-time series-based differential privacy model training method as described above.
[0034] Thirdly, this invention proposes a differential privacy model training system based on K-time series, characterized by comprising: a sampling module, used to sample the current step data according to a preset sampling rate to obtain a corresponding sampling batch, and to calculate the average gradient corresponding to the sampling batch, and to calculate the corresponding unit direction based on the average gradient; a calculation module, used to calculate the current step coefficient vector corresponding to the current step data based on preset weight values, wherein the current step coefficient vector includes elements corresponding to the previous K steps; and a weighting module, used to take the absolute value of each element of the current step coefficient vector. The following steps are performed: a weighted aggregation to generate the current step reference population strength; a correction module, which calculates the corresponding variance amplification factor based on the current step coefficient vector and determines the security correction envelope based on the variance amplification factor and the current step reference population strength; a noise calculation module, which calculates the corresponding noise scale based on the security correction envelope and the ledger quota for each step, and determines the final vector noise based on the current step coefficient vector, the noise scale, and the unit direction; and a parameter update module, which updates the differential privacy model parameters based on the average gradient, the final vector noise, and the historical parameter vector.
[0035] According to the K-time series-based differential privacy model training method of this invention, after the acquisition of the current step data, a corresponding coefficient vector for that step is generated based on the current step data. Thus, using this current step coefficient vector, noise for each step is generated in a linear manner with finite memory; allowing "later noise" to moderately cancel "previous noise," effectively reducing cumulative error. Simultaneously, a secure correction envelope is used to reliably factor the correlation into the meaningful privacy cost, ensuring that the compliance ledger remains additive, auditable, and composable. This means that although the structure of the noise changes over time, each step in the ledger remains an "accumulative" detail, and this detail does not underestimate the actual privacy overhead. In this way, the industry-standard RDP accounting and Poisson sampling amplification can be used unchanged, and the generation and review process of privacy reports will not be disrupted.
[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0037] Figure 1 This is a flowchart illustrating the training method for a differential privacy model based on K-time series according to an embodiment of the present invention.
[0038] Figure 2 This is a block diagram of a differential privacy model training system based on K-time series according to an embodiment of the present invention. Detailed Implementation
[0039] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0040] The following describes a method for training a differential privacy model based on K-time series according to an embodiment of the present invention with reference to the accompanying drawings.
[0041] It should be noted that the trained differential privacy model can be used in user behavior data analysis, personalized recommendation systems, traffic statistics (e.g., foot traffic statistics in commercial areas, app page visit traffic statistics, etc.), finance, healthcare, and cross-institutional collaborative modeling in government, without limiting its specific domain. In other words, during the training process of this application, the training data can be historical user behavior data, historical user privacy preference data, historical traffic access data, historical financial data, historical medical data, etc.
[0042] Please see Figure 1 , Figure 1 This is a flowchart illustrating the training method for a differential privacy model based on K-time series; for example... Figure 1 As shown, the training method for the differential privacy model based on K-time series includes the following steps:
[0043] S101, sample the current step data according to the preset sampling rate to obtain the corresponding sampling batch, calculate the average gradient corresponding to the sampling batch, and calculate the corresponding unit direction based on the average gradient.
[0044] As an example, firstly, the data for the current step is obtained according to the preset sampling rate; then, the probability of sampling per step is determined; if the probability is greater than 0, Poisson sampling is used to obtain the corresponding sampling batch; if the probability is less than or equal to 0, uniform sampling is used to obtain the corresponding sampling batch.
[0045] In some embodiments, calculating the average gradient corresponding to the sampling batch includes: calculating the gradient corresponding to each sample in the sampling batch and clipping the gradient corresponding to each sample; and averaging the clipped gradients to obtain the average gradient corresponding to the sampling batch.
[0046] In some embodiments, calculating the corresponding unit direction based on the average gradient includes: calculating the norm of the average gradient and determining whether the norm is greater than 0; if so, dividing the average gradient by the norm to obtain the unit direction; if not, treating the current step data as invalid data.
[0047] As an example, first, the gradient corresponding to each sample in the sampling batch is calculated using the current parameters; then, these gradients are clipped one by one so that each gradient does not exceed the upper limit; finally, the clipped gradients are averaged to obtain the average gradient.
[0048]
[0049] in, Represents the average gradient. Indicates in the parameter Sampling batch The calculated loss gradient, This indicates the maximum clipping limit.
[0050] Next, calculate the norm corresponding to the average gradient and determine whether the norm is greater than 0;
[0051] If so, divide the average gradient by its norm to obtain the corresponding unit direction;
[0052] If not, then the current data will be considered invalid:
[0053]
[0054] in, Indicates the direction of the unit.
[0055] S102, calculate the coefficient vector of the current step corresponding to the data of the current step based on the preset weight value, wherein the coefficient vector of the current step includes the elements corresponding to the historical K steps.
[0056] In some embodiments, the coefficient vector in this step is calculated using the following formula:
[0057]
[0058] in, Indicates the first The coefficient vector of this step. This represents the element corresponding to the first step in the history. Indicates the first in history The element corresponding to the step;
[0059]
[0060] in, Indicates the first in history The weight value corresponding to each step.
[0061] As an example, firstly, given the reference width The upper bound of the coefficients is 1>β>0. The coefficient vector in this step (i.e., the first...) (step coefficient vector) Fill in the positions in the order of "looking back K steps from near to far". Indicates the first in history step .
[0062] Thus, for each position The following piecewise formula can be used for calculation:
[0063]
[0064] in, Indicates the first in history The weight value corresponding to each step. That is, the coefficient vector of this step is regarded as a row of K cells, which correspond to the historical step numbers in turn. If a step exists, the number in the interval [−β,β] is filled in, and if it does not exist, 0 is filled in.
[0065] S103, take the absolute value of each element of the coefficient vector in this step and then sum them up in a weighted manner to generate the reference overall strength in this step.
[0066] S104, calculate the corresponding variance magnification factor based on the coefficient vector of this step, and determine the safety correction envelope based on the variance magnification factor and the strength of the reference population in this step.
[0067] S105, calculate the corresponding noise scale based on the security correction envelope and the ledger quota for each step, and determine the final vector noise according to the coefficient vector of this step, the noise scale and the unit direction.
[0068] It's important to clarify that the ledger quota per step refers to a pre-allocated portion of the "total privacy budget" before training begins. This quota is determined by a combination of two factors from the previous step: the "reference overall strength" and the "variance amplification factor." Generally, steps with strong correlations / larger amplification are allocated fewer slots, while more secure and stable steps are allocated more. Each step is only allowed to use its own quota, which is used to determine the noise scale for that step and serves as the upper limit for privacy consumption, ensuring that the budget is not exceeded throughout the process. If subsampling exists, a "packaging" process is performed before updating the ledger using industry-standard amplification mapping, but the principle that "each step can only use its own quota" remains unchanged.
[0069] In some embodiments, the step reference total strength, variance amplification factor, safety correction envelope, and noise scale are calculated using the following formulas:
[0070]
[0071] in, Indicates the overall strength as referenced at the current step. Indicates the first The non-negative weights of each reference component;
[0072]
[0073] in, Indicates the variance amplification factor;
[0074]
[0075] in, Indicates a safety correction envelope;
[0076]
[0077] in, The variance representing the noise scale. Indicates the current budget, Represents the gradient of a single sample The square of the upper bound of the norm;
[0078]
[0079] in, Indicates the noise level.
[0080] As an example, calculate the current reference total strength:
[0081]
[0082] In other words, take the absolute value of each cell in the coefficient vector of the current step, multiply it by the weight, and calculate the sum, where... This indicates a preset weight value (e.g., the mean can be selected); then, if the result is out of bounds, it is truncated to [0,1].
[0083] Next, calculate the variance amplification factor:
[0084]
[0085] In other words, starting from 1, the squares of each available element are accumulated to obtain the variance amplification factor;
[0086] Then, calculate the safety correction envelope:
[0087]
[0088] In other words, two candidates are obtained (one based on the variance amplification factor and the other based on the current reference population strength), and the more conservative one is taken as the safety correction envelope, and its value is limited to [0,1].
[0089] Next, the variance of the noise scale is calculated:
[0090]
[0091] For each Obtain the budget for this step. Substitute the existing parameters into a fixed template to calculate a candidate noise scale. Then, use all... The maximum value in the range is taken as the final value of this step.
[0092] Then, the noise scale can be obtained:
[0093]
[0094] in, Indicates the noise level.
[0095] In some embodiments, the final vector noise is calculated using the following formula:
[0096]
[0097] in, Indicates intermediate parameters. Represents a random number. Indicates the first in history The element corresponding to the step, Indicates the reference width;
[0098]
[0099] in, This represents the final vector noise. Indicates the noise level. Indicates the direction of the unit.
[0100] S106 updates the differential privacy model parameters based on the average gradient, final vector noise, and historical parameter vectors.
[0101] As an example, the parameter update vector can be calculated using the following formula:
[0102]
[0103] in, Represents the parameter update vector (i.e., the first...) (parameter vector before step update) Represents the history parameter vector (i.e., the current step, the first step, the second step, the third step, the fourth (parameter vector before step update) Indicates the first The learning rate of the step. This represents the final vector noise. This represents the average gradient.
[0104] In some embodiments, the method further includes: calculating the current cost based on the security correction envelope and the noise scale, and amplifying the security correction envelope, the current cost, and the noise scale based on an amplification function to generate the current privacy cost and its cumulative ledger; and generating a final privacy report based on the current privacy cost and the cumulative ledger.
[0105] As an example, to maintain full compatibility with industry-standard accounting practices and minimize engineering modifications, firstly, let's assume... For RDP order total, For a specific order, the cost at that step can be calculated:
[0106]
[0107] Next, the cumulative cost can be obtained:
[0108]
[0109] Then, the packet size (i.e., the security correction envelope, the current step cost, and the noise scale) is fed into the subsampling amplification function to calculate the current step privacy cost:
[0110]
[0111] This leads to the cumulative ledger:
[0112]
[0113] It should be noted that when the average probability of drawing a sample in each step is sufficiently small, a linear upper bound can be used for approximation, that is, using linear approximation results in minimal error; otherwise, the exact form should be used for calculation.
[0114] Then, summarize. (or after subsampling) According to standard RDP The final result is obtained through conversion. (The formulas are general; accounting tools can be provided directly.) The report includes: General ledger and distributed ledger, whether subsampling is used and its q, and the time points and reasons for all "switch to precise magnification".
[0115] In some embodiments, the method further includes: calculating directional similarity based on the unit direction corresponding to the current step data and the unit direction corresponding to the previous step data; determining whether the current step data is in an overly correlated state based on the directional similarity and the current step reference overall strength; if so, performing a degradation operation to degrade the noise scale.
[0116] As an example, to ensure stability and maintain a conservative approach, directional similarity and the current reference overall strength are monitored online, where directional similarity is calculated using the following formula:
[0117]
[0118] in, Indicates directional similarity. Indicates the first The direction of the unit of step.
[0119] Next, determine whether >0.5 (referencing the overall strength at this step) or " "Too large and the coefficient is close to the upper bound"; if so, it indicates that the data in this step is in an overly strong correlation state, and a downgrade operation should be performed (specifically, it can be changed to use...). Overall reduction Short-term degradation K=1, or in extreme cases, let ), which degenerates into independent noise.
[0120] It should be noted that once the premise of the linear upper bound is found to no longer hold, a switch should be made immediately. The system provides precise zoom-in, and all switching points are logged for auditing purposes.
[0121] In some embodiments, regarding archiving, the final parameters may be saved. Random number seed; key hyperparameters (K, β, { ,C,lr t ); and key trajectories ( , , , The chain of evidence—from the formation of the process to the outcome—can be verified by a third party.
[0122] In summary, the differential privacy model training method based on K-time series according to embodiments of the present invention firstly samples the current step data according to a preset sampling rate to obtain a corresponding sampling batch, and calculates the average gradient corresponding to the sampling batch, and calculates the corresponding unit direction based on the average gradient; then, calculates the current step coefficient vector corresponding to the current step data based on preset weight values, wherein the current step coefficient vector includes elements corresponding to the historical K steps; then, the current step coefficient vector is weighted and summed after taking the absolute value of each element to generate the current step reference population strength; next, the corresponding variance amplification factor is calculated based on the current step coefficient vector, and a security correction envelope is determined based on the variance amplification factor and the current step reference population strength; then, the corresponding noise scale is calculated based on the security correction envelope and the ledger quota for each step, and the final vector noise is determined based on the current step coefficient vector, the noise scale, and the unit direction; finally, the differential privacy model parameters are updated based on the average gradient, the final vector noise, and the historical parameter vectors; thereby improving the training efficiency of the differential privacy model and the accuracy of the final differential privacy model.
[0123] Secondly, embodiments of the present invention propose a computer-readable storage medium storing a K-time series-based differential privacy model training program, which, when executed by a processor, implements the K-time series-based differential privacy model training method as described above.
[0124] Thirdly, such as Figure 2 As shown in the figure, this embodiment of the invention proposes a differential privacy model training system based on K-time series, characterized in that it includes: a sampling module 10, a calculation module 20, a weighting module 30, a correction module 40, a noise calculation module 50, and a parameter update module 60.
[0125] The sampling module 10 is used to sample the current step data according to a preset sampling rate to obtain the corresponding sampling batch, calculate the average gradient corresponding to the sampling batch, and calculate the corresponding unit direction based on the average gradient.
[0126] The calculation module 20 is used to calculate the coefficient vector of the current step corresponding to the data of the current step based on the preset weight value, wherein the coefficient vector of the current step includes the elements corresponding to the historical K steps;
[0127] The weighting module 30 is used to perform element-wise absolute value taking and weighted summation on the coefficient vector of this step to generate the reference overall strength of this step;
[0128] The correction module 40 is used to calculate the corresponding variance magnification factor based on the coefficient vector of this step, and to determine the safety correction envelope based on the variance magnification factor and the strength of the reference population in this step;
[0129] The noise calculation module 50 is used to calculate the corresponding noise scale based on the security correction envelope and the ledger quota for each step, and to determine the final vector noise based on the coefficient vector of this step, the noise scale and the unit direction;
[0130] The parameter update module 60 is used to update the parameters of the differential privacy model based on the average gradient, the final vector noise, and the historical parameter vector.
[0131] It should be noted that the above description of the training method for the differential privacy model based on K-time series also applies to this training system for the differential privacy model based on K-time series, and will not be repeated here.
[0132] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0133] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0134] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0135] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0136] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0137] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0138] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.
[0139] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for training a differential privacy model based on K-time series, characterized in that, Includes the following steps: The data of the current step is sampled according to a preset sampling rate to obtain the corresponding sampling batch, and the average gradient corresponding to the sampling batch is calculated, and the corresponding unit direction is calculated according to the average gradient. The coefficient vector for the current step is calculated based on the preset weight value, wherein the coefficient vector for the current step includes elements corresponding to the historical K steps; The absolute values of each element in the coefficient vector for this step are taken and then weighted and summed to generate the overall reference strength for this step. Calculate the corresponding variance magnification factor based on the coefficient vector of this step, and determine the safety correction envelope based on the variance magnification factor and the current reference population strength; The corresponding noise scale is calculated based on the security correction envelope and the ledger quota for each step, and the final vector noise is determined according to the coefficient vector of this step, the noise scale, and the unit direction. The differential privacy model parameters are updated based on the average gradient, the final vector noise, and the historical parameter vector.
2. The method for training a differential privacy model based on K-time series as described in claim 1, characterized in that, Calculating the average gradient corresponding to the sampling batch includes: Calculate the gradient corresponding to each sample in the sampling batch, and clip the gradient corresponding to each sample; The average gradient corresponding to the sample batch is obtained by averaging the clipped gradient.
3. The training method for a differential privacy model based on K-time series as described in claim 1, characterized in that, The unit direction is calculated based on the average gradient, including: Calculate the norm of the average gradient and determine whether the norm is greater than 0; If so, the average gradient is divided by the norm to obtain the unit direction; If not, then the data at that step is considered invalid.
4. The method for training a differential privacy model based on K-time series as described in claim 1, characterized in that, The coefficient vector for this step is calculated using the following formula: in, Indicates the first The coefficient vector of this step. This represents the element corresponding to the first step in the history. Indicates the first in history The element corresponding to the step; in, Indicates the first in history The weight value corresponding to each step.
5. The method for training a differential privacy model based on K-time series as described in claim 4, characterized in that, The current reference total strength, the variance amplification factor, the safety correction envelope, and the noise scale are calculated using the following formulas: in, Indicates the overall strength as referenced at the current step. Indicates the first The non-negative weights of each reference component; in, Indicates the variance amplification factor; in, Indicates a safety correction envelope; in, The variance representing the noise scale. Indicates the current budget, Represents the gradient of a single sample The square of the upper bound of the norm; in, Indicates the noise level.
6. The method for training a differential privacy model based on K-time series as described in claim 1, characterized in that, The final vector noise is calculated using the following formula: in, Indicates intermediate parameters. Represents a random number. Indicates the first in history The element corresponding to the step, Indicates the reference width; in, This represents the final vector noise. Indicates the noise level. Indicates the direction of the unit.
7. The method for training a differential privacy model based on K-time series as described in claim 1, characterized in that, Also includes: The current step cost is calculated based on the security correction envelope and the noise scale, and the security correction envelope, the current step cost, and the noise scale are amplified based on the amplification function to generate the current step privacy cost and its cumulative ledger. A final privacy report is generated based on the current privacy cost and the cumulative ledger.
8. The method for training a differential privacy model based on K-time series as described in claim 1, characterized in that, Also includes: Calculate the directional similarity based on the unit direction corresponding to the current step data and the unit direction corresponding to the previous step data. Based on the directional similarity and the current step reference overall strength, determine whether the current step data is in an overly strong correlation state; If so, a degradation operation is performed to degrade the noise scale.
9. A computer-readable storage medium, characterized in that, It stores a K-time series-based differential privacy model training program, which, when executed by a processor, implements the K-time series-based differential privacy model training method as described in any one of claims 1-8.
10. A training system for a differential privacy model based on K-time series, characterized in that, include: The sampling module is used to sample the current step data according to a preset sampling rate to obtain the corresponding sampling batch, calculate the average gradient corresponding to the sampling batch, and calculate the corresponding unit direction based on the average gradient. The calculation module is used to calculate the coefficient vector of the current step corresponding to the data of the current step based on the preset weight value, wherein the coefficient vector of the current step includes the elements corresponding to the historical K steps; The weighting module is used to perform element-wise absolute value taking and weighted summation on the coefficient vector of the current step to generate the reference overall strength of the current step; The correction module is used to calculate the corresponding variance amplification factor based on the coefficient vector of the current step, and to determine the safety correction envelope based on the variance amplification factor and the reference population strength of the current step. The noise calculation module is used to calculate the corresponding noise scale based on the security correction envelope and the ledger quota for each step, and to determine the final vector noise based on the coefficient vector of this step, the noise scale and the unit direction; A parameter update module is used to update the parameters of the differential privacy model based on the average gradient, the final vector noise, and the historical parameter vector.