User privacy regression modeling method fusing multi-platform data

Abstract

The invention discloses a user privacy regression modeling method fusing multi-platform data, and relates to the technical field of privacy modeling, and the method comprises the steps: employing a hierarchical attention mechanism to carry out the preprocessing of user behavior data, outputting a feature vector, and according to the attention weight generated by the hierarchical attention mechanism, obtaining a feature vector; and dynamically distributing privacy budget by adopting an exponential decay function, carrying out cross-platform balance through a global privacy budget pool to obtain a personalized privacy budget distribution scheme, and locally carrying out distributed regression training on each data source based on the feature vector and the personalized privacy budget distribution scheme to obtain a personalized privacy budget distribution scheme. After noise adaptive to the privacy budget is added to the gradient, a global regression model is formed through secure aggregation; according to the method, local generalization errors of all platforms are summarized into a global error signal, and the global error signal is reversely propagated to an attention layer and closed-loop optimization of an exponential attenuation function, so that real-time cooperative adjustment of the attention weight and the privacy noise intensity is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of privacy modeling, in particular to a user privacy regression modeling method fusing multi-platform data. BACKGROUND

[0002] With the rapid development of digital economy, user behavior data presents significant multi-platform distribution characteristics, containing rich individual portrait information. In regression modeling scenarios such as credit evaluation, precision marketing, and health risk prediction, single-platform data often has problems such as insufficient sample coverage and single feature dimension, making it difficult to meet the needs of high-precision continuous value prediction. To break through the data island restriction, federated learning, as a typical distributed machine learning paradigm, is widely studied. The core idea is to realize multi-party collaborative training through secure aggregation of model parameters or gradients without directly exchanging original data, and in theory, differential privacy mechanism can be combined to add noise to the gradient to provide strict mathematical privacy guarantee. In recent years, horizontal federated learning, vertical federated learning, and secure multi-party computing technology based on secret sharing or homomorphic encryption have gradually matured and achieved certain application results in financial risk control, medical auxiliary diagnosis and other fields.

[0003] However, the existing privacy modeling method still has room for improvement: first, most schemes use a unified privacy budget allocation strategy for all participating parties or all features, which cannot distinguish according to the data quality differences of different platforms or the sensitivity differences of features, resulting in important platforms and high-sensitivity features being forced to bear excessive noise, while the lower noise part may have insufficient privacy protection, ultimately causing a suboptimal trade-off between overall model utility and privacy protection level; second, the existing privacy modeling method usually uses attention mechanism only for feature selection or model internal representation learning, without establishing a closed-loop feedback with privacy budget allocation, and the attention weight and privacy noise intensity are adjusted independently of each other, making it difficult to dynamically match data importance and protection strength. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a user privacy regression modeling method fusing multi-platform data to solve the problem of being unable to distinguish according to the data quality differences of different platforms or the sensitivity differences of features.

[0006] To solve the above technical problems, the present application provides the following technical solutions: The present application provides a user privacy regression modeling method fusing multi-platform data, which comprises, Collecting user behavior data from multiple data sources, pre-processing the user behavior data using a hierarchical attention mechanism, and outputting a dimension-unified feature vector; According to the attention weight generated by the hierarchical attention mechanism, the privacy budget is dynamically allocated by using an exponential decay function, and is balanced across platforms by a global privacy budget pool to obtain a personalized privacy budget allocation scheme. Based on the feature vector and the personalized privacy budget allocation scheme, distributed regression training is performed locally at each data source, noise adaptive to the privacy budget is added to the gradient, and a global regression model is formed through secure aggregation. The global regression model is iteratively optimized, the attention weight and the privacy budget allocation are adjusted in the reverse direction according to the generalization error of local cross-validation of each platform, and a user privacy regression model and an evaluation report are obtained.

[0007] As a preferred scheme of the user privacy regression modeling method for fusing multi-platform data according to the application, the output dimension-unified feature vector is specifically: User behavior data from different data sources are input into the platform-level attention layer respectively, the global importance score of each platform is calculated through a learnable weight vector and is normalized to obtain platform attention weights; All features from each data source are input into the feature-level attention layer, the contribution score of each feature to the regression task is calculated and normalized to obtain feature attention weights; The platform attention weights are used to weight and fuse the feature vectors of each platform, and the feature attention weights are used to weight the features in each data again to obtain preliminary fused features; Batch normalization and median missing value filling are performed on the preliminary fused features to output a feature vector with unified dimensions.

[0008] As a preferred scheme of the user privacy regression modeling method for fusing multi-platform data according to the application, the attention weight generated by the hierarchical attention mechanism is used to dynamically allocate the privacy budget by using an exponential decay function, and is balanced across platforms by a global privacy budget pool to obtain a personalized privacy budget allocation scheme, specifically: The platform attention weights are arranged in the order of data sources, and the feature attention weights in each data source are appended to the back of the platform attention weights in the order of features to splice into a joint attention vector; Each element in the joint attention vector is mapped by applying an exponential decay to obtain a corresponding local privacy budget, and is summarized to a global privacy budget pool; According to the proportion of the total amount of remaining budget in the global privacy budget pool, the local privacy budgets are redistributed and adjusted among the platforms to output a personalized privacy budget allocation scheme.

[0009] As a preferred solution of the user privacy regression modeling method for fusing multi-platform data provided by the application, wherein: the distributed regression training is performed locally at each data source, and noise adaptive to the privacy budget is added to the gradient, specifically: A global regression model with the same structure as the local model of each data source is initialized before the start of distributed regression training, each data source uses a multi-layer neural network and receives the current global regression model parameters as the initial parameters of the local regression model; Each data source performs forward propagation and loss calculation using local user behavior data and feature vectors to obtain local gradients, determines the noise intensity according to the personalized privacy budget corresponding to each data point in the current batch, adds noise to the local gradient, and performs gradient clipping to obtain the clipped and noisy gradient.

[0010] As a preferred solution of the user privacy regression modeling method for fusing multi-platform data provided by the application, wherein: the global regression model is formed by secure aggregation, specifically: The clipped and noisy gradient uploaded by each data source is divided into multiple secret shares in a secret sharing manner and distributed to multiple aggregation nodes; Each aggregation node performs secure summation operation in the encrypted state on the secret shares of the same position held by itself to obtain encrypted aggregated gradient shares; Multiple aggregation nodes cooperate to reconstruct the plaintext average gradient corresponding to the encrypted aggregated gradient shares, and update the current global regression model parameters to generate a new round of global regression model.

[0011] As a preferred solution of the user privacy regression modeling method for fusing multi-platform data provided by the application, wherein: the global regression model is iteratively optimized, and the attention weight and privacy budget allocation are adjusted in the reverse direction according to the generalization error of local cross-validation of each platform to obtain the user privacy regression model, specifically: After each new round of global regression model is issued, each platform calculates the regression mean square error as the generalization error using the local validation set, and the generalization errors of all platforms are summarized as a global error signal; The global error signal is backpropagated to synchronously update the platform attention weight and feature attention weight in the hierarchical attention mechanism, and the decay coefficient of the exponential decay function is also updated; When the global validation loss does not decrease for N consecutive rounds, the training is stopped, and the final converged user privacy regression model is output.

[0012] As a preferred solution of the user privacy regression modeling method for fusing multi-platform data provided by the application, wherein: the evaluation report includes the mean square error of the final converged user privacy regression model on each platform test set, the accuracy gap with the model without privacy protection, the success rate of member inference attack, and the total privacy budget consumed.

[0013] As a preferred embodiment of the user privacy regression modeling method that integrates multi-platform data according to the present invention, the step of adding noise to the local gradient and performing gradient clipping refers to performing L2 norm gradient clipping on the local gradient according to a preset gradient clipping threshold, and adding Gaussian noise of corresponding intensity to the clipped gradient.

[0014] As a preferred embodiment of the user privacy regression modeling method that integrates multi-platform data as described in this invention, the global privacy budget pool redistributes the unused remaining budget to the next round according to the attention weight ratio of the current round after each iteration.

[0015] As a preferred embodiment of the user privacy regression modeling method integrating multi-platform data described in this invention, the step of redistributing and adjusting the local privacy budgets across platforms according to the proportion of the remaining total budget in the global privacy budget pool, and outputting a personalized privacy budget allocation scheme for each data point, specifically includes: The global redistribution coefficient is obtained by calculating the ratio of the current remaining budget in the global privacy budget pool to the sum of all local privacy budgets. The adjusted local privacy budget is obtained by calculating the global reallocation coefficient with each local privacy budget. Update the sum of all adjusted local privacy budgets to the current remaining total budget, and map the adjusted local privacy budgets back to each corresponding data point, outputting a personalized privacy budget allocation scheme.

[0016] The beneficial effects of this invention are as follows: By combining a hierarchical attention mechanism with an exponential decay function to achieve a personalized privacy budget dynamic allocation mechanism, and a global reallocation and cross-round reuse mechanism for the global privacy budget pool, this invention completely overcomes the suboptimal overall utility-privacy trade-off caused by the unified privacy budget allocation strategy commonly used in existing privacy modeling methods. This strategy forces important platforms and highly sensitive features to bear excessive noise while providing insufficient protection for low-sensitivity parts. At the same time, by aggregating the local generalization errors of each platform into a global error signal and backpropagating it to the closed-loop optimization of the attention layer and the exponential decay function, this invention achieves real-time coordinated adjustment of attention weights and privacy noise intensity. This fundamentally solves the shortcomings of existing methods where the attention mechanism is only used for feature representation and is independent of privacy budget allocation, and cannot dynamically match data importance and protection strength. Attached Figure Description

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0018] Fig. 1 Flow chart for user privacy regression modeling method fusing multi-platform data.

[0019] Fig. 2 Flow chart for dynamic allocation of privacy budget.

[0020] Fig. 3 Flow chart for distributed regression training and secure aggregation.

[0021] Fig. 4 Flow chart for iterative optimization. DETAILED DESCRIPTION

[0022] In order to make the above objectives, features and advantages of the present application more apparent, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0023] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0024] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is separate or alternative to other embodiments.

[0025] REFERENCE Figs. 1-4 For one embodiment of the present application, the embodiment provides a user privacy regression modeling method fusing multi-platform data, comprising the following steps: S1: Collecting user behavior data from multiple data sources, pre-processing the user behavior data by using a hierarchical attention mechanism, and outputting a dimension-unified feature vector.

[0026] S1.1: User behavior data from different data sources (e.g. social media platforms, e-commerce platforms, and mobile applications) respectively enter the platform-level attention layer, which maintains a learnable weight vector for each data source, takes the average of all features of the data source to obtain an average feature vector, and performs multiplication operation with the learnable weight vector to obtain a global importance raw score; perform Softmax normalization on the global importance raw scores of all data sources to obtain platform attention weights. User behavior data includes: clickstream features, transaction record features, social interaction features, time series features, and device and environment features.

[0027] All features within each data source enter the feature-level attention layer, which maintains a global query vector within the data source; for each single feature vector (obtained by splitting the original feature vector) in the data source, a contribution raw score is obtained by multiplying it with the global query vector; perform Softmax normalization on the contribution raw scores of all features within the data source to obtain feature attention weights.

[0028] S1.2: Use platform attention weights to perform first weighted fusion on complete raw feature vectors of each data source to obtain a cross-platform fusion feature vector, expressed as: ; wherein, represents the cross-platform fusion feature vector, represents the index of the data source, represents the platform attention weight of the th data source, represents the complete raw feature vector of the th data source, which is the original feature vector of each data source that retains all dimensions completely.

[0029] Within each data source, use feature attention weights to perform second weighting on the original feature vector to obtain a single-platform refined feature vector of the th data source, expressed as: ; wherein, represents the single-platform refined feature vector of the th data source, represents the index of the internal feature of the th data source, represents the feature attention weight of the th feature in the th data source, represents the single-platform refined feature vector of the th data source, An original feature vector refers to an initial feature vector after each data source is collected without any processing.

[0030] The single-platform refined feature vectors of all data sources are spliced to form a complete preliminary fusion feature. Batch standardization is performed on the preliminary fusion feature to make the mean of each dimension 0 and the variance 1, and a standardized feature vector is obtained. For missing value positions in the standardized feature vector, the median value of the current dimension over all samples is used for filling. After filling is completed, a complete and stable multi-platform fusion feature vector with unified dimensions is obtained.

[0031] Preferably, compared with the existing preprocessing method of directly simply splicing or average weighting the multi-platform data, the method realizes differentiated importance perception between platforms and within features through the hierarchical attention mechanism, so that important data sources and key features obtain higher expression weight in the fusion process, avoiding the interference of irrelevant or noise features on the regression task. At the same time, the combination of batch standardization and median filling further improves the stability and completeness of the feature distribution, thereby significantly improving the prediction accuracy and convergence speed of the subsequent privacy protection regression model, and obtaining a better precision-privacy trade-off performance under the same privacy budget.

[0032] S2: According to the attention weight generated by the hierarchical attention mechanism, the privacy budget is dynamically allocated by using an exponential decay function, and is balanced across platforms through a global privacy budget pool to obtain a personalized privacy budget allocation scheme.

[0033] S2.1: After the platform attention weight and the feature attention weight are all collected, they are arranged and spliced in the order of platform attention weight first and feature attention weight within each platform second to form a complete one-dimensional joint attention vector.

[0034] Each element in the joint attention vector is mapped by applying an exponential decay function, so that the greater the attention weight value, the smaller the mapping result, thereby obtaining the local privacy budget corresponding to each element, expressed as: ; wherein, represents the local privacy budget finally obtained by the th element in the joint attention vector, represents the total amount of remaining budget in the global privacy budget pool at the beginning of the current round, represents the base of the natural logarithm, represents the decay control coefficient, represents the original attention weight value (range 0 to 1) of the th element in the joint attention vector.

[0035] It should be noted that the determination process of the decay control coefficient: before the start of each round of training, a target compression ratio is set, indicating the proportion of the local privacy budget obtained by the element with the highest attention weight (i.e., the element with an attention weight of 1) in the total budget of the current round; the decay control coefficient is equal to the negative value of the natural logarithm of the target compression ratio. After setting, when the attention weight of a certain element is exactly 1, its local privacy budget is exactly equal to the value obtained by multiplying the total budget of the current round by the target compression ratio, achieving the expected budget compression goal of the highest attention element.

[0036] S2.2: After all local privacy budgets are calculated, they are directly aggregated into the global privacy budget pool to form the local privacy budget set of the current round.

[0037] The ratio of the total amount of remaining budget actually available in the global privacy budget pool to the sum of all local privacy budgets just aggregated is the global redistribution coefficient; the current global redistribution coefficient is multiplied by each local privacy budget to obtain the corresponding adjusted local privacy budget. After this multiplication adjustment, the sum of all local privacy budgets is precisely scaled to the total amount of remaining budget of the current round, neither exceeding the budget nor leaving a gap; the adjusted local privacy budget maintains the same relative size relationship as the original local privacy budget, but the whole is uniformly scaled up or down, thereby ensuring that the differentiated protection strength brought by the attention weight is maximally preserved while strictly complying with the upper limit of the total amount of remaining budget of the current round.

[0038] Each adjusted local privacy budget is mapped back to the corresponding platform or specific feature in the original position order in the joint attention vector, and then further mapped to the specific data point that owns the current feature, forming an individualized privacy budget allocation scheme for each data point.

[0039] It should be noted that after the current round of distributed regression training is completed, the part of the budget that is actually not consumed in the global privacy budget pool is retained in the global privacy budget pool. When the next round of iteration begins, this part of the remaining budget is directly supplemented into the corresponding local privacy budget according to the proportion of the platform attention weight and the feature attention weight calculated in the next round, thereby realizing the dynamic reuse and cross-platform balancing of the total privacy budget in the entire multi-round training process.

[0040] Preferably, compared to existing methods that uniformly allocate privacy budgets across all platforms and features or simply distribute them equally across platforms, this method achieves fine-grained dynamic privacy budget allocation through an exponential decay function combined with attention weights. This allows important platforms and sensitive features to automatically receive smaller privacy budgets (stronger protection), while less important parts retain more budget to maintain model utility. Combined with global redistribution of the global privacy budget pool and reuse of remaining budgets across rounds, this ensures that the total budget is strictly controlled while maximizing the utilization rate of the budget in each round. Ultimately, under the same total privacy budget constraint, this significantly improves the accuracy of the regression model, optimizes the accuracy-privacy trade-off, and enhances the fairness of data contributions and training stability across multiple platforms.

[0041] S3: Based on feature vectors and a personalized privacy budget allocation scheme, distributed regression training is performed locally on each data source. After adding noise that is adaptive to the privacy budget to the gradient, a global regression model is formed through secure aggregation.

[0042] Before the distributed regression training begins, a global regression model is initialized. The global regression model adopts the same multi-layer neural network structure as the local data source, and the initialized global regression model parameters are broadcast to each data source as the current global regression model parameters for the first round.

[0043] At the start of each training round, each data source receives the current global regression model parameters and copies them completely into the local multilayer neural network as the initial parameters of the local regression model.

[0044] Each data source uses locally held user behavior data and a uniformly dimensional feature vector obtained from S1 to perform forward propagation on the local regression model, calculating the regression loss function (mean squared error). Then, backpropagation is used to obtain the original local gradients of all samples in the current batch. The formula for calculating the regression loss function is: ; in, This represents the regression loss value calculated for the current batch using this data source. This indicates the number of samples included in the current batch. This indicates the sequence number of the sample in the current batch, from 1 to... , This indicates that the local regression model applies to the current batch. The predicted output value for each sample. Indicates the current batch number The true target value of each sample.

[0045] S3.2: Perform gradient clipping on the original local gradient first: set a uniform gradient clipping threshold, if the L2 norm of the gradient of a certain sample exceeds the gradient clipping threshold, then scale the current sample gradient to the L2 norm that is just equal to the gradient clipping threshold.

[0046] It should be noted that after the calculation of the original local gradient of all samples in the current batch is completed, the L2 norms of all sample gradients are collected, and the norm corresponding to the 95th percentile is selected as the gradient clipping threshold of the current batch. This setting can clip extreme large gradients and retain most normal gradient information without manual parameter tuning.

[0047] For each sample's clipped gradient, the noise intensity is determined according to the personalized privacy budget obtained by the current sample in step S2.2, and Gaussian noise with a mean of 0 is added to the clipped gradient. The standard deviation of the noise is inversely proportional to the personalized privacy budget and proportional to the gradient clipping threshold, so as to obtain the final clipped and noisy gradient.

[0048] S3.3: Each data source uploads the clipped and noisy gradient to the secure aggregation server, and the secure aggregation server receives the clipped and noisy gradient uploaded by all data sources. After receiving the clipped and noisy gradient uploaded by all data sources, the secure aggregation server performs secret sharing operation on the gradient of each data source: for the kth data source's clipped and noisy gradient vector, the secure aggregation server selects a secret sharing threshold scheme (for example, there are N aggregation nodes in total, and at least t nodes are needed for cooperation to recover), and uses additive secret sharing to divide each dimension value of the noisy gradient vector into N secret shares, so that any combination of less than t secret shares cannot recover the original value, but any t or more secret shares can be completely reconstructed. The secure aggregation server sends the N secret shares belonging to the same data source to the N pre-deployed aggregation nodes respectively, ensuring that each aggregation node only receives one secret share in the same position from all data sources, thereby realizing the secure distribution of all clipped and noisy gradients.

[0049] Each aggregation node performs element-wise addition operation on the same dimension secret share it holds in an encrypted state to obtain the encrypted aggregated gradient share of the current dimension.

[0050] The obtained encrypted aggregated gradient shares are sent to a secure aggregation server, and after the secure aggregation server receives all the encrypted aggregated gradient shares uploaded by the aggregation nodes, an addition operation is performed on all the shares of the same dimension to obtain the gradient sum of the current dimension. Since the gradient of each data source has been divided into multiple secret shares and evenly distributed to the aggregation nodes, the addition operation of all the aggregation node shares is exactly equal to the sum of the gradients of all the data sources. The secure aggregation server divides the gradient sum by the total number of data sources participating in the current round of training to obtain the plaintext average gradient of the current dimension. The addition operation and averaging operation are repeated for each dimension of the gradient vector to completely reconstruct the plaintext average gradient of all dimensions. After the reconstruction is completed, the secure aggregation server performs a standard gradient descent update on the current global regression model parameters using the plaintext average gradient to generate a new round of global regression model parameters and broadcast them to each data source.

[0051] Preferably, compared with the existing method of adopting uniform noise intensity in differential privacy federated regression or only dividing the privacy budget equally according to the platform, the method introduces adaptive Gaussian noise based on individualized privacy budget in distributed training, so that the sensitive features and important platform contributed data points automatically obtain stronger privacy protection, while the non-sensitive part retains higher model utility. Combined with adaptive gradient clipping and secure aggregation of additive secret sharing, the amplification of noise by extreme gradient is effectively controlled, and the risk of gradient leakage is avoided. Finally, under the same total privacy budget constraint, the prediction accuracy and training stability of the global regression model are significantly improved, while a more fine-grained privacy-accuracy trade-off and cross-platform fairness are achieved.

[0052] S4: Iterative optimization of the global regression model, adjustment of attention weights and privacy budget allocation according to the generalization error of each platform local cross-validation, to obtain a user privacy regression model and an evaluation report.

[0053] S4.1: After each new round of global regression model parameters is broadcast to each data source, each data source pauses gradient uploading and instead performs forward inference on the current new round of global regression model using the locally independently held validation set (which has never participated in gradient calculation throughout the training process), calculates the mean square error of all validation samples, and takes this time's mean square error as the generalization error of the data source.

[0054] Each data source uploads the generalization error calculated by itself to the secure aggregation server, and the secure aggregation server performs a simple average on the generalization errors of all data sources to obtain the global error signal of the current round.

[0055] The global error signal continues to be back-propagated in a differentiable manner: flowing through the backbone multi-layer neural network of the current global regression model (keeping parameters frozen, only for gradient transmission), then up to the platform-level attention layer and feature-level attention layer of the hierarchical attention mechanism, so that the learnable parameters of the platform attention weights and feature attention weights (i.e. the aforementioned learnable weight vectors and global query vectors) obtain update gradients according to the global error signal and perform parameter updates; at the same time, the global error signal continues to flow to the decay coefficient in the exponential decay function, so that the decay coefficient also obtains update gradients according to the global error signal and performs parameter adjustment, thereby synchronously optimizing the attention weights and the privacy budget allocation strategy.

[0056] S4.2: During the training process, the global error signal value of each round is continuously recorded. When the global error signal of the last three consecutive rounds does not decrease, the early stopping mechanism is triggered immediately, and all iterative training is stopped. Using three rounds instead of one or two rounds can effectively avoid early termination of training caused by random noise or single fluctuation, and compared with a longer round of three rounds, the training can be stopped more timely before the model starts to significantly overfit.

[0057] After the training is stopped, the global regression model parameters of the current round are output as the final converged user privacy regression model, and the mean square error of the final user privacy regression model on each local test set of the data source, the difference between the mean square error of the benchmark model without any privacy protection measures and the same test set, the success rate of the member inference attack simulated by the shadow model training, and the total privacy budget actually consumed during the entire training process are calculated and recorded to form a complete evaluation report and output.

[0058] Preferably, compared with the existing method of fixed attention weights and privacy budget or only adjusted by fixed hyperparameters in the federal regression method, the present method automatically and continuously optimizes the platform attention weights, feature attention weights and privacy budget allocation strategy according to the model generalization performance by back-propagating the global error signal to the hierarchical attention mechanism and the exponential decay function, realizing the dynamic self-adaptive matching of attention and privacy protection strength; combined with the early stopping mechanism based on the local validation set, the budget waste and overfitting caused by invalid iteration are avoided, so that better generalization ability is obtained under the same total privacy budget, and the evaluation report comprehensively quantifies the precision gap and actual privacy consumption, improving the explainability of the model and the reliability of cross-platform deployment.

[0059] The embodiment also provides a computer device suitable for the user privacy regression modeling method fusing multi-platform data, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the user privacy regression modeling method fusing multi-platform data proposed in the above embodiment.

[0060] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0061] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the user privacy regression modeling method for fusing multi-platform data as described in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0062] In summary, the application: through the individualized privacy budget dynamic allocation mechanism combined with the hierarchical attention mechanism and the exponential decay function, and the global reallocation and cross-round reuse mechanism of the global privacy budget pool, the important platform and high sensitive features are forced to bear excessive noise, and the overall utility-privacy suboptimal trade-off problem caused by the insufficient protection of low sensitive part is completely overcome; at the same time, by collecting the local generalization error of each platform into a global error signal and back propagating it to the closed loop optimization of the attention layer and the exponential decay function, the real-time collaborative adjustment of the attention weight and the privacy noise intensity is realized, and the defect that the attention mechanism in the existing method is only used for feature representation and is independent of the privacy budget allocation, and cannot dynamically match the data importance and protection strength is fundamentally solved.

[0063] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for user privacy regression modeling of fused multi-platform data, the method comprising: The application relates to a user privacy regression method and device based on federated learning. ​ User behavior data is collected from multiple data sources, preprocessed by a hierarchical attention mechanism, and a dimension-unified feature vector is outputted; According to the attention weight generated by the hierarchical attention mechanism, the privacy budget is dynamically allocated by using an exponential decay function, and cross-platform balance is performed through a global privacy budget pool to obtain a personalized privacy budget allocation scheme; Based on the feature vector and the personalized privacy budget allocation scheme, distributed regression training is performed locally at each data source, noise adaptive to the privacy budget is added to the gradient, and a global regression model is formed through secure aggregation. The global regression model is iteratively optimized, the attention weight and the privacy budget allocation are adjusted in the reverse direction according to the generalization error of local cross-validation of each platform, and a user privacy regression model and an evaluation report are obtained. 2.The method of claim 1, wherein: The output dimension-unified feature vector is specifically as follows: The user behavior data from different data sources is inputted into a platform-level attention layer, the global importance score of each platform is calculated through a learnable weight vector and normalized to obtain platform attention weights; All features from each data source are inputted into a feature-level attention layer, the contribution score of each feature to the regression task is calculated and normalized to obtain feature attention weights; The platform attention weights are used for weighted fusion of the feature vectors of each platform, and the feature attention weights are used for secondary weighting of the features in each data source to obtain preliminary fusion features; Batch standardization and median missing value filling are performed on the preliminary fusion features to output the dimension-unified feature vector. 3.The method of claim 2, wherein: According to the attention weight generated by the hierarchical attention mechanism, the privacy budget is dynamically allocated by using an exponential decay function, and cross-platform balance is performed through a global privacy budget pool to obtain a personalized privacy budget allocation scheme, specifically as follows: The platform attention weights are arranged in the order of data sources, and the feature attention weights in each data source are appended behind the platform attention weights in the order of features to splice into a joint attention vector; Exponential decay is applied to each element in the joint attention vector to obtain the corresponding local privacy budget, and the local privacy budgets are summarized into a global privacy budget pool; According to the proportion of the total amount of remaining budgets in the global privacy budget pool, the local privacy budgets are redistributed and adjusted among the platforms to output the personalized privacy budget allocation scheme. 4.The method of claim 3, wherein: In the distributed regression training, noise adaptive to the privacy budget is added to the gradient, specifically as follows: Before the distributed regression training starts, a global regression model with the same local model structure as each data source is initialized, and each data source uses a multilayer neural network and receives the current global regression model parameters as the initial parameters of the local regression model; Each data source performs forward propagation and loss calculation on the local user behavior data and the feature vector to obtain a local gradient, determines the noise intensity according to the personalized privacy budget corresponding to each data point in the current batch, adds noise to the local gradient and performs gradient clipping to obtain the clipped and noisy gradient.

5. The method of claim 4, wherein: The global regression model is formed through secure aggregation, specifically as follows: The gradient uploaded by each data source is divided into multiple secret shares by secret sharing, and is distributed to multiple aggregation nodes; Each aggregation node performs secure summation operation on the secret share of the same position held by itself in an encrypted state to obtain an encrypted aggregated gradient share; The multiple aggregation nodes cooperate to reconstruct the plaintext average gradient corresponding to the encrypted aggregated gradient share, and update the current global regression model parameter to generate a new round of global regression model.

6. The user privacy regression modeling method of fusing multi-platform data according to claim 5, wherein: The global regression model is iteratively optimized, and the attention weight and privacy budget allocation are adjusted in the reverse direction according to the generalization error of local cross-validation of each platform to obtain a user privacy regression model, specifically: After each new round of global regression model is issued, each platform calculates the regression mean square error as the generalization error using the local validation set, and the generalization errors of all platforms are summarized as a global error signal; The global error signal is used for back propagation to synchronously update the platform attention weight and feature attention weight in the hierarchical attention mechanism, and to update the decay coefficient of the exponential decay function; When the global validation loss does not decrease for N consecutive rounds, the training is stopped, and the final converged user privacy regression model is output.

7. The user privacy regression modeling method of fusing multi-platform data according to claim 6, wherein: The evaluation report includes the mean square error of the final converged user privacy regression model on each platform test set, the accuracy gap with the non-privacy protection model, the success rate of member inference attack, and the total privacy budget consumed.

8. The method of claim 4, wherein: The method comprises adding noise to the local gradient and performing gradient clipping, which refers to performing L2 norm gradient clipping on the local gradient according to a preset gradient clipping threshold, and adding Gaussian noise with a corresponding intensity to the clipped gradient.

9. The user privacy regression modeling method of fusing multi-platform data according to claim 8, wherein: The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration.

10. The method of claim 3, wherein: The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration. The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration. The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration. The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration. The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration. The global privacy budget pool re-distributes the remaining budget to the next round according to the proportion of the attention weight of the current round after each iteration.

Images