Sensitive attribute forgetting method and device for recommendation system

Abstract

The invention provides a sensitive attribute forgetting method and device for a recommendation system. An embedding layer of the recommendation system comprises a user embedding matrix and an article embedding matrix; the method comprises the following steps: performing matrix decomposition on an article embedding matrix to obtain an independent space orthogonal basis; performing feature decoupling on the user representation vector to obtain a correlation vector and an irrelevant vector; the irrelevant vector is represented as a linear combination of a to-be-optimized parameter and an irrelevant space orthogonal basis; constructing user representation distribution and sensitive attribute distribution, and selecting the optimal bandwidth of the HSIC core; constructing a forgetting loss function used for measuring the dependency between the user representation distribution and the sensitive attribute distribution, and replacing a user representation vector in the forgetting loss function with a combination of a correlation vector and an irrelevant vector to obtain a target function; and under the condition of keeping the correlation vector unchanged, taking a minimization objective function as an objective, obtaining a final user representation vector through iterative optimization, and taking the final user representation vector as a result after the sensitive attribute is forgotten.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and specifically to a method and apparatus for forgetting sensitive attributes in recommendation systems. Background Technology

[0002] With the rapid development of internet and mobile internet technologies, personalized recommendation systems have been widely applied to various online service platforms, such as e-commerce platforms and short video platforms. Recommendation systems collect users' historical behavioral data, including clicks, browsing, favorites, likes, purchases, and ratings, to build user profiles and model users' potential interests, thereby enabling personalized content recommendations. However, as the deployment scale of recommendation systems continues to expand, their potential privacy risks are gradually emerging. Because recommendation systems possess powerful user behavior modeling capabilities, while learning user preferences, they often implicitly encode sensitive attribute information related to gender, age, and race. Once this attribute information is illegally obtained or misused by third parties, it may lead to the leakage of users' personal identity information, thereby causing serious privacy and security problems.

[0003] Currently, the technologies related to the need for forgetting sensitive attributes in recommender systems mainly fall into two categories:

[0004] The first category is the forgetting-in-training method, which introduces adversarial constraints or privacy protection mechanisms during model training to reduce the model's inferability of sensitive attributes. Representative methods include adversarial training, privacy-aware regularization, and perturbing the training data to weaken the representation encoding of sensitive attributes. This type of method can explicitly constrain the model's dependence on sensitive attributes during learning, but it usually requires redesigning and executing the entire training process, which may lead to problems such as training instability, convergence difficulties, and a significant increase in computational cost. Therefore, its application is limited in large-scale deployments or recommendation systems that require frequent updates.

[0005] The second category is post-training attribute forgetting methods. These methods, without requiring complete retraining, reduce the model's inference ability for specific attributes to an unusable level through post-processing mechanisms or parameter fine-tuning. Typical solutions include introducing a dual-objective loss function and designing pluggable fine-tuning modules (such as sub-network structures based on information theory constraints). These methods have good compatibility with deployed models, requiring only limited intervention based on existing parameters. However, this post-training forgetting method still faces two key bottlenecks: First, it lacks an effective performance maintenance mechanism; the forgetting intervention and recommendation accuracy have not been effectively decoupled, and excessive reduction in accuracy in real-world applications will directly affect system effectiveness. Second, the underlying representation support is insufficient. Related methods deeply reveal the entanglement mechanism between user representation vectors and sensitive attributes, resulting in weak universality and adaptability of the forgetting strategy, making it difficult to flexibly handle diverse attribute forgetting requests in real-world scenarios.

[0006] In summary, existing solutions all have limitations in addressing the attribute forgetting problem in recommender systems. Therefore, there is an urgent need for an efficient, low-overhead, and performance-preserving sensitive attribute forgetting method that can both stably protect the performance of recommender systems and flexibly adapt to various sensitive attribute forgetting requirements. Summary of the Invention

[0007] In view of this, this application proposes a method and apparatus for forgetting sensitive attributes in recommendation systems.

[0008] Specifically, this application is implemented through the following technical solution:

[0009] According to a first aspect of the embodiments of this specification, a sensitive attribute forgetting method for a recommendation system is provided. The recommendation system includes an embedding layer, the embedding layer including a user embedding matrix and an item embedding matrix, wherein each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector of a different item; the method includes the following steps:

[0010] Step S1: Perform matrix decomposition on the item embedding matrix to obtain a relevant orthogonal basis and an irrelevant orthogonal basis; and perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold; the difference between the dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold;

[0011] Step S2: Construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attributes, respectively, and maximize the test energy efficiency by optimizing the bandwidth of the Hilbert-Schmidt independence criterion (HSIC kernel) to obtain the optimal bandwidth;

[0012] Step S3: Construct a forgetting loss function using the HSIC kernel under the optimal bandwidth to measure the dependency between the user representation distribution and the sensitive attribute distribution, and replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain the objective function for the parameter to be optimized;

[0013] Step S4: While keeping the relevant vector unchanged, optimize the parameters to be optimized to update the irrelevant vector with the goal of minimizing the objective function, and combine it with the relevant vector to obtain a new user representation vector;

[0014] Step S5: Reconstruct the user representation distribution based on the new user representation vector, and repeat steps S3 to S4 until the preset iteration stopping condition is met. The final user representation vector is then used as the result after forgetting the sensitive attributes.

[0015] According to a second aspect of the embodiments of this specification, a sensitive attribute forgetting device for a recommendation system is provided. The recommendation system includes an embedding layer, the embedding layer including a user embedding matrix and an item embedding matrix, wherein each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector of a different item; the device includes:

[0016] The first processing unit is configured to perform matrix decomposition on the item embedding matrix to obtain a relevant orthogonal basis and an irrelevant orthogonal basis; and to perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold; the difference between the dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold.

[0017] The second processing unit is used to construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attribute, respectively, and to obtain the optimal bandwidth by optimizing the bandwidth of the Hilbert-Schmidt independence criterion HSIC kernel to maximize the test energy efficiency;

[0018] The third processing unit is used to construct a forgetting loss function using the HSIC kernel under the optimal bandwidth to measure the dependency between the user representation distribution and the sensitive attribute distribution, and to replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain the objective function for the parameter to be optimized.

[0019] The objective optimization unit is used to optimize the parameters to be optimized to update the irrelevant vectors with the objective function as the objective while keeping the relevant vectors unchanged, and to combine them with the relevant vectors to obtain a new user representation vector.

[0020] The iterative processing unit is used to reconstruct the user representation distribution based on the new user representation vector and repeatedly execute the contents of the third processing unit and the target optimization unit until the preset iteration stopping condition is met, and the final user representation vector is used as the result after forgetting the sensitive attributes.

[0021] According to a third aspect of the embodiments of this specification, an electronic device is provided, including a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method described in the first aspect.

[0022] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor of the method described in the first aspect.

[0023] The embodiments of this application achieve at least the following beneficial effects:

[0024] First, it has the effect of low overhead and high compatibility. The embodiments of this application do not require deleting the original user data, nor do they require modifying the model structure of the recommendation system or retraining it completely. This avoids the computational overhead caused by retraining and can be directly applied to the deployed black-box model.

[0025] Second, it achieves effective decoupling between recommendation performance and sensitive attributes. By decoupling the user representation vector, the forgetting operation is limited to irrelevant components that are unrelated to recommendation performance. This not only ensures the stability of performance but also provides a wider space for forgetting optimization, overcoming the shortcomings of related schemes that limit the forgetting effect due to the use of strong regularization constraints.

[0026] Third, it has the effect of strong versatility and flexibility. The embodiments of this application formulate forgetting strategies based on the underlying relationship between user representation and sensitive attributes. Only the user representation distribution and sensitive attribute distribution need to be used as input. It has a high degree of flexibility and versatility and can easily cope with the diverse single-attribute and multi-attribute forgetting needs in real-world environments. Attached Figure Description

[0027] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0028] Figure 1 This is a flowchart illustrating a sensitive attribute forgetting method for a recommendation system, as shown in an exemplary embodiment of this application.

[0029] Figure 2 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0030] Figure 3 This is a block diagram illustrating a sensitive attribute forgetting device for a recommendation system, as shown in an exemplary embodiment of this application. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0033] The embodiments described in this specification will now be described in detail.

[0034] This application provides a sensitive attribute forgetting method for a recommender system. In the recommender system of this application embodiment, the embedding layer includes a user embedding matrix. With item embedding matrix Two parts, For the number of users, For the quantity of items, For the embedding dimension, each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector for different items. Furthermore, the recommendation system in this embodiment also maintains an attribute matrix, where each column vector corresponds to a different attribute representation vector. These attributes include sensitive attributes such as gender, age, and race.

[0035] The final recommendation result of the recommendation system is represented by a user representation vector. with item representation vector The dot product score determines the user's score. For items The degree of preference.

[0036] The sensitive attribute forgetting method in this application does not require modification of the recommendation model's structure and can be directly applied in black-box model scenarios. This sensitive attribute forgetting method is flexibly compatible with recommendation models such as DMF and LightGCN, possessing good versatility and practical application value.

[0037] Figure 1 This is a flowchart illustrating an exemplary embodiment of a sensitive attribute forgetting method for a recommendation system, as shown in this application. Figure 1 As shown, the sensitive attribute forgetting method includes at least the following steps:

[0038] Step S1: Perform matrix decomposition on the item embedding matrix to obtain a relevant orthogonal basis and an irrelevant orthogonal basis; and perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold; the dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold.

[0039] It is understood that in this embodiment, both the first preset threshold and the second preset threshold are 0 or values ​​close to 0. That is, the dot product of the relevant vector and the item embedding matrix is ​​approximately the same as the dot product of the original user representation vector and the item embedding matrix, while the dot product of the irrelevant vector and the item embedding matrix is ​​approximately 0. Thus, the user representation vector can be decomposed into relevant vectors carrying the main recommendation information and irrelevant components located in the irrelevant space. The irrelevant space refers to the vector space where the dot product with each row of the item embedding matrix is ​​zero or less than a preset threshold.

[0040] In this way, during the subsequent forgetting of sensitive attributes, the relevant vectors remain unchanged, and the irrelevant vectors are updated only in the irrelevant space, thereby achieving the masking of specific attribute information while maximizing recommendation performance.

[0041] Step S2: Construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attributes, respectively, and obtain the optimal bandwidth by optimizing the bandwidth of the Hilbert-Schmidt independence criterion HSIC kernel to maximize the test energy efficiency.

[0042] The sensitivity of the HSIC kernel to the dependency relationship between two distributions varies depending on the bandwidth value of the HSIC kernel. This embodiment optimizes the bandwidth of the HSIC kernel to limit it to the optimal bandwidth, so that the HSIC kernel can most sensitively measure the dependency relationship between two distributions.

[0043] For the HSIC core, candidate bandwidths can be traversed using a grid search. List, select the optimal bandwidth This makes the HSIC kernel most sensitive to the dependence measure between the two distributions under the optimal bandwidth, that is, the accuracy as an independence test index is the highest.

[0044] The HSIC framework does not require constructing the density functions of two distributions; it only requires constructing the kernel matrix separately using appropriate kernel functions.

[0045] In some embodiments, a user representation distribution is constructed based on all user representation vectors in the user embedding matrix; a sensitive attribute distribution is constructed based on the target sensitive attributes in the attribute matrix of the recommender system, where the target sensitive attributes can be one or more specified sensitive attributes.

[0046] Step S3: Construct a forgetting loss function using the HSIC kernel under the optimal bandwidth to measure the dependency between the user representation distribution and the sensitive attribute distribution, and replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain the objective function for the parameter to be optimized.

[0047] Step S4: While keeping the relevant vector unchanged, optimize the parameters to be optimized to update the irrelevant vector with the goal of minimizing the objective function, and combine the updated irrelevant vector with the relevant vector to obtain a new user representation vector.

[0048] Step S5: Reconstruct the user representation distribution based on the new user representation vector, and repeat steps S3 to S4 until the preset iteration stopping condition is met. The final user representation vector is then used as the result after forgetting the sensitive attributes.

[0049] like Figure 1 As shown in the sensitive attribute forgetting method, this embodiment does not require deleting the original user data, nor does it require modifying the recommendation system model structure or retraining the entire dataset. This avoids the computational overhead caused by retraining in traditional methods and can be directly applied to deployed black-box models. Furthermore, by decoupling the user representation vector, this embodiment limits the forgetting operation to irrelevant components that are unrelated to recommendation performance. This ensures performance stability and provides a wider space for forgetting optimization, overcoming the shortcomings of traditional methods that limit forgetting effectiveness due to the use of strong regularization constraints. In addition, this embodiment formulates the forgetting strategy based on the underlying relationship between the user representation vector and the sensitive attributes. It only requires the user representation distribution and the sensitive attribute distribution as input, which has high flexibility and versatility. It can easily cope with diverse single-attribute and multi-attribute forgetting needs in real-world environments, such as constructing composite attribute kernel matrices through weighted summation or Hadamard product.

[0050] In some embodiments, step S1 specifically includes:

[0051] Singular value decomposition is performed on the item embedding matrix to obtain a left singular value matrix, a right singular value matrix, and a singular value diagonal matrix; based on the singular value diagonal matrix, the previous... The cumulative energy percentage of each singular value; based on the minimum rank corresponding to the preset energy threshold hyperparameter value, the right singular value matrix is ​​decomposed into a mutually orthogonal correlated space orthogonal basis and an uncorrelated space orthogonal basis; based on the correlated space orthogonal basis and the uncorrelated space orthogonal basis, the user representation vector is decoupled into the mutually orthogonal correlated vector and the uncorrelated vector.

[0052] In some embodiments, the linear combination of the irrelevant vectors in step S1 is:

[0053]

[0054] In formula (1), For user-represented vectors, For the relevant vector, For parameters to be optimized, It is an orthogonal basis for independent spaces.

[0055] In some embodiments, step S2 specifically includes:

[0056] The user representation distribution is obtained by distributing all user representation vectors in the user embedding matrix based on Gaussian kernels; the sensitive attribute distribution is obtained by distributing the attribute set of the recommendation system based on label kernels.

[0057] The HSIC framework eliminates the need to construct density functions for two distributions; it only requires constructing kernel matrices using appropriate kernel functions. In this embodiment, a Gaussian kernel is used to construct the user representation distribution, and a label kernel is used to construct the sensitivity attribute distribution.

[0058] In some embodiments, when it is necessary to forget multiple sensitive attributes, step S2 further includes:

[0059] Construct independent distributions for each sensitive attribute that needs to be forgotten;

[0060] The final distribution of sensitive attributes is obtained by weighting the independent distributions of the multiple sensitive attributes to be forgotten, or by performing a Hadamard product on the independent distributions of the multiple sensitive attributes to be forgotten.

[0061] In some embodiments, the forgetting loss function in step S3 is:

[0062]

[0063] In formula (2), Represent the distribution for the user. For sensitive attribute distribution, user representation distribution Distribution of sensitive attributes The optimal bandwidth corresponding to the HSIC cores between is User representation distribution The optimal bandwidth corresponding to the HSIC core between itself is , Forgetting loss function, This is the balance coefficient.

[0064] Forgetting loss function The first item The core dependency test index, whose value approaches zero, indicates that the two distributions are approximately independent, meaning that effective forgetting of sensitive attributes has been achieved; the second term This is a distribution structure preservation term, its role is to avoid information collapse during the optimization process, guiding the update direction to weaken attribute correlation while preserving the inherent structure of the user representation distribution as much as possible. When optimization makes... When the value is close to zero, it can be determined that the forgetting effect has been achieved.

[0065] Step S2 includes calculating the optimal bandwidth through the following steps. :

[0066] For the user representation distribution of the original user representation vector Distribution of sensitive attributes Calculate the unbiased estimator of HSIC The Regarding bandwidth The function;

[0067] Calculate the Standard deviation estimation The Regarding bandwidth The function;

[0068] By maximizing To obtain the optimal bandwidth .

[0069] User representation distribution Including the user representation vector and the Gaussian kernel, the user representation distribution is calculated under the condition of optimal bandwidth. The user representation vector in the figure is the original user representation vector of the recommender system. That is, when calculating the optimal bandwidth, the user representation distribution... The user representation vector in the system is a known value, which is the original user representation vector of the recommender system. In this case, the user representation distribution... The Gaussian kernel (which is the bandwidth of the HSIC kernel) is an unknown quantity. In subsequent steps S3 and S4, it is necessary to update the user representation vector in the user representation distribution (i.e., to update irrelevant vectors within the user representation vector). Therefore, in the subsequent optimization process, the user representation vector variable is unknown, while the bandwidth of the HSIC kernel is a known value.

[0070] It is worth noting that for optimal bandwidth The calculation process can be referenced from the optimal bandwidth. The calculation process only requires calculating the optimal bandwidth. Distribution of sensitive attributes involved Replace with user-represented distribution That's it; the other calculation steps are similar, for optimal bandwidth. The calculation process will not be described in detail in this embodiment.

[0071] In some embodiments, Figure 1 The sensitive attribute forgetting method shown also includes the following steps:

[0072] Step S6: Update the user representation vector in the recommendation system with the user representation vector after the forgetting sensitive attribute.

[0073] Next, the forgetting process of sensitive attributes in the embodiments of this application will be described in detail.

[0074] First, step S1 is executed to decompose the item embedding matrix and decouple the user representation vector.

[0075] Taking a collaborative filtering-based recommendation system as an example, the recommendation score of this system is usually obtained through user representation vectors. with item representation vector The dot product is calculated. Based on this linear structure, this embodiment performs directional decomposition on the user representation vector.

[0076] First, perform singular value decomposition (SVD) on the item embedding matrix:

[0077]

[0078] In formula (3), It is a left singular matrix. It is a singular value diagonal matrix. , It is a right singular matrix.

[0079] Based on the linearity of the dot product operation By using the right singular matrix Dividing into two orthogonal bases can be considered as the user representation having a decomposable structure under the right singular basis.

[0080] Considering the usual situation If the right singular matrix is ​​calculated directly The rank will be very close to This fails to achieve the purpose of decoupling. Therefore, this embodiment sets an energy threshold hyperparameter. The closer the energy threshold hyperparameter value is to 1, the higher the proportion of energy retained by the relevant vector, and the stronger the ability to maintain recommendation performance.

[0081] Before this embodiment is defined The cumulative energy percentage of each singular value for:

[0082]

[0083] Select the one that satisfies minimum value As a right singular matrix approximate rank Based on this, the right singular matrix is ​​divided. Divided into two orthogonal bases:

[0084]

[0085] In formula (5), For the relevant space orthogonal basis, It is an orthogonal basis for independent spaces. .

[0086] Using the two orthogonal bases mentioned above, any user representation vector can be represented. , Perform orthogonal decomposition:

[0087]

[0088] Thus, each user representation vector It is explicitly decomposed into relevance vectors that carry the main recommendation information. orthogonal basis in an independent space irrelevant vectors in In the subsequent attribute forgetting process, only for... Apply constraints or updates to ensure that it always lies in an orthogonal basis in an independent space. This allows for the shielding of specific attribute information while maintaining recommendation performance to the maximum extent.

[0089] Next, proceed to step S2 to construct the distribution.

[0090] Based on the requirements of the HSIC core, it is necessary to separately process the user embedding matrix. and attribute matrix The corresponding distribution is constructed. In this embodiment, a distribution in the form of a kernel matrix is ​​constructed using a kernel function.

[0091] This embodiment uses a Gaussian kernel to construct the user representation distribution and a label kernel to construct the sensitive attribute distribution, as follows:

[0092]

[0093] In formula (7), For a given user embedding matrix Chinese users The corresponding user representation vector, For a given user embedding matrix Chinese users The corresponding user representation vector, For a given attribute matrix Chinese users The corresponding sensitive attributes are represented as follows: Given attribute matrix Chinese users The corresponding sensitive attribute is represented; for example, when the sensitive attribute is gender, The value can be either male or female, for example Indicates user Male Indicates user For women.

[0094] Subsequently, based on and User representation distribution can be obtained Distribution of sensitive attributes By centering the kernel matrix, a dediagonalized kernel matrix is ​​obtained. , .

[0095] In some embodiments, when there is a need for multi-attribute forgetting, it can be achieved by using a linear weighted combination of attribute kernel matrices as shown in formula (8), or by using a Hadamard product combination as shown in formula (9):

[0096]

[0097]

[0098] In formula (8), This represents the final distribution of sensitive attributes. and These are weighting coefficients. For the independent distribution corresponding to the attribute "gender", This represents the independent distribution corresponding to the attribute age.

[0099] Then proceed to step S3, which involves selecting the optimal bandwidth.

[0100] To maximize the testing efficiency of the HSIC kernel, this embodiment utilizes the independence test theorem in statistics, assuming the null hypothesis... Assuming the two distributions are independent, the alternative hypothesis is... The two distributions are not independent. The energy efficiency of the test is defined as the probability of correctly rejecting the null hypothesis that the two distributions are independent. For a given number of users... Unbiased estimation of HSIC Given the null hypothesis that "the two distributions are independent", the central limit theorem holds:

[0101] In formula (10), This represents the variance of the standard distribution. This indicates convergence according to the distribution.

[0102] Therefore, the probability of correctly rejecting the null hypothesis can be regarded as a proxy indicator for testing energy efficiency:

[0103]

[0104] In formula (11), This represents the probability of correctly rejecting the null hypothesis. Indicates the rejection threshold. This represents the standard normal cumulative distribution function.

[0105] When the number of users When sufficient, rejection threshold It will converge to a constant, and the rejection probability is mainly determined by... Dominant. Therefore, maximizing this ratio is equivalent to increasing the sensitivity of the dependency measure to the correlation of the distribution, i.e., finding the optimal kernel function to maximize .

[0106] The unbiased estimate of HSIC has the following closed form:

[0107]

[0108] In formula (12), Let represent a vector of all ones. The computational complexity of this estimate is O(n log n). .

[0109] To maximize The goal also requires the difference between the other party and the target. Make an effective estimate.

[0110] Based on the relevant technical solutions, it can be deduced that... The unbiased estimate can be constructed as follows:

[0111]

[0112] In formula (13), For use in calculation The constructed intermediate statistics, , For Pochhammer notation, It is used for calculation The basic vector, The expression is as follows:

[0113]

[0114] In formula (14), the operator Represents element-wise multiplication of a matrix.

[0115] In this way, the maximum can be achieved. Convert to maximization ,because and All of these are about bandwidth. The functions are complex, so in this embodiment, grid search for the optimal bandwidth is equivalent to traversing the given list to find the optimal bandwidth. That is, distribution With distribution A kernel function with optimal sensitivity.

[0116] Among them, regarding optimal bandwidth The calculation process can be found by referring to... The specific details of this embodiment will not be elaborated upon here.

[0117] Then, step S4 is executed to construct the optimization objective function.

[0118] The above steps have yielded the relevant vectors and irrelevant components for each user at the performance protection level, as well as the HSIC kernel at the forgetting level that is most sensitive to the current data distribution. This step combines the aforementioned decomposition and bandwidth optimization results to construct an optimization objective function to achieve accurate forgetting of sensitive attributes while protecting performance.

[0119] The optimization objective is set to minimize the HSIC dependency metric between the user representation distribution and the sensitive attribute distribution. When this metric is close to zero, it indicates that the two are approximately independent, i.e., complete forgetting is achieved. Thus, the user representation distribution and sensitive attribute distribution constructed in step S2 are input into the HSIC kernel under the optimal bandwidth selected in step S3 to obtain the forgetting loss function as shown in formula (2).

[0120] As shown in Equation (1) above, since the irrelevant vector can be represented as a linear combination of the parameter to be optimized and the orthogonal basis of the irrelevant space, the forgetting loss function shown in Equation (2) can also be represented as the objective function as follows:

[0121]

[0122] In formula (15), ,

[0123] The specific calculation process can be found in step S3, which will not be elaborated upon in this embodiment. It is a very small scalar used to prevent variance. (or The case where there is zero or disordered noise is used to ensure that the overall value is always positive. The parameters to be optimized are the vectors corresponding to all user representations. The resulting matrix.

[0124] Finally, step S5 is executed to update the user representation vector.

[0125] In each iteration, the optimized parameters for the current round are obtained by minimizing the objective function value. In the formula For the first The optimized parameters for each user in the current iteration round, based on This yields the irrelevant vector updated in the current round. Based on Obtain the user representation vector updated in the current round. .

[0126] Based on the updated user representation vector The user representation distribution is reconstructed, and the aforementioned optimization process is repeated until the preset iteration stopping condition is met. The final user representation vector is used as the result after forgetting the sensitive attribute, and the user representation vector in the recommendation system is updated to the user representation vector after forgetting the sensitive attribute.

[0127] The preset iteration stopping conditions here include at least one of the following conditions:

[0128] Preset iteration rounds;

[0129] Updated parameters Or the updated irrelevant vector converges.

[0130] To verify the effectiveness of the sensitive attribute forgetting method in this embodiment, experiments were conducted on the classic MovieLens-100K implicit feedback dataset (denoted as M1-100k). Using the LightGCN model as the base model, the effects of different attribute forgetting methods were verified and compared. The method of this embodiment (denoted as Decompose HSIC) was compared with related methods such as the original model, retraining, adversarial training (Adv), and distribution alignment (D2d), and evaluated from two dimensions: recommendation performance preservation and attribute forgetting effect.

[0131] The performance retention evaluation of the recommendation task uses the Hit Ratio (HR@K) and the Normalized Discounted Cumulative Gain (NDCG@K) as evaluation metrics, and the results are shown in Table 1.

[0132]

[0133] Table 1

[0134] As shown in Table 1, the experimental data demonstrates that the method in this embodiment (Decompose HSIC) closely matches the original model in all metrics. For example, in the HR@10 metric, the method in this embodiment achieves 0.6331, only slightly lower than the original's 0.6384, and superior to Retrain (0.6267) and D2d (0.6246). In the NDCG@10 metric, the method in this embodiment achieves 0.3655, also superior to Retrain (0.3608) and Adv (0.3587), while the difference from the original (0.3690) remains within an acceptable range. This indicates that by strictly limiting the forgetting operation to irrelevant components, it is possible to freeze and protect the core performance of the recommendation system, with performance degradation significantly less than or equal to that of the comparative methods.

[0135] The evaluation of attribute forgetting effectiveness uses two classification models, Multilayer Perceptron (MLP) and XGBoost, as attackers, and employs accuracy, precision, recall, and AUC as evaluation metrics. The results are shown in Tables 2 and 3, respectively. The lower the attacker's classification performance, the better the attribute forgetting effect of the model.

[0136]

[0137] Table 2

[0138]

[0139] Table 3

[0140] Combining the experimental results shown in Tables 2 and 3, it can be seen that the method in this embodiment (Decompose HSIC) achieved the best forgetting effect under both attack models. Taking MLP attackers as an example, the accuracy (0.4035), precision (0.3958), recall (0.3796), and AUC (0.3983) of this embodiment are significantly lower than other comparative methods. In particular, compared with the second-best performing D2d method (Accuracy 0.4080), this embodiment method achieves further suppression in all indicators, indicating that it can more thoroughly remove sensitive attribute information from user representations. On XGBoost attackers, this embodiment method also performs best, with its AUC value (0.4497) being much lower than the original model's 0.6587, demonstrating its excellent privacy protection capabilities.

[0141] In summary, the method in this embodiment can effectively reduce the attacker's ability to infer sensitive attributes from user representations while maintaining the original performance of the recommendation system. It achieves effective decoupling between performance preservation and attribute forgetting, verifying the advanced nature and practicality of its technical solution.

[0142] Figure 2 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 2At the hardware level, the device includes a processor 202, an internal bus 204, a network interface 206, memory 208, a hardware acceleration device 210, and non-volatile memory 212, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 202 reads the corresponding computer program from the non-volatile memory 212 into memory 208 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0143] Figure 3 This is a block diagram illustrating a sensitive attribute forgetting device for a recommendation system according to an exemplary embodiment of this application. The sensitive attribute forgetting device can be applied to, for example... Figure 2 The illustrated electronic device implements the technical solution of this application. The recommendation system includes an embedding layer, which comprises a user embedding matrix and an item embedding matrix. Each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector for a different item.

[0144] like Figure 3 As shown, the sensitive attribute forgetting device includes: a first processing unit 310, a second processing unit 320, a third processing unit 330, a target optimization unit 340, and an iterative processing unit 350, wherein:

[0145] The first processing unit 310 is configured to perform matrix decomposition on the item embedding matrix to obtain a relevant space orthogonal basis and an irrelevant space orthogonal basis; and to perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant space orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold; the dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold.

[0146] The second processing unit 320 is used to construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attribute, respectively, and to maximize the test energy efficiency by optimizing the bandwidth of the Hilbert-Schmidt independence criterion HSIC kernel to obtain the optimal bandwidth;

[0147] The third processing unit 330 is used to construct a forgetting loss function that measures the dependency between the user representation distribution and the sensitive attribute distribution using the HSIC kernel under the optimal bandwidth, and replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain an objective function for the parameter to be optimized.

[0148] The target optimization unit 340 is used to optimize the parameters to be optimized to update the irrelevant vectors with the objective function as the objective while keeping the relevant vectors unchanged, and to combine them with the relevant vectors to obtain a new user representation vector;

[0149] The iterative processing unit 350 is used to reconstruct the user representation distribution based on the new user representation vector and repeatedly execute the contents of the third processing unit and the target optimization unit until the preset iteration stopping condition is met, and the final user representation vector is used as the result after forgetting the sensitive attributes.

[0150] In some embodiments, the first processing unit 310 is configured to perform singular value decomposition on the item embedding matrix to obtain a left singular value matrix, a right singular value matrix, and a singular value diagonal matrix; and to calculate the previous... The cumulative energy percentage of each singular value; based on the minimum rank corresponding to the preset energy threshold hyperparameter value, the right singular value matrix is ​​decomposed into a mutually orthogonal correlated space orthogonal basis and an uncorrelated space orthogonal basis; based on the correlated space orthogonal basis and the uncorrelated space orthogonal basis, the user representation vector is decoupled into the mutually orthogonal correlated vector and the uncorrelated vector.

[0151] In some embodiments, the second processing unit 320 is used to perform distribution processing on all user representation vectors in the user embedding matrix based on Gaussian kernels to obtain user representation distribution; and to perform distribution processing on the attribute set of the recommendation system based on tag kernels to obtain sensitive attribute distribution.

[0152] In some embodiments, the second processing unit 320 is configured to process the user representation distribution with respect to the original user representation vector. Distribution of sensitive attributes Calculate the unbiased estimator of HSIC The Regarding bandwidth The function; calculate the Standard deviation estimation The Regarding bandwidth The function; by maximizing To obtain the optimal bandwidth .

[0153] In some embodiments, the sensitive attribute forgetting device further includes:

[0154] The system update unit is used to update the user representation vector in the recommendation system with the user representation vector after the forgetting sensitive attribute.

[0155] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0156] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0157] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0158] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0159] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0160] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0161] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0162] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0163] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0164] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A sensitive attribute forgetting method for recommendation systems, characterized in that, The embedding layer of the recommendation system includes a user embedding matrix and an item embedding matrix. Each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector for a different item. The method includes the following steps: Step S1: Perform matrix decomposition on the item embedding matrix to obtain a relevant space orthogonal basis and an irrelevant space orthogonal basis; and perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant space orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold. The dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold; Step S2: Construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attributes, respectively, and maximize the test energy efficiency by optimizing the bandwidth of the Hilbert-Schmidt independence criterion (HSIC kernel) to obtain the optimal bandwidth; Step S3: Construct a forgetting loss function using the HSIC kernel under the optimal bandwidth to measure the dependency between the user representation distribution and the sensitive attribute distribution, and replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain the objective function for the parameter to be optimized; Step S4: While keeping the relevant vector unchanged, optimize the parameters to be optimized to update the irrelevant vector with the goal of minimizing the objective function, and combine it with the relevant vector to obtain a new user representation vector; Step S5: Reconstruct the user representation distribution based on the new user representation vector, and repeat steps S3 to S4 until the preset iteration stopping condition is met. The final user representation vector is then used as the result after forgetting the sensitive attributes.

2. The method according to claim 1, characterized in that, Step S1 includes: Singular value decomposition is performed on the item embedding matrix to obtain a left singular value matrix, a right singular value matrix, and a singular value diagonal matrix. Based on the singular value diagonal matrix, calculate the first The cumulative energy percentage of each singular value; Based on the minimum rank corresponding to the energy threshold hyperparameter value that makes the cumulative energy proportion not less than the preset energy threshold, the right singular value matrix is ​​decomposed into mutually orthogonal correlated space orthogonal basis and unrelated space orthogonal basis. Based on the orthogonal basis of the relevant space and the orthogonal basis of the irrelevant space, the user representation vector is decoupled into the mutually orthogonal relevant vector and irrelevant vector.

3. The method according to claim 1, characterized in that, The linear combination of the irrelevant vectors in step S1 is: ； in, For user-represented vectors, For the relevant vector, For parameters to be optimized, It is an orthogonal basis for independent spaces.

4. The method according to claim 1, characterized in that, Step S2 includes: Based on the Gaussian kernel, the distribution of all user representation vectors in the user embedding matrix is ​​constructed to obtain the user representation distribution; The distribution of sensitive attributes is obtained by performing distribution construction processing on the attribute set of the recommendation system based on the tag kernel.

5. The method according to claim 1, characterized in that, The forgetting loss function in step S3 is: ； in, Represent the distribution for the user. For sensitive attribute distribution, user representation distribution Distribution of sensitive attributes The optimal bandwidth corresponding to the HSIC cores between is User representation distribution The optimal bandwidth corresponding to the HSIC core between itself is , Forgetting loss function, This is the balance coefficient.

6. The method according to claim 5, characterized in that, Step S2 includes calculating the optimal bandwidth through the following steps. : For the user representation distribution of the original user representation vector Distribution of sensitive attributes Calculate the unbiased estimator of HSIC The Regarding bandwidth function; Calculate the Standard deviation estimation The Regarding bandwidth The function; By maximizing To obtain the optimal bandwidth .

7. The method according to any one of claims 1 to 6, characterized in that, It also includes the following steps: Step S6: Update the user representation vector in the recommendation system with the user representation vector after the forgetting sensitive attribute.

8. A sensitive attribute forgetting device for a recommendation system, characterized in that, The embedding layer of the recommendation system includes a user embedding matrix and an item embedding matrix. Each row vector of the user embedding matrix corresponds to a different user representation vector, and each column vector of the item embedding matrix corresponds to an item representation vector for a different item. The device includes: The first processing unit is configured to perform matrix decomposition on the item embedding matrix to obtain a relevant orthogonal basis and an irrelevant orthogonal basis; and to perform feature decoupling on the user representation vector to obtain mutually orthogonal relevant vectors and irrelevant vectors; wherein the irrelevant vector is represented as a linear combination of the parameter to be optimized and the irrelevant orthogonal basis; the difference between the dot product of the relevant vector and the item embedding matrix and the dot product of the original user representation vector and the item embedding matrix is ​​less than a first preset threshold; the difference between the dot product of the irrelevant vector and the item embedding matrix is ​​less than a second preset threshold. The second processing unit is used to construct the user representation distribution for the user representation vector and the sensitive attribute distribution for the sensitive attributes, respectively, and to maximize the test energy efficiency by optimizing the bandwidth of the Hilbert-Schmidt independence criterion (HSIC kernel) to obtain the optimal bandwidth; The third processing unit is used to construct a forgetting loss function using the HSIC kernel under the optimal bandwidth to measure the dependency between the user representation distribution and the sensitive attribute distribution, and to replace the user representation vector in the forgetting loss function with a combination of the relevant vector and the irrelevant vector to obtain the objective function for the parameter to be optimized. The objective optimization unit is used to optimize the parameters to be optimized to update the irrelevant vectors with the objective function as the objective while keeping the relevant vectors unchanged, and to combine them with the relevant vectors to obtain a new user representation vector. The iterative processing unit is used to reconstruct the user representation distribution based on the new user representation vector and repeatedly execute the contents of the third processing unit and the target optimization unit until the preset iteration stopping condition is met, and the final user representation vector is used as the result after forgetting the sensitive attributes.

9. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1 to 7.

Images