Federal recommendation method for knowledge purification under guidance of data enhancement

Abstract

The invention belongs to the related technical field of federal recommendation, and particularly relates to a federal recommendation method based on knowledge purification under the guidance of data enhancement, which comprises the following steps: each client performs local data enhancement by using a sequence cutting strategy, converts user interaction data from a time domain to a frequency domain through fast fourier transform, and sends the user interaction data to a database; the frequency domain representation of each commodity and the frequency domain mask, used for low-pass filtering, of the client side are subjected to point multiplication, the representation obtained after point multiplication is input into an adaptive filter layer, different frequency component feature values in the representation are modulated, and it is ensured that higher-quality knowledge is obtained through model training. In the aspect of client parameter aggregation, a supernetwork-driven personalized aggregation strategy is introduced, the strategy dynamically generates aggregation weights specific to the clients under the condition of frequency domain representation and cut-off frequency distribution between different clients, and propagation of noisy knowledge between the clients in the global aggregation process is relieved. According to the method, the problem of serious sparsity of federal learning scene data is effectively relieved.

Description

Technical Field

[0001] This invention belongs to the technical field of federated recommendation, and more specifically, relates to a federated recommendation method based on knowledge purification guided by data augmentation. Background Technology

[0002] With the rapid development of artificial intelligence technology, recommender systems have become an indispensable part of modern internet applications. These systems provide personalized content recommendations by analyzing users' historical behavior. However, most current recommender systems are still based on a centralized architecture, relying on servers to collect and process users' personal information and behavioral data. With increasing global awareness of user privacy, centralized recommender systems face increasingly serious challenges in terms of privacy and security.

[0003] Federated learning, as a distributed machine learning paradigm, offers a feasible solution to the aforementioned problems. This framework allows recommendation models to be trained on the user's local device, with only the model parameters uploaded to the server for aggregation, without transmitting the original data, thus significantly reducing the risk of privacy breaches. In recent years, federated learning has made significant progress in the field of recommender systems, driving the formation and development of federated recommender systems.

[0004] Federated recommendation systems achieve personalized recommendation services while ensuring user data remains on local devices by collaboratively training models among distributed clients. This mechanism demonstrates good adaptability in distributed environments, effectively balancing privacy protection and recommendation performance. However, federated recommendation systems still face the severe challenge of data sparsity—since each client typically contains only a limited number of user historical interaction records, the product sequences are too short, severely impacting the accuracy of sequence recommendation models.

[0005] To address the problem of data sparsity, data augmentation has become one of the mainstream technical approaches. Existing research has attempted to use generative models to synthesize augmented data to enrich local datasets; or to leverage large language models to expand local information through retrieval augmentation methods. However, these methods are often difficult to apply in resource-constrained federated environments, and aggregating generative models on the server side may introduce additional privacy risks.

[0006] Therefore, in federated recommendation scenarios, how to enhance local datasets through simple and privacy-preserving methods has become a key research topic. Solving this problem will not only help improve the efficiency and performance of federated recommendations and effectively alleviate the constraints caused by data sparsity, but will also provide important support for the further development of artificial intelligence technology in privacy-sensitive scenarios. Summary of the Invention

[0007] In view of the above-mentioned defects or improvement needs of existing technologies, this invention provides a federated recommendation method based on knowledge purification guided by data augmentation, which aims to effectively alleviate the serious data sparsity problem that is common in recommendation systems under federated learning scenarios in existing technologies.

[0008] To achieve the above objectives, according to one aspect of the present invention, a federated recommendation method based on knowledge sanitization guided by data augmentation is provided, wherein each client uses a product recommendation model constructed by the following method to recommend products: Client: The local historical product interaction sequence is divided into sliding window segments. For each continuous subsequence obtained by the segmentation, a truncation position is randomly selected. The part from the start point to the truncation position of the continuous subsequence is extracted as an enhanced subsequence, thereby obtaining multiple enhanced subsequences of the continuous subsequence. Feature representations of each item in each enhanced subsequence are extracted through an embedding layer. The feature representations of each item are transformed from the time domain to the frequency domain using FFT, denoted as frequency representations. The frequency domain representation of each item is multiplied by the frequency domain mask used for low-pass filtering on the client side. The multiplied representation is then input into an adaptive filter layer to modulate the values ​​of different frequency components in the representation. The adaptively filtered representation is mapped back to the time domain using iFFT to obtain a new feature representation for each item. The feature representations of each item in each enhanced subsequence before FFT are vector-added and fused with the corresponding new feature representations. All fused features of the enhanced subsequence are then input into a feedforward neural network via a normalization layer for nonlinear feature transformation. After residual connections and a normalization layer, the output recommendation result is used for local iterative parameter updates. After multiple iterations, the parameters of the adaptive filter layer are dimensionality-reduced. The dimensionality-reduced parameter features, along with the parameters of the embedding layer, two normalization layers, and the feedforward neural network, are uploaded to the server. On the server side: A hypernetwork is used to output the affinity weights between each client and all clients, including the current client, based on the parameter features of each client. A frequency domain similarity adjustment factor is calculated between the current client and each client, including the current client, based on the cutoff frequencies corresponding to the frequency domain masks of all clients. The affinity weights and frequency domain similarity adjustment factors between the current client and each client are then fused together as the client's personalized aggregation weights relative to each client. This weighted sum is used to calculate the current model parameters for each client, which are then used as the updated model parameters for that client and sent to the client. The two ends jointly repeat the above process until the termination condition is met, at which point the local federated recommendation model is obtained.

[0009] Furthermore, the embedding layer includes a product encoding layer and a time-location encoding layer. Through the product encoding layer and the time-location encoding layer, the embedding representation and time-location encoding information of each product in each enhanced subsequence are extracted respectively. The embedding representation and time-location encoding information of each product are vector summed to serve as the feature representation of the product.

[0010] Furthermore, before applying FFT, the method performs Dropout regularization and layer normalization operations on the feature representation of each item.

[0011] Furthermore, principal component analysis is used to perform the dimensionality reduction.

[0012] Furthermore, within each client, all augmented subsequences are divided into a training sample set and a validation sample set; Each client employs an adaptive frequency optimization method based on differentiable neural architecture search to determine its corresponding frequency domain mask. The implementation method is as follows: M candidate cutoff frequencies are pre-defined for all clients, denoted as the frequency set. For each candidate frequency, a corresponding learnable low-pass filter branch is constructed, forming... There are 1 parallel low-pass filter branch, and the input of each branch is the output of the FFT in the local product recommendation model. The output is used as the input of the adaptive filter layer in the local product recommendation model. Initialize a set of weight vectors Each element represents the candidate weight corresponding to the candidate cutoff frequency; the candidate weights of each candidate cutoff frequency are converted into a probability distribution through the Softmax normalization operation; this distribution is used to perform a weighted summation of the adaptively filtered representations based on the outputs of each branch of the adaptive filter layer, which serves as the input to the iFFT in the local product recommendation model. A subset of augmented sequences is randomly sampled from the training sample set to form a sub-training sample set, and a subset of augmented sequences is randomly sampled from the validation sample set to form a sub-validation sample set. A two-layer optimization structure is used to train the parameters and weight vectors of the local product recommendation model. In this case, after the training termination condition is met, the candidate cutoff frequency corresponding to the weight element with the highest probability is used as the cutoff frequency required for sample denoising in the construction of the local product recommendation model. Construct a local frequency domain mask, where the structure of the frequency domain mask is the same as that of the frequency domain representation, and the mask elements corresponding to frequencies greater than the cutoff frequency are set to 0, while the mask elements corresponding to frequencies less than or equal to the cutoff frequency are set to 1.

[0013] Furthermore, a two-layer optimization structure is used to train vectors. At that time, an alternating update mechanism is adopted. First, the current model parameters are fixed, and then the architecture parameter vector is updated based on the performance of the sub-validation sample set. Subsequently, the architecture parameter vector is fixed. Update model parameters using sub-training sample sets; The outer objective is to optimize the vector. The loss function that minimizes the sub-validation sample set is expressed as: The inner goal is to optimize the model parameters of the product recommendation model to be built for the corresponding client. The loss function that minimizes the sub-training sample set is expressed as: In the formula, Let represent the loss functions on the sub-validation sample set and the sub-training sample set, respectively, and let be based on the gradient approximation strategy. .

[0014] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0015] According to another aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is run by a processor, it controls the device where the storage medium is located to perform the steps of the method described above.

[0016] According to another aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of the method described above.

[0017] In summary, compared with the prior art, the technical solutions conceived by this invention have the following main advantages: 1. This invention proposes a federated recommendation method with knowledge purification guided by data augmentation. This method proposes a simple and privacy-preserving data augmentation mechanism, addressing the particularly prominent issue of data sparsity in federated learning environments. While a sliding window strategy can typically be used to partition the original sequence, this approach alone is insufficient to adequately alleviate data sparsity. Therefore, a sequence pruning operation is further employed. The cutoff position is randomly selected from the partitioned sequence, and a continuous subsequence of a specified length is extracted as augmentation samples, thereby expanding the training data scale and enhancing the model's learning ability under sparse data conditions. Secondly, this method proposes a post-augmentation knowledge purification approach based on adaptive frequency denoising. While data augmentation expands the samples, it inevitably introduces noise, specifically manifested as incomplete user intent information and impaired temporal dependencies. Therefore, a knowledge purification method based on adaptive frequency denoising is proposed: firstly, the sequence information is transformed from the time domain to the frequency domain using Fast Fourier Transform, a cutoff frequency is set for each client, and a low-pass filtering strategy is used to selectively retain low-frequency components. To further refine user preference modeling and filter out fine-grained noise interference, a learnable filter layer is introduced in the frequency domain, maintaining the local privacy of the filter parameters and preventing them from being uploaded to the server. Furthermore, the method proposes a parameter aggregation approach based on a hypernetwork and similarity awareness. After determining the cutoff frequency corresponding to the optimal frequency domain mask for each client, a hypernetwork is deployed on the server, using the client's dimensionality-reduced filter parameters as input and outputting personalized aggregation weights. To more accurately distinguish client differences based on different frequency features, the aggregation weights are further combined with the cutoff frequency coefficients to achieve personalized parameter aggregation. This invention effectively alleviates the severe data sparsity problem commonly found in existing recommender systems in federated learning scenarios. Attached Figure Description

[0018] Figure 1 A schematic diagram of a federated recommendation method based on knowledge purification guided by data augmentation, provided for an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the data sparsity of a recommendation system provided in an embodiment of the present invention; Figure 3 A schematic diagram illustrating a simple data augmentation strategy provided in an embodiment of the present invention; Figure 4 A schematic diagram of adaptive knowledge purification for a client provided in an embodiment of the present invention; Figure 5 A flowchart illustrating the two-layer optimization strategy for architecture search provided in an embodiment of the present invention; Figure 6 A schematic diagram illustrating parameter aggregation based on hypernetwork and similarity awareness provided in an embodiment of the present invention; Figure 7This is a schematic diagram of a federated recommendation system based on knowledge purification guided by data augmentation, provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0020] The following explanations are provided for the relevant terms mentioned in the embodiments of this specification: Federated Recommendation: Federated recommendation is a distributed machine learning paradigm that achieves personalized recommendations within a privacy-preserving framework. Its core mechanism involves training recommendation models on multiple client devices using local data, and then securely aggregating only the model parameters or gradients to a server, without collecting or exposing any raw user data. This system aims to meet increasingly stringent data privacy regulations and overcome the limitations of traditional recommendation systems in data silo scenarios, achieving a balance between user privacy and recommendation performance in a distributed environment.

[0021] Data augmentation is a technique that expands the training sample size and improves the model's generalization ability by generating synthetic data or transforming original data. In the field of recommender systems, typical data augmentation methods include sequence noise enhancement, interactive interpolation, and synthesizing user behavior sequences based on generative models. This technique is mainly used to alleviate the data sparsity and cold start problems commonly found in recommender scenarios. It can effectively enrich the model training signal without introducing additional real user data, and is particularly suitable for scenarios where client-side local data is limited in federated recommender environments.

[0022] Federated recommender systems, a novel recommender architecture that has emerged in recent years, aim to provide personalized recommendations through cross-device collaborative training while ensuring user privacy and data security. Unlike traditional centralized recommender systems, federated recommender systems retain user data on local devices, participating in collaborative training only through shared model parameters or gradient updates, thus avoiding direct transmission of raw data. This mechanism not only adapts to increasingly stringent privacy regulations such as GDPR and CCPA, but is also suitable for distributed application scenarios such as mobile devices, the Internet of Things, and cross-platform services, achieving an effective balance between privacy protection and recommendation performance.

[0023] Building upon federated recommendation, federated sequence recommendation further incorporates time-series features of user behavior to capture the dynamic evolution of user interests. Under this architecture, the system collaboratively models user behavior sequences across multiple clients using a federated learning framework, thereby achieving privacy-preserving sequential personalized recommendations without disclosing raw interaction data. Its basic process includes the following stages: (1) Global model initialization and distribution: The server initializes the global sequence recommendation model and distributes the model parameters to each participating client as the initial state for local training; (2) Client-side local sequence modeling: Each client models user behavior patterns and interest transfer characteristics based on its own user interaction history. During local training, only local data is used, and the training objective is usually set to maximize the predicted probability of the next interaction behavior; (3) Model Upload and Global Aggregation: After local training is completed, the client uploads the updated model to the server. The server uses a privacy-preserving aggregation algorithm to merge the model updates from different clients and generate a new generation of global model.

[0024] The above process is executed iteratively until the model reaches the preset convergence criterion in terms of verification performance or communication rounds.

[0025] Compared to traditional centralized sequence recommendation, federated sequence recommendation can effectively capture the temporal dynamics of user interests without requiring data to leave the local machine, achieving a balance between privacy protection and model performance. Furthermore, this system is naturally adapted to the challenges posed by data heterogeneity and client dynamism, such as uneven client activity, significant differences in behavioral sequence lengths, and sparse cold-start user data. By combining local sequence modeling with a global parameter sharing mechanism, the system maintains the temporal expressiveness of sequence recommendations while achieving a balance between personalization and generalization capabilities in a distributed environment.

[0026] However, federated sequence recommendation systems still face the severe challenge of data sparsity. Since each client typically contains only a limited number of historical interaction records, sparsity makes it difficult for sequence recommendation models to capture effective temporal dependencies, thus affecting recommendation accuracy. To address this issue, two key challenges need to be addressed: First, some existing methods rely on generative models to synthesize augmented data. While this avoids directly performing augmentation operations on the local dataset, during the global parameter aggregation phase, the server may infer the client's original data based on the received model parameters, posing a privacy risk. Second, the training and inference processes of generative models typically require significant computational and communication resources, making them unsuitable for resource-constrained federated learning environments, especially for large-scale real-time recommendations. It is worth noting that these methods essentially still rely on generative models or large language models to augment the local dataset through retrieval, failing to fundamentally solve the data sparsity problem in privacy-preserving scenarios.

[0027] In distributed scenarios, the data volume of each client is generally sparse, making it difficult to support accurate local model building. To address this, this invention proposes a federated recommender system method based on knowledge purification guided by data augmentation. At the client-side level, each client first performs local data augmentation using a sequence pruning strategy to alleviate the data sparsity problem. Subsequently, a frequency-based adaptive purification mechanism filters out noise contained in the augmented data, thereby providing higher-quality knowledge representation for model training. Specifically, a preferred approach is designed with a two-layer optimization strategy, adaptively selecting cutoff frequency similarity for each client to achieve personalized denoising and avoid information loss caused by uniform filtering. At the client-to-client level, a hypernetwork-driven personalized aggregation mechanism is introduced. This mechanism dynamically generates aggregation weights adapted to specific clients based on the distribution characteristics of frequency domain representations and cutoff frequencies among different clients, thereby suppressing the propagation of noisy knowledge among clients during the global aggregation process. Through these dual purification mechanisms, this method significantly improves the personalization effect and overall model performance in federated recommender scenarios.

[0028] Example 1 A federated recommendation method based on knowledge sanitization guided by data augmentation is proposed, in which each client uses a product recommendation model constructed by the following method to recommend products, such as... Figure 1 As shown: Client: The local historical product interaction sequence is divided into sliding window segments. For each continuous subsequence obtained by the segmentation, a truncation position is randomly selected. The part from the start point to the truncation position of the continuous subsequence is extracted as an enhanced subsequence. Thus, multiple enhanced subsequences of the continuous subsequence are obtained. Each enhanced subsequence is used as a sample to obtain the sample set of the client. Feature representations of each item in each enhanced subsequence are extracted through an embedding layer. The feature representations of each item are transformed from the time domain to the frequency domain using FFT, denoted as frequency representations. The frequency domain representation of each item is multiplied by the frequency domain mask used for low-pass filtering on the client side. The multiplied representation is then input into an adaptive filter layer to modulate the values ​​of different frequency components in the representation. The adaptively filtered representation is mapped back to the time domain using iFFT to obtain a new feature representation for each item. The feature representations of each item in each enhanced subsequence before FFT are vector-added and fused with the corresponding new feature representations. All fused features of the enhanced subsequence are then input into a feedforward neural network via a normalization layer for nonlinear feature transformation. After residual connections and a normalization layer, the output recommendation result is used for local iterative parameter updates. After multiple iterations, the parameters of the adaptive filter layer are dimensionality-reduced. The dimensionality-reduced parameter features, along with the parameters of the embedding layer, two normalization layers, and the feedforward neural network, are uploaded to the server. On the server side: A hypernetwork is used to output the affinity weights between each client and all clients, including the current client, based on the parameter features of each client. A frequency domain similarity adjustment factor is calculated between the current client and each client, including the current client, based on the cutoff frequencies corresponding to the frequency domain masks of all clients. The affinity weights and frequency domain similarity adjustment factors between the current client and each client are then fused together as the client's personalized aggregation weights relative to each client. This weighted sum is used to calculate the current model parameters for each client, which are then used as the updated model parameters for that client and sent to the client. The two ends jointly repeat the above process until the termination condition is met, at which point the local federated recommendation model is obtained.

[0029] like Figure 2 The diagram illustrates the data sparsity under federated recommendation. Data sparsity is one of the most prominent challenges in federated recommendation systems. In this embodiment, because user data is distributed across various clients, the system cannot directly aggregate all user interaction records as in centralized recommendation, resulting in extremely limited data volume for each client. Most users have short historical product interaction sequences (interactions with a single product, all interactions), containing only a small number of items, making it difficult to form a complete user interest profile. Furthermore, the data distribution varies significantly across different clients; many items only appear in the local data of a very small number of users, resulting in a highly sparse structure in the global user-product interaction matrix. This sparsity not only limits the model's ability to learn user preferences but also weakens the effectiveness of model parameter updates during the federated aggregation phase.

[0030] More seriously, data sparsity in federated scenarios often exhibits a dual characteristic: on the one hand, "local sparsity," meaning that interaction samples are scarce within a single client; on the other hand, "global sparsity," meaning that a large number of users or items are not shared among clients. This dual sparsity makes it difficult for product recommendation models (encoders) based on historical product interaction sequences on each client to capture the similarities between users and the potential relationships between items, leading to a decline in recommendation performance. To alleviate this problem, researchers typically employ strategies such as cross-client knowledge distillation, latent vector sharing, data augmentation, or generative sample synthesis to improve data utilization efficiency without compromising privacy, thereby mitigating the sparsity problem in federated recommendations.

[0031] Based on this, embodiments of the present invention provide a model training method for knowledge purification guided by data augmentation, so as to achieve personalized model recommendation.

[0032] First, this method proposes a simple and privacy-preserving data augmentation mechanism. Data sparsity is particularly prominent in federated learning environments. While a sliding window strategy can typically be used to partition the original sequence, this approach alone is insufficient to adequately alleviate data sparsity. Therefore, this embodiment further employs a sequence pruning operation, randomly selecting cutoff positions on the partitioned sequence and extracting continuous subsequences of a specified length as augmentation samples. This expands the training data scale and enhances the model's learning ability under sparse data conditions.

[0033] Secondly, this method proposes a post-augmentation knowledge cleansing approach based on adaptive frequency denoising. While data augmentation expands the sample, it inevitably introduces noise, specifically manifested as incomplete user intent information and impaired temporal dependencies. Therefore, this embodiment proposes a knowledge cleansing method based on adaptive frequency denoising: First, the sequence information is transformed from the time domain to the frequency domain using Fast Fourier Transform (FFT). A cutoff frequency is set for each client, and a low-pass filtering strategy is employed to selectively retain low-frequency components. To further achieve refined user preference modeling and filter out fine-grained noise interference, this embodiment introduces a learnable filter layer in the frequency domain, maintaining the filter parameters locally private and not uploading them to the server. This involves pre-setting the optimal cutoff frequency for each client for low-pass filtering. A preferred scheme for determining this is proposed later in this embodiment: introducing an adaptive frequency selection strategy to customize a personalized noise removal mechanism for each client.

[0034] Furthermore, this method proposes a parameter aggregation approach based on a hypernetwork and similarity awareness. After determining the cutoff frequency corresponding to the optimal frequency domain mask for each client, this embodiment deploys a hypernetwork on the server side, using the client's dimensionality-reduced filter parameters as input and outputting personalized aggregation weights. To more accurately distinguish client differences with different frequency features, the aggregation weights are further combined with the cutoff frequency coefficients to achieve personalized parameter aggregation. It should be noted that the embedding layer parameters are still aggregated using the traditional FedAvg method to maintain global consistency, while regularization constraints are used to alleviate the distribution differences between global and local embeddings.

[0035] In this way, while protecting user privacy, local data can be effectively enhanced for model training, and the enhanced knowledge can be sanitized, thereby improving the model's generalization ability and adaptability. Specifically, this embodiment has the following advantages: (1) High deployment flexibility: This embodiment can achieve superior performance with low resource investment, and can flexibly expand the model structure according to actual resource needs, thereby further improving the recommendation effect and having good scene adaptability.

[0036] (2) High privacy protection: This embodiment effectively addresses the data sparsity problem in federated recommendations. Compared with existing data generation methods based on generative models, this embodiment does not generate original client data during model aggregation, significantly reducing the risk of privacy leakage and better protecting the privacy and security of each client.

[0037] (3) High efficiency in resource utilization: Under the federated learning framework, each client only needs to perform simple data augmentation based on local data, without the need for additional data caching or generation processes, thereby improving the efficiency of system resource utilization and better meeting the needs of a sustainable federated recommendation environment.

[0038] As a preferred embodiment, the above-mentioned embedding layer includes a product encoding layer and a time location encoding layer. Through the product encoding layer and the time location encoding layer, the embedding representation and time location encoding information of each product in each enhanced subsequence are extracted respectively. The embedding representation and time location encoding information of each product are vector summed to serve as the feature representation of the product.

[0039] like Figure 3 The diagram illustrates a simple data augmentation strategy.

[0040] In federated sequence recommendation scenarios, the data distribution is highly sparse because each client only holds local data about its own users, and the number of user behaviors varies significantly. This sparsity not only weakens the model's ability to capture dynamic user interests but also makes the global model face problems of statistical discrepancies and generalization difficulties during aggregation. To alleviate the insufficient training signal caused by data sparsity, a common approach is to use a sliding window strategy to slice the original interaction sequence, dividing the long sequence into multiple short, continuous subsequences (of the same length), thereby generating more training samples and preserving local temporal dependencies. Another method is sequence pruning, which involves randomly selecting a truncation point in the input sequence and extracting a continuous subsequence of variable length from the starting point to that position.

[0041] However, relying solely on sliding window partitioning is insufficient to adequately improve sample diversity because the sequence patterns between different windows often overlap significantly, failing to effectively simulate the randomness and diversity of real user behavior. Therefore, this invention further utilizes a sequence pruning enhancement strategy to expand the sample space based on the sliding window and enhance the model's learning ability for sparse sequences. Specifically, sequence pruning involves randomly selecting a truncation position for each continuous subsequence, extracting the portion from the start point of the continuous subsequence to the truncation position as an enhanced subsequence, thereby obtaining multiple enhanced subsequences for the continuous subsequence. All enhanced subsequences corresponding to all continuous subsequences constitute the training sample set and the validation sample set, with each enhanced subsequence serving as a training sample.

[0042] This process not only significantly increases the number of training samples without introducing additional data, but also exposes the model to user behavior fragments of different lengths and stages during the learning process, thereby better capturing the phased evolution characteristics of user interests. Compared to the traditional sliding window method, sequence pruning has stronger sample diversity and generalization advantages in sparse data environments, effectively improving the model's performance on cold clients and short-sequence users. Furthermore, this enhancement mechanism has a natural advantage in federated scenarios—all its operations are completed locally on the client, requiring no additional communication overhead and not increasing the risk of privacy leaks.

[0043] like Figure 4 The diagram illustrates the adaptive knowledge cleansing process on the client side. The adaptive frequency module in the diagram performs the following functions: It uses FFT to transform the feature representations of each item from the time domain to the frequency domain, denoted as frequency representation; it multiplies the frequency domain representation of each item with the client's frequency domain mask used for low-pass filtering, and inputs the multiplied representation into the adaptive filter layer to modulate the values ​​of different frequency components in the representation; finally, it uses iFFT to map the adaptively filtered representation back to the time domain, obtaining a new feature representation for each item.

[0044] For each augmented subsequence, the embedding representation of each product in that subsequence is obtained through a product encoding layer. Considering the crucial role of the relative temporal positional relationships between products in historical product interaction sequences in modeling user behavior patterns for product recommendation tasks based on historical product interaction sequences, this embodiment further introduces a learnable temporal positional encoding layer to characterize the temporal order relationships of products in each augmented subsequence, obtaining the temporal positional encoding information for each product. For example, if product a appears in the second position of the sequence, its encoding information is represented as follows: The length is the same as the length of the augmented subsequence. Since the parameters of the product encoding layer and the temporal location encoding layer are usually randomly initialized, the model may be unstable in the early stages of training. Therefore, in the input stage, preferably, the embedding representation of each product is vector-summed with the temporal location encoding information to obtain the feature representation of the product. The feature representation of each product is then processed by Dropout regularization and layer normalization to enhance the stability and generalization ability of the model.

[0045] After obtaining stable sequence representations (the feature representations of each item in each enhanced subsequence, collectively referred to as sequence representations), a Fast Fourier Transform is used to transform the user's behavior sequence (the sequence representations corresponding to each enhanced subsequence) from the time domain to the frequency domain, denoted as the frequency domain representation, thereby capturing longer-range global dependencies in the sequence. Subsequently, this embodiment designs a low-pass filtering strategy: first, a cutoff frequency is set based on the frequency domain representation, a frequency domain mask is constructed, and only low-frequency components are retained to form a new frequency domain representation, in order to suppress high-frequency noise and extract long-term stable interest signals in the sequence.

[0046] To further refine user preference modeling (constructing the recommendation model) and effectively remove fine-grained noise, a learnable filter layer is introduced in the frequency domain. This filter layer adaptively modulates different frequency components using a complex weight matrix, thereby preserving the effective signal while selectively suppressing noise in specific frequency bands. The filtered frequency domain representation is then mapped back to the time domain via an inverse fast Fourier transform, generating a smoothed sequence representation.

[0047] To maintain the integrity of the information flow and prevent the vanishing gradient problem, a residual connection mechanism is introduced in the time domain. The original input (the sequence representation of each enhanced subsequence before its Fast Fourier Transform) is added to and fused with the filtered output (the sequence representation of the enhanced subsequence after its Inverse Fast Fourier Transform and smoothing). The fused features undergo layer normalization to improve numerical stability before being input into a feedforward neural network for nonlinear feature transformation. This feedforward network employs a two-layer linear mapping structure and introduces nonlinear feature representation capabilities through an activation function. Finally, through residual connections and layer normalization operations, the output feature representation of the l-th layer of the model is obtained.

[0048] In the output stage of the model, the user's preference score for candidate items at the current moment is calculated based on the user's historical interaction records through matrix transpose and dot product operations, thereby generating the final recommendation result.

[0049] As a preferred implementation, this embodiment also proposes a personalized cutoff frequency selection strategy based on dual-layer frequency optimization.

[0050] In a federated scenario, the data distribution among clients exhibits significant heterogeneity, with fundamentally different noise characteristics and signal patterns. Using a uniform global cutoff frequency is insufficient to meet the individual needs of all clients, easily leading to over-filtering or under-filtering. Therefore, this embodiment designs a two-layer optimization framework based on differentiable neural architecture search to adaptively determine the optimal cutoff frequency for each client.

[0051] Within each client, all enhanced subsequences are divided into training and validation sample sets; each client uses an adaptive frequency optimization method based on differentiable neural architecture search to determine the frequency domain mask for that client. The implementation is as follows: M candidate cutoff frequencies are pre-defined for all clients, denoted as the frequency set. For each candidate frequency, a corresponding learnable low-pass filter branch is constructed, forming... There are 1 parallel low-pass filter branch, and the input of each branch is the output of the FFT in the local product recommendation model. The output is used as the input of the adaptive filter layer in the local product recommendation model. Initialize a set of weight vectors Each element represents the candidate weight corresponding to the candidate cutoff frequency; the candidate weights of each candidate cutoff frequency are converted into a probability distribution through the Softmax normalization operation; this distribution is used to perform a weighted summation of the adaptively filtered representations based on the outputs of each branch of the adaptive filter layer, which serves as the input to the iFFT in the local product recommendation model. A subset of augmented sequences is randomly sampled from the training sample set to form a sub-training sample set, and a subset of augmented sequences is randomly sampled from the validation sample set to form a sub-validation sample set. A two-layer optimization structure is used to train the parameters and weight vectors of the local product recommendation model. In this case, after the training termination condition is met, the candidate cutoff frequency corresponding to the weight element with the highest probability is used as the cutoff frequency required for sample denoising in the construction of the local product recommendation model. Construct a local frequency domain mask, where the structure of the frequency domain mask is the same as that of the frequency domain representation, and the mask elements corresponding to frequencies greater than the cutoff frequency are set to 0, while the mask elements corresponding to frequencies less than or equal to the cutoff frequency are set to 1.

[0052] like Figure 5The diagram illustrates a flowchart of a two-tier optimization strategy for architecture search.

[0053] In federated recommendation scenarios, the data distribution (products interacted with by users) among clients exhibits significant heterogeneity, with marked differences in noise characteristics and signal patterns (frequency domain representation) across clients. Using a uniform global cutoff frequency for filtering makes it difficult to simultaneously meet the personalized needs of different clients, potentially leading to over-filtering or under-filtering on some clients, thus impacting model performance. Therefore, this embodiment proposes an adaptive frequency optimization method based on differentiable neural architecture search, dynamically determining the optimal cutoff frequency for each client through a two-layer optimization framework.

[0054] Specifically, firstly, make a reservation Candidate cutoff frequencies, frequency set And for each candidate frequency, a corresponding learnable low-pass filter module is constructed, thereby forming Each client maintains a set of architecture parameter vectors. (Multiple parallel filtering branches.) Each element represents the weight of a corresponding candidate cutoff frequency. Through Softmax normalization, the weights of the candidate cutoff frequencies for each branch are converted into a probability distribution. A subset of augmented subsequences is randomly sampled from the training sample set as training samples, forming a sub-training sample set. A subset of augmented subsequences is randomly sampled from the validation sample set as validation samples, forming a sub-validation sample set. The weights are then trained. After obtaining the training termination condition, the candidate cutoff frequency corresponding to the weight with the highest probability is used as the cutoff frequency required for denoising the training samples in the client-side product recommendation model. This achieves weighted selection of branches with different cutoff frequencies. This method of determining the cutoff frequency belongs to the continuous relaxation strategy, which transforms the originally discrete frequency selection problem into a differentiable optimization problem, thereby supporting an end-to-end gradient optimization process.

[0055] In this framework, the cutoff frequency search process is modeled as a two-layer optimization structure: the outer layer objective is to optimize the architecture parameters. The inner objective is to optimize the model parameters of the product recommendation model to be built for the corresponding client under a given architecture, by minimizing the loss function of the sub-validation sample set. To minimize the loss function of the sub-training sample set.

[0056] The outer target is represented as: The inner target is represented as: In the formula, Let represent the loss functions on the sub-validation sample set and the sub-training sample set, respectively.

[0057] Because of the strong coupling between architecture parameters and model parameters, simultaneous optimization can easily lead to overfitting. To reduce computational complexity, this embodiment employs a one-step gradient approximation strategy to simplify the inner optimization process and avoid the need to fully train and converge the model parameters each time the architecture parameters are updated. This approximation can be expressed as: .

[0058] During the optimization process, this embodiment adopts an alternating update mechanism: first, the current model parameters are fixed, and the architecture weight parameters are updated based on the performance of the validation set; then, the architecture weight parameters are fixed, and the model parameters are updated using the training set.

[0059] In the parameter aggregation stage, this embodiment adopts a differentiated aggregation strategy for different types of parameters: Learnable filter parameters and architecture search parameters Keep it local and private, and do not upload it to the server. This is because filter parameters directly characterize the unique noise distribution and frequency domain preference characteristics of each client, while architecture parameters reflect the client's optimal frequency selection strategy; both reflect the individual differences between clients.

[0060] Other network parameters (product coding layer, time and location coding layer, normalization layer, feedforward neural network, etc.) are uploaded to the server for aggregation to make full use of global knowledge, thereby improving the model's generalization ability and training stability.

[0061] Through the above design, this embodiment can realize personalized frequency domain modeling in a heterogeneous federated environment, enabling each client to adaptively select the optimal cutoff frequency based on its own data characteristics, thereby improving recommendation performance and model robustness while ensuring privacy and security.

[0062] like Figure 6 The diagram illustrates a server-side parameter aggregation based on hypernetworks and similarity awareness.

[0063] Determine the optimal cutoff frequency for each client. Subsequently, although frequency domain filtering can effectively suppress local noise interference, significant heterogeneity conflicts may still exist between subsequent network layer parameters due to differences in data feature distribution and behavioral patterns among various clients. The traditional FedAvg aggregation method uses equal weights to average the parameters of each client, ignoring the impact of differences in frequency domain features. This can easily lead to the aggregated global model failing to fully adapt to the personalized characteristics of each client, thereby affecting the overall convergence performance and personalized expressive ability of the model.

[0064] To address this, this embodiment proposes an intelligent aggregation mechanism based on hypernetworks. By learning the feature similarity between clients, it achieves a more targeted and personalized parameter fusion strategy. Specifically, each client... Learnable filter parameters It contains rich user frequency domain preference information and noise feature distribution. However, directly modeling the similarity of these parameters in high-dimensional space leads to excessive computational complexity and difficulty in achieving stable convergence. Therefore, this embodiment first performs principal component analysis (PCA, the preferred approach) to reduce the dimensionality of the learnable filter parameters and extract their core feature representation:

[0065] in, The feature dimension after dimensionality reduction.

[0066] Deploy a hypernetic network on the server side This network consists of a multi-layered fully connected structure and possesses powerful capabilities for nonlinear mapping and relation modeling. The hypernetwork utilizes the dimensionality reduction characteristics of each client. As input, output the affinity weight distribution between this client and other clients. The more similar the frequency domain features, the greater the weight.

[0067] Relying solely on filter characteristics may not fully reflect the differences in frequency domain distribution among clients. Therefore, based on the proximity of clients at the optimal cutoff frequency, a frequency similarity adjustment factor is further introduced:

[0068] The more similar the optimal cutoff frequencies, the higher the frequency similarity value. By combining the affinity weights output by the hypernetwork with the frequency similarity, this invention constructs the final personalized aggregation weights:

[0069] This aggregation strategy takes into account both feature correlation and frequency domain consistency among clients, enabling personalized aggregation of model parameters for each client.

[0070] During the aggregation phase, a hierarchical parameter aggregation mechanism is employed: the aforementioned personalized weights are applied to the high-level network parameters that have undergone low-pass filtering. Aggregation is performed to achieve customized model integration based on frequency domain features; learnable filter parameters are kept locally private and do not participate in global sharing to prevent leakage of personalized frequency domain features; embedding layer parameters continue to use standard FedAvg aggregation to ensure the consistency and stability of the global commodity representation space.

[0071] Furthermore, considering that data augmentation operations exacerbate the heterogeneity of data distribution among clients while expanding the sample size, an embedding regularization constraint is introduced to maintain semantic consistency between the global model and the local model in the item embedding space. This constraint uses the squared Euclidean distance between the embedding representation obtained from the server-aggregated embedding layer and the embedding representation obtained from the local model's embedding layer as a penalty term to ensure semantic space alignment, thereby improving cross-client knowledge transfer performance. Finally, the training objective function for each client consists of cross-entropy loss and embedding regularization loss, achieving a balance between personalized modeling and global consistency.

[0072] like Figure 7 This diagram illustrates a federated recommendation method based on knowledge purification guided by data augmentation. The core idea is to address the data sparsity problem prevalent in federated recommendation scenarios by proposing a recommendation modeling method that combines data augmentation, adaptive frequency denoising, and a personalized parameter aggregation mechanism based on a hypernetwork. This method significantly improves recommendation performance and model generalization ability in sparse data environments by enhancing the diversity of training samples at the data level, suppressing high-frequency noise interference at the model level, and achieving differentiated modeling among clients at the aggregation level.

[0073] Specifically, the algorithm process mainly includes the following two key stages: The first stage is the personalized cutoff frequency search and local data augmentation stage. In this stage, each client first slices its local user interaction sequence using a sliding window strategy and expands the training sample size through sequence pruning to alleviate the data sparsity problem. Subsequently, the user sequence interaction signal is mapped to the frequency domain using Fast Fourier Transform, and high-frequency noise components are identified and removed by analyzing its spectral distribution characteristics. Furthermore, an adaptive frequency selection strategy is designed to dynamically determine the optimal cutoff frequency for each client, realizing a personalized frequency denoising mechanism. Simultaneously, to enhance the ability to characterize user preference features, a learnable filter module is introduced to achieve fine-grained modeling of frequency domain features, and the filter parameters are kept in a local private state, thereby strengthening the personalized feature expression while protecting client privacy.

[0074] The second stage is the intelligent aggregation and model retraining stage based on the hypernetwork. After determining the optimal cutoff frequency for each client, a hypernetwork model is deployed on the server. It takes the filter parameter vectors of each client (dimension reduced by principal component analysis) as input and outputs personalized aggregation weights for different clients. This hypernetwork learns the feature similarity and frequency domain distribution relationship between clients to generate a weight distribution reflecting the differences between clients, thus achieving more targeted model fusion. To further characterize the differences in frequency domain features, the aggregation weights output by the hypernetwork are combined with the cutoff frequency coefficients of each client, forming a dual similarity-driven personalized parameter aggregation mechanism. Based on this, higher-level network parameters are personalized based on this mechanism, while the item embedding matrix still uses the traditional federated aggregation strategy to maintain the consistency of the global representation space. Simultaneously, regularization constraints between global and local embeddings are introduced to alleviate the distribution offset problem of the embedding representation, further improving the model's stability and cross-client semantic alignment capability.

[0075] Through the above design, a full-process optimization mechanism from data augmentation and frequency adaptation to personalized aggregation is realized, which can effectively improve the recommendation accuracy and personalization performance of the federated recommendation system in sparse and heterogeneous data environments while protecting user privacy.

[0076] In summary, this embodiment discloses a federated recommender system method based on knowledge purification guided by data augmentation. The method includes: each client first performing local data augmentation using a sequence pruning strategy to alleviate data sparsity. Next, leveraging signal processing theory, the method transforms user interaction data from the time domain to the frequency domain using Fast Fourier Transform (FFT), and utilizes a frequency-based adaptive knowledge purification mechanism to filter noise from the augmented data, thereby ensuring higher-quality knowledge acquired during model training. Specifically, the method employs a two-layer optimization strategy, adaptively selecting a cutoff frequency for each client to achieve a personalized denoising strategy and avoid information loss caused by uniform filtering. Then, regarding client parameter aggregation, the method introduces a hypernetwork-driven personalized aggregation strategy. This strategy dynamically generates client-specific aggregation weights based on the frequency domain representation and cutoff frequency distribution among different clients, thereby mitigating the propagation of noisy knowledge among clients during the global aggregation process.

[0077] In view of the common problems of severe data sparsity, limited computing resources, and difficulty in privacy protection in the recommendation system of federated learning in the present invention, an efficient federated recommendation method with knowledge purification under data augmentation guidance is proposed in this embodiment, which effectively alleviates the above technical bottlenecks and improves the overall performance and efficiency of the system.

[0078] Example 2 This application also relates to an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0079] The electronic device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The memory can be used to store computer programs and / or modules. The processor performs various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory.

[0080] The relevant technical solutions are the same as above, and will not be repeated here.

[0081] Example 3 This application also relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0082] Specifically, the memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0083] The relevant technical solutions are the same as above, and will not be repeated here.

[0084] Example 4 This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method described in the above embodiments of this application.

[0085] The relevant technical solutions are the same as above, and will not be repeated here.

[0086] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A federated recommendation method based on knowledge sanitization guided by data augmentation, characterized in that, Each client uses a product recommendation model constructed using the following method to recommend products: Client: The local historical product interaction sequence is divided into sliding window segments. For each continuous subsequence obtained by the segmentation, a truncation position is randomly selected. The part from the start point to the truncation position of the continuous subsequence is extracted as an enhanced subsequence, thereby obtaining multiple enhanced subsequences of the continuous subsequence. Feature representations of each item in each enhanced subsequence are extracted through the embedding layer; The feature representation of each product is transformed from the time domain to the frequency domain space using FFT, which is denoted as frequency representation. The frequency domain representation of each product is multiplied by the frequency domain mask used for low-pass filtering by the client. The multiplied representation is then input into an adaptive filter layer to modulate the value of different frequency component features in the representation. The iFFT is used to map the adaptively filtered representation back to the time domain, obtaining a new feature representation for each product. The feature representation of each product in each enhanced subsequence before FFT is vector-added and fused with the corresponding new feature representation. All fused features corresponding to the enhanced subsequence are then input into a feedforward neural network through a normalization layer for nonlinear feature transformation. After that, the recommendation result is output through a residual connection and a normalization layer for local iterative parameter update. After multiple iterations, the parameters of the adaptive filter layer are dimensionality reduced. The dimensionality-reduced parameter features, as well as the parameters of the embedding layer, two normalization layers, and the feedforward neural network, are uploaded to the server. Server-side: Using a hypernetwork based on the parameter features of each client, output the affinity weight between the client and all clients including the client; calculate the frequency domain similarity adjustment factor between the client and each client including the client based on the cutoff frequency corresponding to the frequency domain mask of all clients; The affinity weights and frequency domain similarity adjustment factors of the client and each other are combined to form the personalized aggregation weights of the client relative to each other. These weights are used to sum the current model parameters of each client and serve as the updated model parameters of the client, which are then sent to the client. The two ends jointly repeat the above process until the termination condition is met, at which point the local federated recommendation model is obtained.

2. The federal recommendation method as described in claim 1, characterized in that, The embedding layer includes a product encoding layer and a time-location encoding layer. Through the product encoding layer and the time-location encoding layer, the embedding representation and time-location encoding information of each product in each enhanced subsequence are extracted respectively. The embedding representation and time-location encoding information of each product are vector summed to serve as the feature representation of the product.

3. The federal recommendation method as described in claim 1, characterized in that, Before applying FFT, the method also performs Dropout regularization and layer normalization operations on the feature representation of each item.

4. The federal recommendation method as described in claim 1, characterized in that, Principal component analysis was used to perform the dimensionality reduction.

5. The federal recommendation method as described in claim 1, characterized in that, Within each client, all augmentation subsequences are divided into a training sample set and a validation sample set; Each client employs an adaptive frequency optimization method based on differentiable neural architecture search to determine its corresponding frequency domain mask. The implementation method is as follows: M candidate cutoff frequencies are pre-defined for all clients, denoted as the frequency set. For each candidate frequency, a corresponding learnable low-pass filter branch is constructed, forming... There are several parallel low-pass filter branches. The input of each branch is the output of the FFT in the local product recommendation model, and the output is used as the input of the adaptive filter layer in the local product recommendation model. Initialize a set of weight vectors Each element represents the candidate weight corresponding to the candidate cutoff frequency; the candidate weights of each candidate cutoff frequency are converted into a probability distribution through the Softmax normalization operation; this distribution is used to perform a weighted summation of the adaptively filtered representations based on the outputs of each branch of the adaptive filter layer, which serves as the input to the iFFT in the local product recommendation model. A subset of augmented sequences is randomly sampled from the training sample set to form a sub-training sample set, and a subset of augmented sequences is randomly sampled from the validation sample set to form a sub-validation sample set. A two-layer optimization structure is used to train the parameters and weight vectors of the local product recommendation model. In this case, after the training termination condition is met, the candidate cutoff frequency corresponding to the weight element with the highest probability is used as the cutoff frequency required for sample denoising in the construction of the local product recommendation model. Construct a local frequency domain mask, where the structure of the frequency domain mask is the same as that of the frequency domain representation, and the mask elements corresponding to frequencies greater than the cutoff frequency are set to 0, while the mask elements corresponding to frequencies less than or equal to the cutoff frequency are set to 1.

6. The federal recommendation method as described in claim 5, characterized in that, Training vectors using a two-layer optimization structure At that time, an alternating update mechanism is adopted. First, the current model parameters are fixed, and then the architecture parameter vector is updated based on the performance of the sub-validation sample set. Subsequently, the architecture parameter vector is fixed. Update model parameters using sub-training sample sets; The outer objective is to optimize the vector. The loss function that minimizes the sub-validation sample set is expressed as: The inner goal is to optimize the model parameters of the product recommendation model to be built for the corresponding client. The loss function that minimizes the sub-training sample set is expressed as: In the formula, Let represent the loss functions on the sub-validation sample set and the sub-training sample set, respectively, and let be based on the gradient approximation strategy. .

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program, when executed by a processor, controls the device on which the storage medium is located to perform the steps of the method as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 6.

Images