Social network public opinion analysis method and system based on personalized federal learning
By adopting personalized federal learning methods in social network public opinion analysis, using dynamic hierarchical aggregation and super network update mechanism to process non -independent and distributed data, solve the problem of data deviation affecting the generalized performance of models, and achieve more efficient and adaptability Public opinion analysis model.
Patent Information
- Application Number
- CN202411414401.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2025-01-21
AI Technical Summary
Data deviations caused by non-independent and same-distributed data in social network public opinion analysis affect the generalization performance of the model.
Using a method based on personalized federated learning, dynamic hierarchical aggregation and super network update mechanisms are adopted on the participant terminal and server side respectively to process non-independent and identically distributed data, reduce the negative impact of model aggregation, and generate public opinion analysis models that adapt to the habits of each user. .
Through the dynamic hierarchical aggregation and ultra -network update mechanism, the effect of data deviation on the generalization performance of data deviations on models has been reduced, the adaptability and performance of the model is improved, and the scalability of federal learning is enhanced.
Smart Images
Figure CN119338616A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of public opinion analysis, and in particular to a social network public opinion analysis method and system based on personalized federated learning. Background Art
[0002] Social network public opinion analysis is a public opinion analysis method based on social media and network platforms. It collects, organizes and analyzes the speech, opinions and emotional information posted by a large number of users on social media to understand the user's attitude, emotion and tendency towards specific topics, events or products, so as to adaptively provide relevant information for users to browse and use. With the rapid development of big data-driven artificial intelligence, protecting personal privacy and data security has become a new topic of concern in the field of social network public opinion analysis. Researchers have highlighted the potential security and privacy issues in social networks. Although there are some methods to protect user privacy and security, such as strengthening data security protection, strengthening user privacy protection, and limiting data collection and use, these methods all have their own shortcomings, such as hacker attacks and technical vulnerabilities that may lead to the theft or leakage of user data, and social network platforms provide more stringent privacy policies and setting options, which will reduce user experience. In addition, the existing protection methods ignore the increasing computing power of the participants and only consider how to protect data privacy issues on the server side. However, data privacy protection on the participant side also plays an important role in the security field and has received much attention in recent years. Therefore, federated learning came into being.
[0003] Federated learning is a collaborative machine learning method that allows multiple parties to jointly train machine learning models while keeping their original training data locally, without centralizing the data to a central server. This method iteratively trains specific machine learning models through distributed computing, allowing a large number of participants to cooperate while protecting privacy. In federated learning, the user's original data does not need to be transmitted to a central server, but the model is trained on each local terminal, which guarantees the data privacy and security of each participant. However, since the data is scattered across different terminals, the data usually does not have the same distribution, because the data collected by different terminals varies depending on the user's social network structure, interest areas, habits or other factors. The non-independent and identically distributed nature of this data leads to inconsistent model training directions for each terminal, which may ultimately affect the generalization performance of the model. Summary of the invention
[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a social network public opinion analysis method and system based on personalized federated learning to solve the problem that data bias caused by non-independent and identically distributed data affects the generalization performance of the model during social network public opinion analysis.
[0005] To solve the above technical problems, the present invention is implemented by adopting the following solutions: The present invention provides a social network public opinion analysis method based on personalized federated learning, which is executed by a participant terminal and includes: Step A: Use local social network data to train the model and generate a local model; Step B: Divide the local model into a weighted head layer and a full substitute body layer according to the distribution of local social network data, and generate a hierarchical weight matrix; Step C: Upload the local model to the server to generate a global model; Step D: Receive the global model sent by the server, and dynamically update the local model by hierarchical aggregation based on the global model, the local model, and the hierarchical weight matrix; Step E: looping steps C to D until the maximum number of iterations is reached to obtain the final local model; Step F: Apply the final local model to perform social network public opinion analysis.
[0006] Optionally, the method further includes: pre-processing the local social network data before step A; the pre-processing includes: The missing values of local social network data are processed using linear interpolation method; The bag-of-words model is used to extract features from local social network data after missing value processing, and the extracted features are standardized.
[0007] Optionally, the formula for dynamic hierarchical aggregation update is:
[0008] in, Indicates The hierarchical weight matrix of the participant terminals, Indicates Participant Terminal No. Round local model, represents the dot product matrix, Indicates that the server sends Participant Terminal No. The global model of the wheel, Indicates the round number.
[0009] Optionally, The hierarchical weight matrix of the participant terminals The update formula is:
[0010] in, represents the learning rate of the weighted first layer, Indicates The participant terminal is in It's a turn % of local social network data for calculation, represents the cross entropy loss function.
[0011] The present invention also provides a social network public opinion analysis method based on personalized federated learning, which is executed by a server and includes: Step a: Receive the local model uploaded by the participant terminal, and generate or update the model matrix according to the local model; Step b: Generate a model weight matrix based on the local model and the hypernetwork; Step c: Generate a global model based on the model weight matrix and the model matrix, and send the global model to the participant terminals to dynamically aggregate and update the local model; Step d: Loop steps a to c until the maximum number of iterations is reached.
[0012] Optionally, receiving a local model uploaded by a participant terminal, and generating or updating a model matrix according to the local model, including: Receive local models uploaded by participant terminals , according to the local model Aggregation generates or updates the model matrix ; Among them, the model matrix for:
[0013] In the formula, Indicates The local model of the participant terminal, , represents the number of participant terminals, Representative The local model of the participant terminal Layer model parameters, , Indicates the number of layers of the participant terminal's local model.
[0014] Optionally, the model weight matrix is:
[0015] in, Indicates The participant terminal model weight matrix, Indicates Participant Terminal No. The set of layer model weights; Indicates The participant terminal is about The model The weights of the layer parameters, , .
[0016] Optionally, the generation formula of the global model is:
[0017] in, Indicates that the corresponding A global model of the terminals of each participant.
[0018] The present invention also provides a social network public opinion analysis system based on personalized federated learning, including a server and a participant terminal, wherein the server is communicatively connected with a plurality of participant terminals; The server is configured to perform step a: receiving a local model uploaded by a participant terminal, and generating or updating a model matrix according to the local model; step b: generating a model weight matrix according to the local model and the hypernetwork; step c: generating a global model according to the model weight matrix and the model matrix, and sending the global model to the participant terminal to dynamically aggregate and update the local model; step d: looping steps a to c until the maximum number of iterations is reached; The participant terminal is configured to execute step A: use local social network data to train the model and generate a local model; step B: divide the local model into a weight head layer and a full substitute body layer according to the distribution of local social network data, and generate a hierarchical weight matrix; step C: upload the local model to the server to generate a global model; step D: receive the global model sent by the server, and update the local model according to the global model, the local model and the hierarchical weight matrix; step E: loop step C to step D until the maximum number of iterations is reached to obtain the final local model; step F: apply the final local model to perform social network public opinion analysis.
[0019] Compared with the prior art, the beneficial effects achieved by the present invention are: 1. The present invention adopts dynamic hierarchical aggregation and super network update mechanism on the participant side and server side respectively, combines the participant side with the server side, jointly handles the data deviation problem caused by non-independent and identically distributed data, reduces the negative impact on model aggregation, and obtains a social network public opinion analysis model that adapts to the habits of each user.
[0020] 2. The present invention uses a hypernetwork update mechanism on the server side, which enables the server side to select models or feature extractors suitable for different data types, so as to better process diverse social network data. In addition, the hypernetwork can automatically select or adjust the appropriate model according to the characteristics of the current task to maximize the model performance.
[0021] 3. The present invention uses dynamic hierarchical aggregation on the participant side, which can screen excellent data and allow each participant to independently decide the number of layers for local model updates and decide when to pass its model parameters to the upper aggregation layer. This provides greater flexibility to adapt to the computing power and data distribution of different participants. Only the update and aggregation of model parameters need to be communicated between participants and servers, without the need to transmit large amounts of raw data, which reduces the burden of network transmission and makes federated learning more scalable. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a flow chart of a social network public opinion analysis method based on personalized federated learning provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of dynamic hierarchical aggregation of participant terminals in a social network public opinion analysis method based on personalized federated learning provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of a super network structure of a server in a social network public opinion analysis method based on personalized federated learning provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0024] Embodiment 1: like Figure 1 As shown, this embodiment provides a social network public opinion analysis method based on personalized federated learning, which uses dynamic hierarchical aggregation on the participant terminal and a super network fusion mechanism on the server to analyze and detect social network public opinion, so as to more accurately provide users with personalized recommendations and advertising and other related information.
[0025] The social network public opinion analysis method based on personalized federated learning can be used in the AGNews dataset (i.e., news article dataset, which is a collection of news articles collected by the academic news search engine ComeToMyHead from more than 2,000 news sources) and the Movielens dataset (i.e., movie review dataset, including 120,000 training samples and 7,600 test samples, each of which is a short text with four types of labels; this dataset describes the 5-star rating and free text labeling activities of the movie recommendation service MovieLens, which contains 100,836 ratings and 3,683 label applications for 9,742 movies, which were created by 610 users between March 29, 1996 and September 24, 2018. This dataset was generated on September 26, 2018. Randomly selected users are included. All selected users have rated at least 20 movies. Demographic information is not included. Each user is represented by an id and no other information is provided). The AGNews dataset and the Movielens dataset are distributed to each participant terminal as their local social network data. The dataset allocated to each participant terminal is divided into a training set and a test set. The ratio of training set to test set is 8:2, and the local social network data of each participant terminal is the same distribution. The local social network data of each participant terminal are not independent and identically distributed, which is used to verify the effectiveness of the social network public opinion analysis method based on personalized federated learning.
[0026] The social network public opinion analysis method based on personalized federated learning includes the following steps: Step 1: The participant terminal preprocesses the local social network data. Data preprocessing can ensure data quality, reduce the impact of errors, and help ensure data availability and adaptability to improve the performance and interpretability of the model.
[0027] 1.1 Use linear interpolation method to process missing values in news article data set and movie review data set.
[0028] News article datasets and movie review datasets may contain errors, missing values, duplicates, or outliers, which can adversely affect the performance of the model. Linear interpolation is a computationally efficient method that is particularly suitable for large-scale datasets because it only requires basic linear calculations.
[0029] 1.2 Use the bag-of-words model to extract features from the data set after missing value processing and standardize the extracted features.
[0030] Feature extraction and normalization are key steps in data preprocessing, which are used to convert text features (such as vocabulary) in the AGNews and Movielens datasets into numerical features and ensure that these features are of the same order of magnitude so that the machine learning model can process them. For text data, the bag-of-words model can be used for feature extraction, and the text can be converted into numerical features through vocabulary construction, text vectorization, and sparse matrices; to ensure that the features of the data are of the same order of magnitude, standard normalization is used.
[0031] Step 2: The participant terminal trains the local model and uploads it to the server.
[0032] 2.1 After data preprocessing, the participant terminal uses local social network data to pre-train the model and generate a local model .
[0033] 2.2 According to the distribution of local social network data, the local model is divided into a weighted head layer and a full substitute body layer to generate a hierarchical weight matrix.
[0034] 2.3 When selected by the server, the local model Upload to the server.
[0035] Step 3: If Figure 3 As shown, the server receives the local model, generates a global model through the hypernetwork update mechanism, and sends it to the participant terminals.
[0036] 3.1 Server initialization parameters, receiving local models uploaded by each participant terminal , the local model Aggregation generates model matrix ,as follows:
[0037] in, Indicates The local model of the participant terminal, , represents the number of participant terminals, Representative The local model of the participant terminal Layer model parameters, , Indicates the number of layers of the participant terminal's local model.
[0038] Model Matrix It will be updated in real time based on the local model uploaded by the participant's terminal. < That is, there are participant terminals that do not participate in training or do not participate in training in time, that is, there are not enough local models, and the server will self-generate to complete the model matrix .
[0039] 3.2 The server selects a subset of participant terminals to participate in the training.
[0040] For each participant in the subset , the server uploads the local model , through the hypernetwork Generate a unique model weight matrix for it , the model weight matrix With the model matrix Similar, as follows:
[0041] in, Indicates The participant terminal model weight matrix, Indicates Participant Terminal No. The set of layer model weights; Indicates The participant terminal is about The model The weights of the layer parameters, , .
[0042] The main function of the model weight matrix is to find local models of participant terminals with similar data distribution for each participant terminal on the server, and give these models higher weights so as to learn more relevant information from them.
[0043] In a hypernetwork, is its own parameter, and its update method is:
[0044] 3.3 Server based on model matrix With the model weight matrix Generate global model Send to participant terminals , global model The generation method is as follows:
[0045] Step 4: The participant terminal receives the global model sent by the server, and updates the local model by dynamic hierarchical aggregation based on the global model, local model and hierarchical weight matrix.
[0046] 4.1 The participant terminal receives the global model sent by the server , based on the weighted head layer and the full stand-in body layer Figure 2The dynamic hierarchical aggregation update is shown to update the local model. The dynamic hierarchical aggregation update formula is:
[0047] in, Indicates Participant Terminal No. Round local model, Indicates that the server sends Participant Terminal No. The global model of the wheel, represents the dot product matrix, and are weight matrices, corresponding to the global model and the local model respectively, and Contains the weight values of each element in each layer, and the sum of the corresponding element weight values is 1. For the full stand-in body layer, let The corresponding weight value of is 1. The corresponding weight value of is 0. In order to facilitate learning under constraints, the above dynamic hierarchical aggregation update formula can be rewritten as:
[0048] in, Indicates The hierarchical weight matrix of the participant terminals, , Contains the weights of the weighted head layer and the full stand-in body layer. In each round of calculation, based on the previous round ,renew The weight value of the medium weight head layer, The specific update formula is as follows:
[0049] in, represents the learning rate of the weighted first layer, Indicates i The participant terminal is in It's a turn % of local social network data for calculation, Indicates the number of rounds, represents the cross entropy loss function.
[0050] 4.2 After the dynamic hierarchical aggregation update is completed, the participant terminal uses the updated local model and local social network data for training, and uploads the trained local model to the server.
[0051] Step 5: Repeat steps 3 to 4 until the maximum number of iterations is reached, output the final local model, and apply the final local model to perform social network public opinion analysis.
[0052] After completing the model training, each participant's terminal will call the trained final local model for analysis and detection when encountering relevant news or movie information, and provide relevant information to the user based on the analysis results output by the model for easy use.
[0053] Embodiment 2: Based on the social network public opinion analysis method based on personalized federated learning described in Example 1, this embodiment provides a social network public opinion analysis system based on personalized federated learning, including a server and a participant terminal, and the server is communicatively connected with a plurality of participant terminals; The server is configured to perform step a: receiving a local model uploaded by a participant terminal, and generating or updating a model matrix according to the local model; step b: generating a model weight matrix according to the local model and the hypernetwork; step c: generating a global model according to the model weight matrix and the model matrix, and sending the global model to the participant terminal to dynamically aggregate and update the local model; step d: looping steps a to c until the maximum number of iterations is reached; The participant terminal is configured to execute step A: use local social network data to train the model and generate a local model; step B: divide the local model into a weight head layer and a full substitute body layer according to the distribution of local social network data, and generate a hierarchical weight matrix; step C: upload the local model to the server to generate a global model; step D: receive the global model sent by the server, and update the local model according to the global model, the local model and the hierarchical weight matrix; step E: loop step C to step D until the maximum number of iterations is reached to obtain the final local model; step F: apply the final local model to perform social network public opinion analysis.
[0054] In summary, the social network public opinion analysis method and system based on personalized federated learning adopts dynamic hierarchical aggregation and super network update mechanism on the participant side and server side respectively, combines the participant side with the server side, jointly handles the data deviation problem caused by non-independent and identically distributed data, reduces the negative impact on model aggregation, and obtains a social network public opinion analysis model that adapts to the habits of each user; the super network update mechanism is used on the server side, which enables the server side to select models or feature extractors suitable for different data types, so as to better handle diverse social network data, and the use of super network can automatically select or adjust the appropriate model according to the characteristics of the current task to maximize the model performance; dynamic hierarchical aggregation is used on the participant side, which allows each participant to independently determine the number of layers of local model updates and decide when to pass its model parameters to the upper aggregation layer, which provides greater flexibility to adapt to the computing power and data distribution of different participants. Only the update and aggregation of model parameters need to be communicated between participants and servers, without the need to transmit a large amount of raw data, which reduces the burden of network transmission and makes federated learning more scalable.
[0055] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A social network public opinion analysis method based on personalized federated learning, characterized in that: The method is executed by a participant terminal, and includes: Step A: Use local social network data to train the model and generate a local model; Step B: Divide the local model into a weighted head layer and a full substitute body layer according to the distribution of local social network data, and generate a hierarchical weight matrix; Step C: Upload the local model to the server to generate a global model; Step D: Receive the global model sent by the server, and dynamically update the local model by hierarchical aggregation based on the global model, the local model, and the hierarchical weight matrix; Step E: looping steps C to D until the maximum number of iterations is reached to obtain the final local model; Step F: Apply the final local model to perform social network public opinion analysis.
2. The social network public opinion analysis method based on personalized federated learning according to claim 1 is characterized in that: Also includes: Preprocessing the local social network data before step A; Preprocessing includes: The missing values of local social network data are processed using linear interpolation method; The bag-of-words model is used to extract features from local social network data after missing value processing, and the extracted features are standardized.
3. The social network public opinion analysis method based on personalized federated learning according to claim 1 is characterized in that: The formula for dynamic hierarchical aggregation update is: ; in, Indicates The hierarchical weight matrix of the participant terminals, Indicates Participant Terminal No. Round local model, represents the dot product matrix, Indicates that the server sends Participant Terminal No. The global model of the wheel, Indicates the round number.
4. The social network public opinion analysis method based on personalized federated learning according to claim 3 is characterized in that: No. The hierarchical weight matrix of the participant terminals The update formula is: ; in, represents the learning rate of the weighted first layer, Indicates The participant terminal is in It's a turn % of local social network data for calculation, represents the cross entropy loss function.
5. A social network public opinion analysis method based on personalized federated learning, characterized in that: The method is executed by a server and includes: Step a: Receive the local model uploaded by the participant terminal, and generate or update the model matrix according to the local model; Step b: Generate a model weight matrix based on the local model and the hypernetwork; Step c: Generate a global model based on the model weight matrix and the model matrix, and send the global model to the participant terminals to dynamically aggregate and update the local model; Step d: Loop steps a to c until the maximum number of iterations is reached.
6. The social network public opinion analysis method based on personalized federated learning according to claim 5 is characterized in that: Receive the local model uploaded by the participant terminal, and generate or update the model matrix according to the local model, including: Receive local models uploaded by participant terminals , according to the local model Aggregation generates or updates the model matrix ; Among them, the model matrix for: ; In the formula, Indicates The local model of the participant terminal, , represents the number of participant terminals, Representative The local model of the participant terminal Layer model parameters, , Indicates the number of layers of the participant terminal's local model.
7. The social network public opinion analysis method based on personalized federated learning according to claim 6 is characterized in that: The model weight matrix is: ; in, Indicates The participant terminal model weight matrix, Indicates Participant Terminal No. The set of layer model weights; Indicates The participant terminal is about The model The weights of the layer parameters, , .
8. The social network public opinion analysis method based on personalized federated learning according to claim 7 is characterized in that: The generation formula of the global model is: ; in, Indicates that the corresponding A global model of the terminals of each participant.
9. A social network public opinion analysis system based on personalized federated learning, characterized in that: It includes a server and a participant terminal, wherein the server is communicatively connected with a number of participant terminals; The server is configured to perform step a: receiving a local model uploaded by a participant terminal, and generating or updating a model matrix according to the local model; step b: generating a model weight matrix according to the local model and the hypernetwork; step c: generating a global model according to the model weight matrix and the model matrix, and sending the global model to the participant terminal to dynamically aggregate and update the local model; step d: looping steps a to c until the maximum number of iterations is reached; The participant terminal is configured to perform step A: using local social network data to perform model training and generate a local model; step B: dividing the local model into a weighted head layer and a full substitute body layer according to the distribution of local social network data, and generating a hierarchical weight matrix; Step C: Upload the local model to the server to generate a global model; Step D: Receive the global model sent by the server, and dynamically update the local model based on the global model, the local model and the hierarchical weight matrix; Step E: looping step C to step D until the maximum number of iterations is reached to obtain the final local model; Step F: applying the final local model to perform social network public opinion analysis.