Keyword Filtering for Social Network Message Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Social networking systems face challenges in accurately associating keywords with messages due to noisy keyword data from external webpages, as many webpages lack reliable keywords, and existing methods struggle to predict relevant keywords for messages without explicit links.
Innovation Solution
The system filters keywords and webpages to generate relevant models by excluding blacklisted domains, keywords with low domain usage, and those appearing on excessive numbers of webpages, then trains computer models using remaining data to predict keyword associations with messages, even if no explicit link is present.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the system uses webpage keywords provided by operators for message association, then keyword data is available for analysis, but the keyword data becomes noisy and unreliable
Solution Approach 1:
The system performs preliminary filtering of webpage keywords before using them for message association. The filtering process excludes keywords from blacklisted domains, removes keywords that appear on excessive numbers of webpages, and eliminates keywords with low domain usage. This preliminary action ensures that only high-quality, reliable keywords are retained for training computer models and associating with messages, thereby resolving the contradiction between keyword availability and reliability.
2Adaptability or versatility
If the system trains computer models for all webpage keywords, then keyword prediction coverage is maximized, but the training process becomes computationally expensive and time-consuming
Solution Approach 1:
The system extracts and removes low-quality keywords from the training dataset through filtering operations. By taking out keywords from blacklisted domains, keywords with insufficient domain usage, and keywords appearing on too many webpages, the system reduces the total number of keywords requiring model training. This extraction process maintains coverage of meaningful keywords while significantly reducing computational expense and training time.
3Quantity of substance
If the system includes all keywords from webpages in the training data, then the training dataset is comprehensive, but the model learns from noisy and irrelevant keywords
Solution Approach 1:
The system applies preliminary filtering to the training data before model training. The filtering process removes keywords from blacklisted domains, eliminates keywords that appear on excessive numbers of webpages, and excludes keywords with low domain usage. This preliminary action purifies the training dataset, ensuring that the model learns from high-quality, relevant keywords rather than noisy and irrelevant ones, thereby improving keyword association accuracy while maintaining sufficient training data volume.
Data Source
AI summary
A social networking system receives messages from users that include links to webpages that designate keywords of the webpage. The social networking system identifies webpages linked by users to generate computer models that predict whether a webpage or message should be associated with particular keywords. The social networking system generates computer models that are trained on example webpages and related keywords linked by users in messages. Prior to generating computer models, the social networking system applies one or more filters to exclude webpages and keywords from consideration. The filters may exclude webpages that have low-reliability, are associated with an excessive number of keywords, or keywords that appear on an insufficient number of domains. After training the computer models, messages composed by users may be analyzed and a keyword predicted for the message, which may be suggested to the user to categorize the message.