Messaging Search System Using Category-Based Keyword Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current messaging systems in social networks face challenges in searching and retrieving relevant short messages due to their brevity, ambiguity, and high volume, leading to inefficient filtering and false positives, especially with techniques like TF-IDF weighting being ineffective for short documents.
Innovation Solution
A system that identifies relevant categories and keywords associated with them, performs multiple searches using the original query and scored keywords, and scores messages based on category relevance, excluding used keywords for fair scoring, while considering time elapsed since posting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If regular expression matching is used to search short messages, then search speed is improved, but search accuracy deteriorates due to ambiguous terms and false positives
Solution Approach 1:
The patent segments the search process into multiple stages: initial filtering using regular expressions for speed, followed by secondary filtering using TF-IDF scoring and category-based relevance assessment. This multi-stage segmentation allows the system to maintain high search speed while progressively eliminating false positives and improving accuracy.
Solution Approach 2:
The patent changes the parameter of message evaluation from simple keyword matching to a composite scoring system that incorporates TF-IDF weights, category relevance, and message recency. By transforming the search criterion from binary (match/no match) to a continuous relevance score, the system achieves both speed and accuracy.
2Measurement precision
If TF-IDF weighting is used for message search, then search accuracy is improved, but effectiveness deteriorates due to short document length
Solution Approach 1:
The patent performs preliminary categorization of messages into topic groups before applying TF-IDF weighting. By pre-organizing short messages into categories based on their content themes, the system creates a broader contextual framework that compensates for the brevity of individual messages, making TF-IDF scoring more reliable and effective.
Solution Approach 2:
The patent adds a categorical dimension to the search space, organizing messages not just by keywords but by topic categories. This dimensional transformation allows the system to leverage category-level patterns and relationships that are invisible at the individual message level, thereby improving the reliability of TF-IDF weighting for short documents.
3Measurement precision
If comprehensive search terms are used to avoid ambiguity, then search accuracy is improved, but message volume deteriorates due to omission of relevant messages
Solution Approach 1:
The patent applies partial matching by allowing searches to succeed with subsets of the full query terms rather than requiring exact matches of all terms. The system retrieves messages that contain relevant keywords even if they don't contain every term in the query, then ranks them by overall relevance. This approach recovers relevant messages that would be omitted by strict comprehensive matching while still maintaining accuracy through the scoring system.
4Measurement precision
If frequent term search is performed to increase relevance, then search accuracy is improved, but time period deteriorates due to shorter result window
Solution Approach 1:
The patent introduces dynamic temporal weighting that adjusts the time decay factor based on the frequency and recency of search terms. For frequently occurring terms, the system extends the effective search window by reducing the time decay rate, allowing relevant older messages to remain in the results. This dynamic adjustment maintains search accuracy while expanding the usable time period for frequent terms.
Data Source
AI summary
Systems and methods of identifying and retrieving messages that satisfy a search query using the context of the message and term frequencies are provided. One embodiment includes identifying at least one category relevant to the search query, wherein a plurality of scored keywords are associated with each category, selecting at least one of the scored keywords that is relevant to an identified category, performing a plurality of searches of messages from a social networking messaging service to retrieve messages, where at least one search includes retrieving messages based on the original search query and one of the selected scored keywords, scoring the retrieved messages with respect to each of the at least one identified categories using at least the scored keywords relevant to each category, and returning at least the message with the highest score as the search result.