Search Query Filtering to Reduce Offensive Result Rejection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search technologies, particularly those utilizing machine learning, often inadvertently return offensive or inappropriate results due to the nature of block lists, which can be overly restrictive and prevent useful search queries.
Innovation Solution
A method that analyzes user queries to identify potential inappropriate terms, constructs a separate query, and filters out results associated with these terms, using embedding vectors to enhance search relevance and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a block list is used to detect and limit queries with risky terms, then protection from offensive search results is improved, but query rejection rate increases and useful search results are lost
Solution Approach 1:
The patent segments the search result filtering process into multiple independent components: (1) block list-based term detection, (2) embedding-based semantic similarity filtering, and (3) query result analysis. This segmentation allows each component to operate independently with different sensitivity levels, preventing over-rejection while maintaining protection. The embedding-based filter processes queries separately from the block list, enabling nuanced handling of potentially problematic terms.
Solution Approach 2:
The patent introduces embedding vectors as an intermediary layer between the block list and the final search results. Instead of directly blocking queries containing sensitive terms, the system uses embeddings to compute semantic similarity scores and filter results based on contextual relevance. This intermediary approach allows useful results to pass through while blocking only genuinely offensive content, reducing false positives.
2Object-affected harmful factors
If block lists limit queries with risky terms, then offensive search results are prevented, but overly restrictive filtering occurs
Solution Approach 1:
The patent changes the filtering parameters from binary block list matching to continuous embedding-based similarity scoring. Instead of treating all occurrences of sensitive terms equally, the system computes similarity scores and applies threshold-based filtering. This allows the system to adapt to different contexts and maintain search flexibility while preventing offensive results. The parameter change enables nuanced differentiation between harmful and benign uses of sensitive terms.
3Productivity
If machine learning models are used for search, then search capability is improved, but inappropriate results are inadvertently returned
Solution Approach 1:
The patent implements a feedback mechanism where search results are analyzed using the same embedding model to detect potential inappropriate content. The system generates embeddings for search results and compares them against embeddings of sensitive terms, creating a feedback loop that identifies and filters problematic results. This feedback approach allows the system to maintain high search capability while automatically detecting and removing inappropriate results that may have slipped through initial filtering.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed techniques improve search results by reducing the rate at which queries are rejected for potentially yielding offensive, grossly inaccurate, or otherwise inappropriate search results. This enables a broader set of useful search results to be returned to the user. In some configurations, the user-provided query is analyzed to identify terms that could yield an inappropriate search result. A query is constructed using the identified terms. The user-provided query and the constructed query are performed independently, yielding two sets of results. Results from the constructed query are removed from the user-provided query, allowing safer and more relevant results to be returned to the user.