Search Query Filtering to Reduce Overblocking of Risky Terms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing OS search technologies inadvertently return offensive or inappropriate search results due to the nature of machine learning models, leading to overclassification and restriction of useful queries.
Innovation Solution
An approach that analyzes user-provided queries to identify potentially inappropriate terms, constructs a separate query, and filters out results associated with these terms using embedding vectors and block lists, ensuring safer and more relevant search results are returned.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a block list is used to detect and limit queries with risky terms, then protection from offensive search results is improved, but the query reject rate increases and useful queries are unnecessarily blocked
Solution Approach 1:
The patent segments the search result set into two categories: results from the original user query and results from a constructed query containing only risky terms. By separately executing these queries and comparing results, the system can identify and remove only the problematic subset without blocking the entire original query, thus reducing unnecessary rejections while maintaining protection.
Solution Approach 2:
The constructed query acts as an intermediary mechanism to indirectly identify problematic results. Instead of directly blocking queries containing risky terms (which causes overclassification), the system creates a separate query with only those terms, executes it, and uses its results as a filter to remove problematic items from the original search results.
2Adaptability or versatility
If machine learning models are used to expand search capabilities, then search versatility is improved, but the risk of returning inappropriate results increases
Solution Approach 1:
The patent converts the harmful effect of machine learning models returning inappropriate results into a benefit by using the same models to generate embeddings for both the original query and the constructed query. The risk of inappropriate results is transformed into a mechanism for identifying and filtering them, allowing the system to leverage ML capabilities while mitigating their drawbacks.
Solution Approach 2:
The system performs preliminary identification of risky terms in the user query and constructs a separate query with those terms before executing the main search. This preliminary action allows the system to preemptively identify potential problematic results by comparing the constructed query results against the original results, filtering out inappropriate content before it reaches the user.
Data Source
AI summary
The disclosed techniques improve search results by reducing the rate at which queries are rejected for potentially yielding offensive, grossly inaccurate, or otherwise inappropriate search results. This enables a broader set of useful search results to be returned to the user. In some configurations, the user-provided query is analyzed to identify terms that could yield an inappropriate search result. A query is constructed using the identified terms. The user-provided query and the constructed query are performed independently, yielding two sets of results. Results from the constructed query are removed from the user-provided query, allowing safer and more relevant results to be returned to the user.