Probabilistic Search System for Natural Language Relevance Estimation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional search engines are limited in handling large volumes of natural language text, require users to distill queries into keywords, and may incorrectly discard relevant content if index terms do not match user-inputted keywords.

Innovation Solution

A probabilistically-motivated search system that uses statistical models trained on diverse data sources to estimate the relevance of content based on text input, incorporating feature extraction and relevance mechanisms to provide content suggestions without manual keyword selection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If traditional search engines use keyword matching with index terms, then search speed is improved, but search accuracy deteriorates when index terms do not match user keywords

Engineering Contradiction:

Improvesearch speedVSAvoidsearch accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent replaces the mechanical keyword-matching system with a probabilistic relevance estimation system using statistical models. Instead of deterministic matching, the system computes likelihood scores based on multiple evidence sources including text input, user profiles, and content metadata, thereby improving accuracy without sacrificing speed through efficient probabilistic computation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system changes the parameter of relevance determination from binary match/non-match to a continuous probability score. By using statistical models to estimate likelihood of relevance rather than simple keyword presence, the system achieves more nuanced and accurate search results while maintaining computational efficiency through optimized probability calculations.

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If traditional search engines require users to distill queries into keywords, then device complexity is reduced, but ease of operation deteriorates

Engineering Contradiction:

Improvesystem complexityVSAvoidease of use

Core Design Contradiction:

Device complexityVSEase of operation

Solution Approach 1:

The system enables self-service by automatically performing the task of query distillation. The statistical model processes natural language input and autonomously extracts relevant features and evidence, eliminating the need for users to manually select keywords. This improves ease of operation while the automated feature extraction keeps system complexity manageable.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system performs preliminary processing of natural language input by automatically extracting features and evidence before the main search execution. This preliminary action includes tokenization, feature extraction, and evidence gathering, which simplifies the user interface while the automated processing pipeline manages the complexity of handling unstructured natural language.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If traditional search engines return only specific documents, then productivity is improved through focused results, but adaptability deteriorates when relevant content exists beyond indexed documents

Engineering Contradiction:

Improvesearch efficiencyVSAvoidcontent coverage

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The system achieves universality by returning multiple types of content suggestions including documents, entities, and other relevant information objects. The statistical model can estimate relevance for different content types based on the same probabilistic framework, allowing the system to adapt to diverse content formats while maintaining efficient ranked results that improve user productivity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system dynamically adapts the type and format of results based on the input query and statistical relevance estimation. Rather than returning fixed document types, the system can return documents, entities, or other content forms depending on what the probabilistic model determines to be most relevant, thereby improving both adaptability and user efficiency.

Inventive Principle:

Principle #15Dynamics

4Measurement precision

If traditional search engines perform exact keyword matching, then measurement precision is improved for matched terms, but loss of information increases when relevant content uses different terminology

Engineering Contradiction:

Improvematching precisionVSAvoidrelevant content loss

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The system adds another dimension to the matching process by incorporating multiple evidence sources beyond just text keywords. The statistical model considers user profiles, content metadata, and contextual information as additional dimensions, allowing relevant content to be identified even when terminology differs, thereby reducing information loss while maintaining precision through multi-dimensional relevance estimation.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS11200503B2Search system and corresponding method

Publication Date: 2021.12.14 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

There is provided a search system comprising a statistical model trained on text associated with a piece of content. The text associated with the piece of content is drawn from a plurality of different data sources. The system is configured to receive text input and generate using the statistical model an estimate of the likelihood that the piece of content is relevant given the text input. A corresponding method is also provided.