Two-Phase Results Summarization for Search Latency Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing search engine systems face latency issues due to sequential operations in generating summaries for user queries, where summarization cannot begin until all results are retrieved, affecting the overall time to serve responses.

Innovation Solution

The architecture splits the generation of results summaries into two phases: a preliminary phase for quick summary generation and a second phase that runs in parallel with other modules, allowing for additional contextual information to improve summary quality, with the summaries from both phases being merged for final output.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If a single summarization phase is used, then the summary generation is simple and fast, but the overall latency increases because summarization cannot begin until all results are retrieved

Engineering Contradiction:

Improveoverall latencyVSAvoidsummarization architecture complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The summarization process is divided into two distinct phases: a first summarization phase that generates preliminary summaries quickly, and a second summarization phase that generates more comprehensive summaries with additional contextual information. This segmentation allows the system to provide early results while maintaining the option to improve quality later, thereby reducing overall latency without requiring complete summarization before result delivery.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The first summarization phase performs preliminary summarization on retrieved results before all results are fully processed. This preliminary action enables the system to generate and return initial summaries to users while the second summarization phase continues to work on more comprehensive versions, effectively overlapping operations to reduce waiting time.

Inventive Principle:

Principle #10Preliminary action

2Speed

If a first summarization phase is implemented for quick summary generation, then latency is reduced, but the summary content may be less comprehensive

Engineering Contradiction:

Improvesummary generation speedVSAvoidsummary content completeness

Core Design Contradiction:

SpeedVSLoss of information

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system merges the outputs of the first summarization phase and the second summarization phase to produce final summaries. This merging combines the speed advantage of the first phase with the comprehensiveness of the second phase, ensuring that neither speed nor content completeness is sacrificed.

Inventive Principle:

Principle #5Merging (Combining)

3Loss of time

If the second summarization phase is run in parallel with other modules, then latency is offset, but processing resources increase

Engineering Contradiction:

Improveoverall response timeVSAvoidprocessing resource utilization

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The first summarization phase performs a partial summarization that is sufficient for immediate use but not exhaustive. This partial action reduces the computational burden compared to complete summarization, allowing the system to allocate resources efficiently while still providing useful results within acceptable timeframes.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

Inventive Principle:

Principle #10Preliminary action

4Productivity

If summaries are generated sequentially, then resource utilization is efficient, but the overall time to serve responses increases

Engineering Contradiction:

Improveresponse serving efficiencyVSAvoidtime to serve response

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system maintains continuous useful action by overlapping the first and second summarization phases with other processing modules. Instead of waiting for complete summarization before returning results, the system continuously provides value through preliminary summaries while background processes continue to refine the content, maximizing productivity without sacrificing quality.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS10095783B2Multiple rounds of results summarization for improved latency and relevance

Publication Date: 2018.10.09 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Architecture that splits the generation of results summaries into at least two phases for reduced latency and improved relevance. A first phase generates a summary quickly and thereby enables subsequent modules to begin processing. A second round then executes in parallel to these other modules to offset the latency. The second round can also provide additional contextual information to the summarization module to improve the quality (relevance) of the summaries. Ultimately, the summaries generated in the two phases are merged. The multiple rounds of summarization enable the first round to be cheaper in processing resources to save overall latency, whereas the second phase can be more expensive, since the second phase can be run in parallel with other modules. Additionally, contextual information can be used to build the full content summaries in the second round.