Search Apparatus Using Frequency-Based Document Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing search systems face challenges in efficiently processing large-scale document sets due to memory constraints and reduced search speed, especially when comparing query-specific document distributions with entire document sets.

Innovation Solution

A search apparatus and method that divides a document set into regions, calculates word frequencies within these regions, and identifies characteristic words by narrowing down the data to be read during search, using an abstract matrix storage unit, region upper limit calculation unit, word frequency calculation unit, and document frequency reference unit.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If all document words are stored in memory for high-speed search, then search speed is improved, but memory capacity requirements increase significantly

Engineering Contradiction:
Improvesearch speedVSAvoidmemory capacity
Core Design Contradiction:
SpeedVSQuantity of substance

Solution Approach 1:

The patent divides the large document set into multiple regions based on document frequencies. The word document matrix is segmented into multiple sub-matrices, each representing a specific frequency range. This allows the system to process and store data in smaller, manageable chunks rather than requiring all data to be loaded into memory simultaneously, thus reducing memory capacity requirements while maintaining search efficiency.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent pre-calculates and stores frequency information in an abstract matrix before the actual search operation. By performing frequency calculations in advance and organizing data by frequency ranges, the system prepares the data structure optimally for quick access during search operations, eliminating the need to load and process entire document sets during searching.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If the entire document set is processed to ensure accuracy, then measurement precision is improved, but processing time increases

Engineering Contradiction:
Improvekeyword frequency accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the document set into frequency-based regions and processes only the relevant regions for each search query. By dividing the large document set into smaller frequency ranges and using region upper limit calculations, the system can determine which regions need to be examined, thereby reducing processing time while maintaining accurate frequency measurements for the queried keywords.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements a region upper limit calculation that determines the minimum necessary processing scope. Instead of processing the entire document set, the system calculates and processes only up to the upper limit of relevant frequency regions, achieving sufficient accuracy for search purposes without the excessive time cost of complete processing.

Inventive Principle:
Principle #16Partial or excessive action

3Productivity

If document data is read sequentially to improve access speed, then data access efficiency is improved, but flexibility in processing different query types is reduced

Engineering Contradiction:
Improvedata access efficiencyVSAvoidquery processing flexibility
Core Design Contradiction:
ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent introduces a frequency-based dimensional organization to the data structure. By arranging document words in frequency-ordered regions rather than traditional sequential or alphabetical ordering, the system enables efficient sequential access within frequency ranges while also allowing flexible navigation across different frequency dimensions. This multi-dimensional organization supports both fast access and versatile query processing.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS8892574B2Search apparatus, search method, and non-transitory computer readable medium storing program that input a query representing a subset of a document set stored to a document database and output a keyword that often appears in the subset
Publication Date: 2014.11.18 NEC CORP
  • US8892574B2 patent drawing
  • US8892574B2 patent drawing
  • US8892574B2 patent drawing

AI summary

Provided is a search apparatus, a search method, and a program that can improve search speed for a document set even when an object to be searched is a large-scale document set. A search apparatus, in an embodiment, includes an abstract matrix storage unit, a word frequency calculation unit, and a document frequency reference unit.