Keyword Spotting via Substring Pre-processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword searching methods face high computational complexity when searching for a large number of keywords in high-bandwidth data, limiting the number of keywords and traffic bandwidth that can be searched effectively in real-time.
Innovation Solution
The method represents keywords as a list of substrings, specifically n-bit prefixes, which are cached in internal processor memory to identify locations in the data where these substrings occur, reducing the volume of data to be searched and improving efficiency by focusing searches only on those locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional keyword searching methods are used to search for a large number of keywords in high-bandwidth data, then the search can be performed, but the computational complexity becomes excessively high and real-time processing is limited
Solution Approach 1:
The patent segments the keyword search process into two distinct phases: a pre-processing phase that creates a compressed representation (bit vector) of the keyword list, and a search phase that uses this compressed representation to quickly identify candidate locations. This segmentation allows the computationally intensive preprocessing to be performed once, enabling fast real-time searches thereafter.
Solution Approach 2:
The patent performs preliminary action by pre-processing the keyword list before the actual search operation. The bit vector representing the keyword list is constructed in advance and stored in cache memory, allowing the search process to skip the keyword list construction step and directly use the pre-compressed representation for rapid pattern matching.
2Reliability
If the entire input data is searched for all keywords, then complete keyword detection is achieved, but the search time and computational resources increase significantly
Solution Approach 1:
The patent extracts and utilizes only the essential information from the keyword list by creating a bit vector representation that captures the presence/absence of keywords at different positions. This extraction reduces the search space by focusing only on locations where keywords might appear, rather than searching the entire data stream with all keywords simultaneously.
Solution Approach 2:
The patent applies local quality by using the bit vector to identify specific locations in the input data where keyword matches are possible. Instead of uniformly searching all data with all keywords, the method focuses computational effort only on local regions identified as potential matches, thereby reducing overall search time while maintaining detection completeness.
3Adaptability or versatility
If a large number of keywords are searched simultaneously in real-time traffic, then comprehensive security monitoring is achieved, but the processing speed and bandwidth capacity are reduced
Solution Approach 1:
The patent changes the parameter representation of the keyword list from a traditional format to a compressed bit vector format. This parameter transformation allows a large number of keywords to be represented in a compact form that can be quickly processed, enabling the system to maintain high processing speeds even when monitoring for thousands of keywords simultaneously.
Data Source
Figure 1~2
Figure 3~4
AI summary
Methods and systems related to keyword searching processes. A list of keywords may be first represented by a set of short substrings. The substrings are selected such that an occurrence of a substring indicates a possible occurrence of one or more of the keywords. Input data may be initially pre-processed, so as to identify locations in the input data in which the substrings occur. Then, the identified locations are searched for occurrences of the actual keywords. The pre-processing scheme enables the keyword search process to search only in the identified locations of the substrings instead of over the entire input data.