Finite-State Machine for Sensitive Text Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting sensitive texts in network literature are inefficient due to the need for multiple scans of a text against a keyword list, leading to high time costs and low detection efficiency.
Innovation Solution
A sensitive text detecting method using a finite-state machine to acquire a feature text string, calculate keyword category weights based on frequency and preset weights, and determine sensitivity by scanning the text only once, improving detection speed and precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the text is scanned throughout multiple times according to each keyword in the keyword list, then the frequency of occurrence of each keyword can be obtained, but the time cost increases significantly and detection efficiency decreases
Solution Approach 1:
The patent merges multiple keyword matching operations into a single text scan by constructing an AC automaton that integrates all keywords from the keyword list into one unified state machine. This allows simultaneous detection of all keywords during one pass through the text, resolving the contradiction between detection accuracy and time consumption.
Solution Approach 2:
The patent performs preliminary action by pre-processing the keyword list into an AC automaton structure before text detection. The automaton is constructed in advance with all keyword patterns embedded, so that during actual text detection, no additional keyword-by-keyword scanning is needed, thus reducing detection time while maintaining accuracy.
2Loss of information
If multiple scans are performed for each keyword, then comprehensive keyword frequency statistics can be collected, but the detection process becomes complex and resource-intensive
Solution Approach 1:
The patent combines multiple separate keyword matching processes into a single integrated AC automaton detection process. The automaton maintains state information for all keywords simultaneously, allowing comprehensive frequency statistics to be collected in one scan rather than through multiple separate scans, thus reducing process complexity.
Solution Approach 2:
The AC automaton serves as an intermediary data structure that mediates between the keyword list and the text detection process. It pre-organizes all keyword patterns into a unified state machine structure, simplifying the detection process by replacing complex multiple-scan logic with a single automated traversal through the automaton states.
Data Source
AI summary
A feature text string of a currently detected text is acquired, and the feature text string is detected according to a finite-state machine established in advance, so as to obtain frequency of occurrence of each keyword in the feature text string. A weight of the keyword category in the text is calculated for each keyword category of multiple keyword categories based on the frequency of occurrence of each keyword corresponding to the keyword category and a preset weight of each keyword. The text is determined to be sensitive when the weight of at least one keyword category is greater than a preset threshold.