Keyword identification method for streaming data
By constructing a combined keyword rule lexicon and the Aho-Corasick automaton algorithm, the problems of keyword recognition delay and misjudgment in streaming output in traditional methods are solved, realizing real-time, hierarchical sensitive word filtering, which is suitable for streaming output scenarios of large language models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU QUALITY ASSURANCE TECH CO LTD
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional sensitive word filtering methods cannot perform real-time and accurate keyword judgment in the streaming output scenario of large language models, resulting in response delays, misjudgments, and the inability to process keywords in a tiered manner.
By constructing a rule-based vocabulary of combined keywords, extracting independent keyword indexes, performing word-by-word preprocessing of streaming text, and employing the Aho-Corasick automaton algorithm for streaming matching and backtracking verification, real-time, hierarchical keyword recognition is achieved.
It achieves real-time and accurate keyword matching during streaming generation, reduces detection latency and false positive rate, supports hierarchical processing and fine-grained management of text content, and adapts to rapid updates in content security requirements.
Smart Images
Figure CN122045397A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of large language models, and more particularly to a keyword recognition method for streaming data. Background Technology
[0002] With the widespread application of Large Language Models (LLMs), it has become crucial to conduct security audits of the content they generate to filter inappropriate information. Traditional sensitive word filtering methods mainly rely on static keyword matching, which directly compares the text input by the user or output by the system with a predefined list of sensitive words. If a word is found to be a complete match, it is blocked or replaced.
[0003] However, this traditional approach faces significant challenges in streaming output scenarios using large language models: 1. Large language models generate text token by token. Traditional keyword matching requires waiting for complete words or sentences to appear, making real-time, incremental judgment impossible during streaming generation, leading to response delays or missed detections; 2. The sensitivity of many words is highly context-dependent. For example, "fire a gun" might be neutral in a military discussion but sensitive in a context of violent incitement. Traditional methods cannot distinguish between these, easily leading to misjudgments or missed detections; 3. Traditional methods typically employ a "one-size-fits-all" blocking strategy, failing to classify and categorize sensitive information based on its severity. In traditional dialogue applications with fewer words, direct blocking has little impact. However, when large models generate long texts, simple global blocking severely affects text coherence and usability.
[0004] Therefore, there is an urgent need for a keyword information recognition technology that can adapt to the characteristics of streaming output, make accurate judgments based on contextual semantics, and support hierarchical processing. Summary of the Invention
[0005] To overcome the above-mentioned shortcomings, the present invention aims to provide an efficient, accurate, and tiered keyword recognition method for streaming data. This method processes sensitive information from the streaming output of a large language model by first tracking independent keywords and then backtracking for matching.
[0006] This invention achieves the above objective through the following scheme: a keyword recognition method for streaming data, comprising:
[0007] (1) Construct a rule-based thesaurus of combined keywords;
[0008] (2) Extract and construct an independent keyword index;
[0009] (3) Preprocess the streaming text character by character and output the standard character stream output text;
[0010] (4) Perform streaming matching and backtracking verification on the standard character stream output text;
[0011] (5) Implement a hierarchical response strategy based on the combined word tags corresponding to the matched combined keyword rules.
[0012] Preferably, the keyword combination rule list is defined by the keyword combination rules; the keyword combination rules include, but are not limited to: keyword combination, context window distance, and combination word tags.
[0013] Preferably, the keyword combination is a combination of words necessary to define a sensitive context.
[0014] Preferably, the context window distance indicates that all words in the above keyword combination must appear simultaneously within a certain distance of a continuous text segment for the rule to be triggered.
[0015] Preferably, the combined word tag can be a content sensitivity level tag, used to define the sensitivity severity level corresponding to the rule; or it can be a label for keyword type.
[0016] Preferably, step (2) specifically includes the following steps:
[0017] (2.1) Traverse the keyword combinations of each rule;
[0018] (2.2) Select each word in each keyword combination as an independent keyword;
[0019] (2.3) Remove duplicate words from independent keywords.
[0020] Preferably, step (2) needs to be performed once after each update of the combined keyword rule vocabulary to generate a new independent keyword index.
[0021] Preferably, step (3) specifically includes the following steps:
[0022] (3.1) Obtain a raw character from the upstream streaming interface;
[0023] (3.2) Determine the current character type: If the current character type is Chinese, proceed to step (3.3); if the current character type is English or number, proceed to step (3.4); if the current character type is space, proceed to step (3.5); if the current character type is punctuation or other non-text characters, proceed to step (3.6).
[0024] (3.3) Append the current Chinese character according to the predecessor character type and update the predecessor character type, then execute step (3.7);
[0025] (3.4) Append the current English or numeric character according to the preceding character type, update the preceding character type, and execute step (3.7).
[0026] (3.5) Append the current space character according to the type of the preceding character, update the type of the preceding character, and execute step (3.7);
[0027] (3.6) Ignore directly, do not perform any append operation, the previous character type status remains unchanged, and execute step (3.7).
[0028] (3.7) Determine if the original character stream has ended: If not, repeat step (3.1); if yes, end the current normalized text stream and output the standard character stream output text.
[0029] Preferably, step (3.3) specifically includes the following steps:
[0030] (3.3.1) If the current character type is Chinese, determine whether the preceding character type is English or number: if not, proceed to step (3.3.2); if yes, proceed to step (3.3.3).
[0031] (3.3.2) Directly append the Chinese character to the normalized buffer and execute step (3.3.4);
[0032] (3.3.3) If the preceding character is of English or number type, first append a space character to the normalized buffer, and then append the current Chinese character;
[0033] (3.3.4) Update the predecessor character type to Chinese and execute step (3.7).
[0034] Preferably, step (3.4) specifically includes the following steps:
[0035] (3.4.1) If the current character type is English or number, determine if the preceding character type is Chinese: if yes, proceed to step (3.4.2); if no, proceed to step (3.4.3).
[0036] (3.4.2) If the preceding character type is Chinese, first append a space character to the normalized buffer, then append the current English or numeric character, and execute step (3.4.4).
[0037] (3.4.3) If the preceding character is of type English, number, or space, the character is directly appended to the normalized buffer;
[0038] (3.4.4) Update the preceding character type to alphanumeric and proceed to step (3.7).
[0039] Preferably, step (3.5) specifically includes the following steps:
[0040] (3.5.1) If the current character type is a space, determine whether the preceding character type is English or number: if not, proceed to step (3.5.2); if yes, proceed to step (3.5.3).
[0041] (3.5.2) If the preceding character type is Chinese or space, ignore the space character, keep the preceding character type unchanged, and proceed to step (3.7).
[0042] (3.5.3) If the preceding character type is alphanumeric, then this space character is appended to the normalized buffer, and the preceding character type is updated to space. Then proceed to step (3.7).
[0043] Preferably, the preceding character type is a state variable used to identify the type of the previously processed character.
[0044] Preferably, step (4) specifically includes the following steps:
[0045] (4.1) Obtain the position of the next node based on the state transition of the automaton;
[0046] (4.2) Determine whether the node has already matched an independent keyword. If it has, proceed to step (4.3); otherwise, proceed to step (4.1).
[0047] (4.3) Update the status element "Matched independent keyword and position" in the status table;
[0048] (4.4) Obtain the rules for combining keywords corresponding to the independent keyword;
[0049] (4.5) For each corresponding keyword combination rule, determine whether the output text has been hit by the context window distance. If yes, execute step (4.6); otherwise, loop through step (4.1).
[0050] (4.6) Record the hit results.
[0051] Preferably, a dynamic state table needs to be created and maintained for each streaming match and backtracking verification. The state table contains state elements that correspond one-to-one with the character positions in the standard character stream output text. The state elements include: the current automaton node, the matched individual keywords and their positions, and the list of matched combined keywords.
[0052] Preferably, in step (4.5), for each combination keyword rule containing a matching independent keyword, it is found whether other keywords in the combination have been matched and are within the context window distance. The search method can be achieved by viewing the current status table. Specifically, the matched independent keywords and positions are confirmed by searching the status element "Matched independent keywords and positions".
[0053] Preferably, the status table is updated when the standard character stream outputs text and a character is normalized.
[0054] The beneficial effects of this invention are as follows: 1. By first tracking independent keywords and then backtracking for matching, this invention achieves real-time matching of independent keywords during the text streaming generation process, without waiting for complete sentences or paragraphs, significantly reducing detection latency; 2. By determining the context window distance through the superposition distance of combined keywords, the judgment of independent keywords is elevated from isolated words to related context fragments, effectively reducing the misjudgment rate caused by missing context; 3. By assigning different sensitivity level labels (such as level 1, level 2, and level 3) to different keyword combinations, the system can adopt differentiated processing strategies according to the hit level, achieving refined and intelligent management of output content; 4. The modular word list design allows for convenient updating and expansion of keyword rules, enabling the system to quickly adapt to new content security requirements; 5. Furthermore, by using keyword recognition to label text, this invention can also accurately achieve text content classification management, expanding the application scenarios of this invention. Attached Figure Description
[0055] Figure 1 This is a flowchart illustrating the steps of a keyword recognition method for streaming data provided in an embodiment of the present invention;
[0056] Figure 2 This is a schematic diagram of the steps of the streaming text word-by-word preprocessing provided in an embodiment of the present invention;
[0057] Figure 3 This is a schematic diagram of the steps for performing streaming matching and backtracking verification on standard character stream output text provided in an embodiment of the present invention. Detailed Implementation
[0058] The present invention will be further described below with reference to specific implementation examples, but the scope of protection of the present invention is not limited thereto:
[0059] Example: Figure 1 As shown, a keyword recognition method for streaming data includes:
[0060] (1) Construct a rule list of combined keywords.
[0061] Unlike traditional simple word lists, this thesaurus is defined by rule combinations of keywords and stored in an enumerable data structure. The thesaurus can be stored as a database table, JSON, or other formats.
[0062] All keyword combination rules are loaded into memory during system initialization. In one specific embodiment, each rule needs to include, but is not limited to, the following elements: keyword combination, context window distance, and combined word tag.
[0063] The keyword combination is the lexical combination necessary to define a sensitive context. It can be a combination of two words, three words, or more.
[0064] For example, in an embodiment describing a violent scenario, a rule may contain a combination of two words, such as ["shoot", "kill"]; in a more complex scenario embodiment, a rule may contain a combination of three words, such as ["issue on behalf", "value-added tax", "invoice"].
[0065] The context window distance refers to the distance within which all words in the keyword combination must appear simultaneously in a continuous text segment for the rule to be triggered. This distance is expressed as a positive integer, expressed in Chinese characters or English letters. Taking the violence scenario example, the keyword combination in the rule is ["fire", "kill"]. The context window distance is 10, meaning that the words "fire" and "kill" must appear together within a 10-character window to trigger this keyword combination rule.
[0066] The combined word tags can be content sensitivity level tags, used to define the severity level of sensitivity corresponding to the rule. These tags can be represented by numbers, with smaller numbers indicating higher levels. For example, "Level 1" represents the highest risk, and "Level 3" represents a lower risk. Different levels will trigger different subsequent processing strategies.
[0067] Furthermore, the combined word tags can be labels for keyword types, such as tags for sensitive words like "violence," "pornography," and "politics," to classify texts containing sensitive words. Alternatively, depending on the choice of combined keywords, they can correspond to other neutral tags, such as "literature," "medicine," and "digital," to annotate the text content and prepare data annotations for subsequent text analysis.
[0068] (2) Extract and construct an independent keyword index.
[0069] To improve streaming matching efficiency, all independent keywords need to be extracted from the complete keyword combination rule vocabulary. The extraction strategy is to iterate through the keyword combinations of each rule, selecting each word in each keyword combination as an independent keyword, removing duplicate words. This step needs to be performed once after each vocabulary update to generate an independent keyword index.
[0070] In one embodiment, the keyword rule thesaurus contains keyword combinations ["invoice", "value-added tax"] and ["invoice", "individual income tax", "issuance on behalf"]. Five independent keywords ["invoice", "value-added tax", "invoice", "individual income tax", "issuance on behalf"] are extracted, among which "invoice" is a repeated independent keyword, so one is deleted. Finally, the four words ["invoice", "value-added tax", "individual income tax", "issuance on behalf"] are retained as independent keywords and an index is generated. This index table will be used for subsequent efficient preliminary scanning.
[0071] (3) Preprocess the streaming text character by character and output the standard character stream output text.
[0072] The large language model outputs a raw character stream as a token stream. A token may be decoded into multiple characters, such as a Chinese character, a Chinese word, or an English word. To reduce segmentation errors caused by character encoding, mixed Chinese and English characters, or punctuation, which could lead to missed keyword detection, standardization preprocessing is required during character-by-character processing.
[0073] The goal of this step is to generate a standard character stream sequence consisting only of Chinese characters, English characters, numbers, and spaces. The rules are as follows: all Chinese characters are tightly connected without spaces in between; Chinese characters are separated from English characters, and Chinese characters are separated from numbers by a single space; consecutive English letters and numbers are treated as a single English word unit, with different words separated by a single space; all punctuation marks (such as , . ! ? ; : " ' ' ' ) and other non-text characters are removed.
[0074] The system maintains a normalized buffer and a predecessor character type state variable. The normalized buffer is used to temporarily store the preprocessed standard character stream output text; the predecessor character type is a state variable used to identify the type of the previously processed character.
[0075] like Figure 2 As shown, the specific steps include:
[0076] (3.1) Obtain a raw character from the upstream streaming interface;
[0077] (3.2) Determine the current character type: If the current character type is Chinese, then execute step (3.3.1); if the current character type is English or number, then execute step (3.4.1); if the current character type is space, then execute step (3.5.1); if the current character type is punctuation or other non-text characters, then execute step (3.6).
[0078] (3.3.1) If the current character type is Chinese, determine whether the preceding character type is English or number: if not, proceed to step (3.3.2); if yes, proceed to step (3.3.3).
[0079] (3.3.2) If the preceding character is Chinese or a space, directly append the Chinese character to the normalized buffer and execute step (3.3.4).
[0080] (3.3.3) If the preceding character is of English or number type, first append a space character to the normalized buffer, and then append the current Chinese character;
[0081] (3.3.4) Update the predecessor character type to Chinese and execute step (3.7);
[0082] (3.4.1) If the current character type is English or number, determine if the preceding character type is Chinese: if yes, proceed to step (3.4.2); if no, proceed to step (3.4.3).
[0083] (3.4.2) If the preceding character type is Chinese, first append a space character to the normalized buffer, then append the current English or numeric character, and execute step (3.4.4).
[0084] (3.4.3) If the preceding character is of type alphanumeric or space, the current character is directly appended to the normalized buffer;
[0085] (3.4.4) Update the preceding character type to alphanumeric and proceed to step (3.7).
[0086] (3.5.1) If the current character type is a space, determine whether the preceding character type is English or number: if not, proceed to step (3.5.2); if yes, proceed to step (3.5.3).
[0087] (3.5.2) If the preceding character type is Chinese or space, ignore the current space character, keep the preceding character type unchanged, and proceed to step (3.7).
[0088] (3.5.3) If the preceding character type is alphanumeric, then append the current space character to the normalized buffer, update the preceding character type to space, and execute step (3.7).
[0089] (3.6) If the current character type is a punctuation mark or other non-target character, ignore it directly and do not perform any append operation. The previous character type status remains unchanged, and proceed to step (3.7).
[0090] (3.7) Determine if the original character stream has ended: If not, repeat step (3.1); if yes, end the current normalized text stream and output the standard character stream output text.
[0091] By performing real-time character-by-character processing on the original character stream through the above steps, the resulting continuous standard character stream sequence in the buffer is the output text of the standardized character stream that meets the requirements for subsequent keyword matching and has a uniform format.
[0092] (4) Perform streaming matching and backtracking verification on the standard character stream output text.
[0093] For standard character stream output text, keyword matching and backtracking verification are required. Matching algorithms for real-time streaming output can be implemented using Aho-Corasick, Commentz-Walter, Wu-Manber, or a double-array Trie.
[0094] In one specific embodiment, the Aho-Corasick automaton algorithm is preferably used. This algorithm, by pre-constructing a finite state automaton containing failure pointers, can complete the matching of all independent keywords in a single scan of the input character stream, with linear time complexity, making it suitable for real-time processing of streaming text.
[0095] In one specific embodiment, the matching step needs to create and maintain a dynamic state table for each streaming output session to record the intermediate states of the matching process. The state table contains state elements that correspond one-to-one with the character positions in the standard character stream output text. The state table is updated each time a character in the standard character stream output text is output and standardized. The state elements include: the current automaton node, the matched individual keywords and their positions, and the list of matched combined keywords. The standardized standard character stream output text includes Chinese characters, English letters, numbers, and spaces. As each character in the standard character stream output text is processed sequentially, the following steps are performed: Figure 3 The following matching and validation loop is shown:
[0096] (4.1) Obtain the position of the next node based on the state transition of the automaton.
[0097] In one specific implementation, for each standard character output, the Aho-Corasick automaton is driven to perform a state transition based on the current automaton node in the current state element, resulting in a new automaton node. The "current automaton node" in the state element is then updated.
[0098] (4.2) Determine whether the node has already matched an independent keyword. If it has, proceed to step (4.3); otherwise, proceed to step (4.1).
[0099] In one specific implementation, the identification of the independent keyword node can be achieved internally by the Aho-Corasick automaton.
[0100] (4.3) Update the status element “Matched independent keyword and position” in the status table.
[0101] In one specific embodiment, after an independent keyword has been matched, the status element in the status table is updated, and the independent keyword and its starting position in the internal text buffer are recorded in the list of matched independent keywords of the new status element.
[0102] (4.4) Obtain the rules for combined keywords corresponding to the independent keyword.
[0103] In one specific implementation, once an independent keyword is matched, all combined keyword rules containing that independent keyword are extracted from the combined keyword rule vocabulary.
[0104] (4.5) For each corresponding keyword combination rule, determine whether the output text has been hit by the context window distance. If yes, execute step (4.6). If not, loop through step (4.1).
[0105] For each combined keyword rule containing a matching independent keyword, it checks whether other keywords in the combination have already matched and are within the context window distance. This search can be achieved by viewing the current status table. Specifically, the matched independent keywords and their positions are confirmed by searching the status element "Matched Independent Keywords and Positions". In one specific implementation, for each corresponding combined keyword rule, let the newly identified independent keyword be K1, its starting position be P_1, the keyword combination of the rule be {K1, K2, ..., Kn}, and the context window distance constraint be X. It checks whether all other keywords {K2, ..., Kn} besides K1 in the rule have appeared, and obtains the position with the earliest starting position (i.e., the smallest value), denoted as P_min.
[0106] Then, a distance judgment is performed: calculate the difference between P_1 and P_min. If the difference is less than or equal to the distance constraint X, then the word combination rule is determined to be hit.
[0107] (4.6) Record the hit results.
[0108] In a specific implementation, the combined keyword rule that is determined to be hit needs to update the status table, and record the unique identifier of the rule, the combined word tag, and the start and end positions of the text range covered by the hit into the list of hit combined keywords in the status element.
[0109] (5) Implement a hierarchical response strategy based on the combined word tags corresponding to the matched combined keyword rules.
[0110] The system monitors the updates of the matched keyword list in the status table in real time. When a new matched record is generated, the system immediately triggers a corresponding tiered response operation based on its associated keyword tags.
[0111] The tiered response operations include, but are not limited to: for Level 1 sensitivity, performing immediate interruption of generation and content replacement; for Level 2 sensitivity, performing post-generation content filtering to restrict topic expansion; and for Level 3 sensitivity, performing background log recording and monitoring.
[0112] In one specific embodiment, the combined word tags are represented by numbers, with smaller numbers indicating higher levels. "Level 1" represents the highest risk, "Level 2" represents medium risk, and "Level 3" represents lower risk.
[0113] In another specific embodiment, the combined word tags are used for content classification, which can be classified according to different content. In this embodiment, the tags include "literature", "medicine", "digital", "astronomy", etc., to provide data annotation for subsequent text analysis.
[0114] The above description describes specific embodiments of the present invention and the technical principles employed. Any changes made in accordance with the concept of the present invention that do not exceed the spirit of the specification and drawings should still fall within the protection scope of the present invention.
Claims
1. A keyword recognition method for streaming data, characterized in that... include: (1) Construct a rule-based thesaurus of combined keywords; (2) Extract and construct an independent keyword index; (3) Preprocess the streaming text character by character and output the standard character stream output text; (4) Perform streaming matching and backtracking verification on the standard character stream output text; (5) Implement a hierarchical response strategy based on the combined word tags corresponding to the matched combined keyword rules.
2. The keyword recognition method for streaming data according to claim 1, characterized in that, The keyword combination rule list is defined by the combination rules of keywords; the keyword combination rules include, but are not limited to: keyword combination, context window distance, and combination word tags.
3. A keyword recognition method for streaming data according to claim 1 or 2, wherein the keyword combination is a vocabulary combination necessary to define a sensitive context.
4. A keyword recognition method for streaming data according to any one of claims 1-3, characterized in that, Context window distance indicates the distance within which all words in the above keyword combination must appear simultaneously in a continuous text segment for the rule to be triggered.
5. A keyword recognition method for streaming data according to any one of claims 1-4, characterized in that, The combined word tag can be a content sensitivity level tag, used to define the sensitivity severity level corresponding to the rule; It can also be a label for keyword types.
6. A keyword recognition method for streaming data according to any one of claims 1-5, characterized in that, Step (2) specifically includes the following steps: (2.1) Traverse the keyword combinations of each rule; (2.2) Select each word in each keyword combination as an independent keyword; (2.3) Remove duplicate words from independent keywords.
7. A keyword recognition method for streaming data according to any one of claims 1-6, characterized in that, Step (2) needs to be executed once after each update of the combined keyword rule vocabulary to generate a new independent keyword index.
8. A keyword recognition method for streaming data according to any one of claims 1-7, characterized in that, Step (3) specifically includes the following steps: (3.1) Obtain a raw character from the upstream streaming interface; (3.2) Determine the current character type: If the current character type is Chinese, proceed to step (3.3); if the current character type is English or number, proceed to step (3.4); if the current character type is space, proceed to step (3.5); if the current character type is punctuation or other non-text characters, proceed to step (3.6). (3.3) Append the current Chinese character according to the predecessor character type and update the predecessor character type, then execute step (3.7); (3.4) Append the current English or numeric character according to the preceding character type, update the preceding character type, and execute step (3.7). (3.5) Append the current space character according to the type of the preceding character, update the type of the preceding character, and execute step (3.7); (3.6) Ignore directly, do not perform any append operation, the previous character type status remains unchanged, and execute step (3.7). (3.7) Determine if the original character stream has ended: If not, repeat step (3.1); if yes, end the current normalized text stream and output the standard character stream output text.
9. A keyword recognition method for streaming data according to any one of claims 1-8, characterized in that, Step (3.3) specifically includes the following steps: (3.3.1) If the current character type is Chinese, determine whether the preceding character type is English or number: if not, proceed to step (3.3.2); if yes, proceed to step (3.3.3). (3.3.2) Directly append the Chinese character to the normalized buffer and execute step (3.3.4); (3.3.3) If the preceding character is of English or number type, first append a space character to the normalized buffer, and then append the current Chinese character; (3.3.4) Update the predecessor character type to Chinese and execute step (3.7).
10. A keyword recognition method for streaming data according to any one of claims 1-9, characterized in that, Step (3.4) specifically includes the following steps: (3.4.1) If the current character type is English or number, determine if the preceding character type is Chinese: if yes, proceed to step (3.4.2); if no, proceed to step (3.4.3). (3.4.2) If the preceding character type is Chinese, first append a space character to the normalized buffer, then append the current English or numeric character, and execute step (3.4.4). (3.4.3) If the preceding character is of type English, number, or space, the character is directly appended to the normalized buffer; (3.4.4) Update the preceding character type to alphanumeric and proceed to step (3.7).
11. A keyword recognition method for streaming data according to claims 1-10, characterized in that, Step (3.5) specifically includes the following steps: (3.5.1) If the current character type is a space, determine whether the preceding character type is English or number: if not, proceed to step (3.5.2); if yes, proceed to step (3.5.3). (3.5.2) If the preceding character type is Chinese or space, ignore the space character, keep the preceding character type unchanged, and proceed to step (3.7). (3.5.3) If the preceding character type is alphanumeric, then this space character is appended to the normalized buffer, and the preceding character type is updated to space. Then proceed to step (3.7).
12. A keyword recognition method for streaming data according to any one of claims 1-11, characterized in that, The preceding character type is a state variable used to identify the type of the previously processed character.
13. A keyword recognition method for streaming data according to any one of claims 1-12, characterized in that, Step (4) specifically includes the following steps: (4.1) Obtain the position of the next node based on the state transition of the automaton; (4.2) Determine whether the node has already matched an independent keyword. If it has, proceed to step (4.3); otherwise, proceed to step (4.1). (4.3) Update the status element "Matched independent keyword and position" in the status table; (4.4) Obtain the rules for combining keywords corresponding to the independent keyword; (4.5) For each corresponding keyword combination rule, determine whether the output text has been hit by the context window distance. If yes, execute step (4.6); otherwise, loop through step (4.1). (4.6) Record the hit results.
14. A keyword recognition method for streaming data according to any one of claims 1-13, characterized in that, A dynamic state table needs to be created and maintained for each streaming match and backtracking verification. The state table contains state elements that correspond one-to-one with the character positions in the standard character stream output text. The state elements include: the current automaton node, the matched individual keywords and their positions, and the list of matched combined keywords.
15. A keyword recognition method for streaming data according to claim 14, characterized in that, In step (4.5), for each combined keyword rule containing a matching independent keyword, it is found whether other keywords in the combination have been matched and are within the context window distance. The search method can be achieved by viewing the current status table. Specifically, the matched independent keywords and positions are confirmed by searching the status element "Matched independent keywords and positions".
16. A keyword recognition method for streaming data according to any one of claims 1-15, characterized in that, The state table is updated when the standard character stream outputs text and a character is normalized.