A method, system, apparatus, equipment, and medium for sensitive word detection.

Abstract

This application provides a method, system, apparatus, device, and medium for sensitive word detection. The method includes: acquiring a file to be detected, wherein the file includes multiple characters; sequentially inputting each character in the file into a sensitive word detection model; matching each character using the sensitive word detection model to obtain a sensitive word detection result for the file; wherein the sensitive word detection model includes at least one set of matching forms constructed from a sensitive word database. Some embodiments of this application can improve the speed and accuracy of sensitive word detection.

Description

Technical Field

[0001] This application relates to the field of network security, specifically to a method, system, apparatus, device, and medium for sensitive word detection. Background Technology

[0002] With the rapid development of the internet, corporate websites and personal websites are becoming increasingly widespread online. For website operators, content review, especially the review of sensitive words, is a crucial task. Manual review is not only time-consuming but also prone to omissions.

[0003] To address the aforementioned issues, related technologies rely on a built-in sensitive word database. This database reads webpage content, segments and indexes the content, and then matches each sensitive word against the index to obtain the matching result. However, existing sensitive word detection methods in website monitoring suffer from significant efficiency bottlenecks due to the extensive disk operations required when creating the local index, making disk I / O performance a major obstacle. Matching content based on sensitive words requires multiple page content matches, resulting in poor efficiency. Furthermore, since the sensitive word database primarily compares directly with the content on the webpage, it is limited by the potential for contextual associations between words, leading to a high likelihood of false positives.

[0004] Therefore, improving the efficiency of sensitive word detection has become a problem that needs to be solved. Summary of the Invention

[0005] This application provides a method, system, apparatus, device, and medium for sensitive word detection. Some embodiments of this application can at least improve the speed and accuracy of sensitive word detection.

[0006] In a first aspect, this application provides a method for sensitive word detection, the method comprising: obtaining a file to be detected, wherein the file to be detected includes multiple characters; sequentially inputting each character in the file to be detected into a sensitive word detection model, matching each character through the sensitive word detection model, and obtaining a sensitive word detection result of the file to be detected, wherein the sensitive word detection model includes at least one set of matching forms constructed from a sensitive word library.

[0007] Therefore, unlike related technologies that use sensitive words to match the content of the file to be detected multiple times, this application establishes a matching form for each sensitive word and matches each character in the file to be detected with the matching form, which can quickly obtain sensitive word matching results and improve the accuracy of sensitive word detection.

[0008] In conjunction with the first aspect, in one embodiment of this application, the matching form is a hash table, and at least one set of hash tables includes a first set of hash tables, wherein the first set of hash tables is any one of the at least one set of hash tables; before inputting each character in the file to be detected sequentially into the sensitive word detection model, the method further includes: using the first character of the first sensitive word as the first index in the first set of hash tables, wherein the first sensitive word is any one of the sensitive word libraries, and one sensitive word corresponds to one set of hash tables; using the remaining characters in the first sensitive word other than the first character as the values ​​corresponding to the first index; establishing a nested index based on the remaining characters other than the first character in the values ​​corresponding to the first index to obtain the first set of hash tables; the step of matching each character through the sensitive word detection model to obtain the sensitive word detection result of the file to be detected includes: matching each character through the at least one set of hash tables to obtain the sensitive word detection result of the file to be detected.

[0009] Therefore, this application improves matching speed by using the first character of the sensitive word as the first index and establishing nested indexes, which enables the quick acquisition of the matching status of the current character when matching words.

[0010] In conjunction with the first aspect, in one embodiment of this application, the step of establishing a nested index based on the remaining characters other than the first character in the value corresponding to the first index to obtain the first set of hash tables includes: using the i-th character among the remaining characters other than the first character as the index, and using the remaining characters other than all characters before the i-th character as the value, where i is an integer greater than or equal to 1; nesting the index and value of the i-th character in the corresponding value of the (i-1)-th character; repeating the above steps until the i-th character is the last character in the first sensitive word, setting the value of the i-th character to a preset identifier, and obtaining the first set of hash tables.

[0011] Therefore, by nesting the index and value of the i-th character into the corresponding value of the (i-1)-th character, this application can combine this feature based on the requirement that sensitive words must be consecutive characters to obtain the matching status of the current character.

[0012] In conjunction with the first aspect, in one embodiment of this application, the sensitive word detection result includes sensitive words in the file to be detected; the step of matching each character through the at least one set of hash tables to obtain the sensitive word detection result of the file to be detected includes: finding the index corresponding to the j-th character in the at least one set of hash tables, obtaining the value corresponding to the j-th character, where j is an integer greater than or equal to 1; repeating the above steps until the value corresponding to the j-th character is a preset identifier, then the word composed of multiple consecutively matched characters before the j-th character is taken as the sensitive word.

[0013] Therefore, this application can determine the status of the current character and improve detection stability by identifying words composed of multiple consecutively matched characters as sensitive words when a preset identifier is matched.

[0014] In conjunction with the first aspect, in one embodiment of this application, after matching each character using the sensitive word detection model to obtain the sensitive word detection result of the file to be detected, the method further includes: if it is confirmed that there are sensitive words in the file to be detected, obtaining the position of the sensitive word in the file to be detected, and forming a word group to be verified with the sensitive word by combining the K characters before and after the position; further verifying the word group to be verified using the sensitive word verification model to obtain normal word groups in the word group to be verified; deleting the sensitive words appearing in the normal word groups from the sensitive words in the file to be detected, and obtaining the target sensitive word detection result of the file to be detected.

[0015] In conjunction with the first aspect, in one embodiment of this application, before further verifying the word group to be verified using a sensitive word verification model to obtain normal word groups in the word group to be verified, the method further includes: establishing at least one set of verification hash tables based on the word group to be verified; the further verification of the word group to be verified using a sensitive word verification model to obtain normal word groups in the word group to be verified includes: matching the word group to be verified using the at least one set of verification hash tables to obtain normal word groups in the word group to be verified.

[0016] Therefore, by further verifying the detected sensitive words, this application can find phrases that are not truly sensitive words when considered in context, thereby reducing the false positive rate of sensitive word detection.

[0017] Secondly, this application provides a sensitive word detection apparatus, the apparatus comprising: a file acquisition module configured to acquire a file to be detected, wherein the file to be detected includes multiple characters; and a sensitive word detection module configured to sequentially input each character in the file to be detected into a sensitive word detection model, and match each character through the sensitive word detection model to obtain a sensitive word detection result for the file to be detected, wherein the sensitive word detection model includes at least one set of matching forms constructed from a sensitive word library.

[0018] In conjunction with the second aspect, in one embodiment of this application, the matching form is a hash table, and at least one set of hash tables includes a first set of hash tables, wherein the first set of hash tables is any one of the at least one set of hash tables; the sensitive word detection module is further configured to: use the first character of the first sensitive word as the first index in the first set of hash tables, wherein the first sensitive word is any one of the sensitive word libraries, and one sensitive word corresponds to one set of hash tables; use the remaining characters of the first sensitive word other than the first character as the values ​​corresponding to the first index; establish a nested index based on the remaining characters other than the first character in the values ​​corresponding to the first index to obtain the first set of hash tables; and match each character through the at least one set of hash tables to obtain the sensitive word detection result of the file to be detected.

[0019] In conjunction with the second aspect, in one embodiment of this application, the sensitive word detection module is further configured to: use the i-th character among the remaining characters excluding the first character as an index, and use the remaining characters excluding all characters before the i-th character as values, where i is an integer greater than or equal to 1; nest the index and value of the i-th character within the corresponding value of the (i-1)-th character; repeat the above steps until the i-th character is the last character in the first sensitive word, set the value of the i-th character to a preset identifier, and obtain the first set of hash tables.

[0020] In conjunction with the second aspect, in one embodiment of this application, the sensitive word detection result includes sensitive words in the file to be detected; the sensitive word detection module is further configured to: find the index corresponding to the j-th character in at least one set of hash tables, obtain the value corresponding to the j-th character, where j is an integer greater than or equal to 1; repeat the above steps until the value corresponding to the j-th character is a preset identifier, then the word composed of multiple consecutively matched characters before the j-th character is taken as the sensitive word.

[0021] In conjunction with the second aspect, in one embodiment of this application, the sensitive word detection module is further configured to: if it is confirmed that there are sensitive words in the file to be detected, obtain the position of the sensitive word in the file to be detected, and form a word group to be verified with the K characters before and after the position and the sensitive word; further verify the word group to be verified through a sensitive word verification model to obtain normal word groups in the word group to be verified; delete the sensitive words that appear in the normal word groups in the sensitive words of the file to be detected, and obtain the target sensitive word detection result of the file to be detected.

[0022] In conjunction with the second aspect, in one embodiment of this application, the sensitive word detection module is further configured to: establish at least one set of verification hash tables based on the word group to be verified; the step of further verifying the word group to be verified through the sensitive word verification model to obtain normal word groups in the word group to be verified includes: matching the word group to be verified through the at least one set of verification hash tables to obtain normal word groups in the word group to be verified.

[0023] Thirdly, this application provides a sensitive word detection system, the system comprising: a client configured to send a file to be detected; and a server configured to acquire the file to be detected and, based on the file to be detected, execute the method described in any embodiment of the first aspect to obtain a sensitive word detection result for the file to be detected.

[0024] Fourthly, this application provides an electronic device, including: a processor, a memory, and a bus; the processor is connected to the memory via the bus, the memory stores a computer program, and the computer program, when executed by the processor, can implement the method as described in any embodiment of the first aspect.

[0025] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when executed, can implement the method as described in any embodiment of the first aspect. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the system composition for sensitive word detection according to an embodiment of this application;

[0027] Figure 2 This is a flowchart of existing methods for sensitive word detection.

[0028] Figure 3 This is one of the flowcharts illustrating a sensitive word detection method according to an embodiment of this application;

[0029] Figure 4 This is a second flowchart illustrating a method for detecting sensitive words according to an embodiment of this application;

[0030] Figure 5 This is a schematic diagram illustrating the composition of a sensitive word detection device according to an embodiment of this application;

[0031] Figure 6 This is a schematic diagram illustrating the composition of an electronic device according to an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the protection scope of this application.

[0033] This application can be applied to scenarios involving the detection of sensitive words in web pages or files. To address the problems in the background art, some embodiments of this application establish a matching form for each sensitive word to obtain sensitive word detection results. For example, in some embodiments of this application, each character in the file to be detected is sequentially input into the sensitive word detection model, and the sensitive word detection model matches each character to obtain the sensitive word detection results for the file to be detected.

[0034] The method steps in the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0035] Figure 1 The present application provides a structural diagram of a sensitive word detection system according to some embodiments. The system includes a client 110 and a server 120. Specifically, the client 110 sends text from a webpage or a file (i.e., each character in the file to be detected) to the server 120. The server 120 sequentially inputs each character in the file to be detected into a sensitive word detection model, matches each character using the sensitive word detection model, obtains the sensitive word detection result of the file to be detected, and then returns the sensitive word detection result to the client 110 for display.

[0036] Unlike the embodiments of this application, related technologies rely on a built-in sensitive word database. By reading webpage content, segmenting and indexing the content, and then matching each built-in sensitive word with the index, a matching result is obtained. For example, ... Figure 2As shown, the method for sensitive word detection in related technologies includes the following steps: S201 read web page content, S202 perform web page word segmentation, S203 establish word segmentation index, S204 obtain sensitive words in the dictionary, S205 cyclically determine whether each sensitive word is in the index, if not, then execute S207 to end, if yes, then execute S206 record the sensitive word matching situation, and then execute S207 to end.

[0037] This application obtains sensitive word detection results by creating a matching form for each sensitive word. Therefore, this application can detect sensitive words more stably and accurately.

[0038] The following uses a server as an example to illustrate a sensitive word detection method performed by a server in some embodiments of this application.

[0039] At least in order to solve the problems in the background technology, such as Figure 3 As shown, some embodiments of this application provide a method for detecting sensitive words, the method including:

[0040] S310, obtain the file to be tested.

[0041] In one embodiment of this application, the file to be detected includes multiple characters. These characters can be from a webpage or a document. The file to be detected can be any file, website, page, etc., that requires sensitive word detection and content monitoring. This application is not limited to this.

[0042] S320: Input each character in the file to be detected into the sensitive word detection model in sequence. The sensitive word detection model matches each character to obtain the sensitive word detection result of the file to be detected.

[0043] It should be noted that the sensitive word detection model includes at least one set of matching forms constructed from a sensitive word database.

[0044] In related technologies, how to efficiently monitor sensitive words is an important indicator for measuring the effectiveness of a sensitive word monitoring system. It is the most important aspect of system design and directly affects the core competitiveness of a sensitive word monitoring platform.

[0045] This application utilizes the concept of finite-state machines to establish a finite-state machine suitable for sensitive word detection scenarios. A finite-state machine (FSM), also known as a finite-state automaton (FSA), is a mathematical computational model representing a finite number of states and the transitions and actions between these states. A finite-state machine consists of a set of states, a start state, a set of input symbols (alphabet), and a transition function that maps the input symbols to the next state. When an input symbol string is input, the model immediately enters the start state. Its transition to a new state depends on the transition function.

[0046] In one embodiment of this application, before S320, it is necessary to establish a hash table for each sensitive word in the sensitive word database, obtaining at least one set of hash tables. Taking any one of the at least one set of hash tables (i.e., the first set of hash tables) as an example, the specific process of establishing the first set of hash tables is described as follows:

[0047] First, the first character in the first sensitive word is used as the first index in the first set of hash tables.

[0048] Understandably, the first sensitive word is any one of the sensitive words in the sensitive word library, and each sensitive word corresponds to a set of hash tables.

[0049] For example, the first sensitive word contains three characters, namely A, B and C. A is the first character in the first sensitive word, and A is used as the first index in the first hash table, namely key1.

[0050] Then, the remaining characters in the first sensitive word (excluding the first character) are used as the values ​​corresponding to the first index.

[0051] For example, if the remaining characters in the first sensitive word list are B and C (excluding the first character), then B and C will be used as the values ​​corresponding to key1, i.e., value1.

[0052] Finally, based on the remaining characters excluding the first character, nested indexes are built within the value corresponding to the first index to obtain the first hash table.

[0053] In other words, nested indexes of B and C are created in value1.

[0054] Specifically, the process of creating a nested index includes:

[0055] First, take the i-th character among the remaining characters excluding the first character as the index, and take the remaining characters excluding all characters before the i-th character as the value, where i is an integer greater than or equal to 1.

[0056] Then, nest the index and value of the i-th character within the corresponding value of the (i-1)-th character. Repeat the above steps until the i-th character is the last character in the first sensitive word, then set the value of the i-th character to the preset identifier to obtain the first hash table.

[0057] In other words, the first character among the remaining characters excluding the first character is used as the index, and the remaining characters excluding all characters before the first character are used as the values. Then, the index and value of the first character are nested in the value of the previous character. At this point, it is determined whether the first character among the remaining characters excluding the first character is the last character in the first sensitive word. If it is the last character, the value of the first character is set to the preset identifier. If not, i is incremented by 1, and the above steps are continued.

[0058] For example, the first sensitive word contains three characters: A, B, and C. A is the first character in the first sensitive word, with an index of key1 and a value of value1. The first character of the remaining characters B and C (excluding the first character A) is B, so B is used as the index, key2. The remaining characters C (excluding the first character B and all characters before A and B) are used as the value, value2. The index key2 and value2 of the first character B are nested within the value value1 of the previous character A. At this point, it is determined that the first character B among the remaining characters (excluding the first character) is not the last character in the first sensitive word, so i is incremented by 1, and the above steps continue.

[0059] Next, i+1 = 2. At this point, the second character among the remaining characters B and C (excluding the first character A) is C. We use C as the index, i.e., key3. We use the remaining characters (excluding all characters before the second character C, i.e., A, B, and C) as the value (in this example, C is the last character, so value3 is currently empty and will be filled in the next step) as the value, i.e., value3. We nest the index key3 and value3 of the second character C within the value value2 of the previous character B. We then determine that the second character C among the remaining characters (excluding the first character) is the last character in the first sensitive word, so we set a preset flag in value3, for example, -1. This yields a set of hash tables for sensitive words A, B, and C. Following this method, we establish corresponding hash tables for at least one sensitive word in the sensitive word database, obtaining at least one set of hash tables.

[0060] A set of hash tables for sensitive words A, B, and C can be represented as follows:

[0061] "key1value1[key2value2(key3 value3)]"

[0062] In other words, the content of value1 includes [key2value2(key3value3)], and the content of value2 includes (key3 value3).

[0063] In simple terms, the above process involves building a finite state machine based on a sensitive word database. Specifically, sensitive words are obtained from the database, and the initial state of the state machine is built based on the prefix word (i.e., the first word) of each sensitive word. This stage mainly uses a hash table (key-value pairs) to store the prefix word of the sensitive word as the key, and then the value is nested in the hash table, where each of the remaining words of the sensitive word is placed. The last key is the last word, and the value is the exit identifier - 1.

[0064] In one embodiment of this application, after establishing at least one set of hash tables, each character is matched using at least one set of hash tables to obtain the sensitive word detection results of the file to be detected. The specific process includes:

[0065] First, find the index corresponding to the j-th character in at least one set of hash tables, and obtain the value corresponding to the j-th character, where j is an integer greater than or equal to 1.

[0066] Then, repeat the above steps until the value corresponding to the j-th character is the preset identifier. Then, the word composed of multiple consecutively matched characters before the j-th character is taken as the sensitive word.

[0067] For example, if the current file to be detected contains multiple characters A, D, F, and E, when j=1, the index corresponding to the first character (A) is key1, and its value is value1. If the content of value1 is not a preset identifier, then j+1 is incremented, and the search continues in at least one hash table to find the index corresponding to the second character D. If the content of value1 is a preset identifier, then A is determined to be a sensitive word, and the search continues in at least one hash table to find the index of the next character.

[0068] For example, if the current file to be detected contains multiple characters A, B, C, and E, when j=1, the index corresponding to the first character (A) is key1, and the value is value1. Since the content of value1 is not a preset identifier, j is incremented by 1. Then, the search continues in at least one hash table to find the index corresponding to the second character B, which is key2, and the value is value2. Since the content of value2 is not a preset identifier, j is incremented by 1. The search continues in at least one hash table to find the index corresponding to the third character C, which is key3, and the value is value3. At this point, value3 is the preset identifier. Therefore, the word consisting of multiple consecutively matched characters before the third character is considered a sensitive word, i.e., ABC is a sensitive word. The search continues in at least one hash table to find the index of the next character until the file to be detected is completely detected, thus obtaining all the sensitive words in the file.

[0069] In simple terms, during the sensitive word detection phase, the system primarily reads the content to be detected in a loop, inputting it into the state machine's entry state. Based on the characters read (i.e., words in the file to be detected), and according to the state transition conditions (i.e., at least one hash table), the system automatically selects the next state value. If the read characters do not meet the transition conditions (i.e., no match is found in at least one hash table), the system returns to the entry state and continues reading and matching. If the state machine's state value transitions to -1 (i.e., a preset flag is matched), it confirms that the current detected content contains the specified sensitive word. After marking the length, the system returns to the entry state to continue matching.

[0070] In one embodiment of this application, sensitive words are detected and further verified after S320. The specific implementation steps are as follows:

[0071] First, if sensitive words are confirmed to exist in the file to be detected, the position of the sensitive words in the file to be detected is obtained, and the K characters before and after the position are combined with the sensitive words to form a word group to be verified.

[0072] In other words, to prevent the detected sensitive words from not being truly sensitive words when combined with the contextual semantics, this application will perform sensitive word detection again on the K-character phrase before and after the detected sensitive word in the text.

[0073] Then, the sensitive word verification model is used to further verify the word group to be verified, and the normal word groups in the word group to be verified are obtained.

[0074] Specifically, at least one set of verification hash tables is established based on the phrase to be verified. The phrase to be verified is matched using at least one set of verification hash tables. In other words, at least one set of verification hash tables is established using the same method as the above method. Each character in the phrase to be verified is matched with at least one set of verification hash tables. If a match is successful, it means that the current sensitive word is a false alarm. In other words, the currently matched sensitive word is a normal phrase and the alert for the current sensitive word can be ignored.

[0075] Finally, sensitive words that appear in normal word groups are deleted from the sensitive words in the file to be detected, and the target sensitive word detection results of the file to be detected are obtained.

[0076] In other words, the sensitive words detected by the normal word groups obtained in the previous step are not truly sensitive words when combined with the contextual semantics. Therefore, the sensitive words included in these normal word groups are deleted to obtain the final target sensitive word detection results.

[0077] For example, if the sensitive word detected by the sensitive word state machine is AB (i.e., the word group to be verified in this embodiment), it re-enters the verification state machine (i.e., at least one set of verification hash tables) for matching. If the matched word is DABF, it means that the current sensitive word AB is a normal word group and no alarm should be triggered. It can be understood that the word groups corresponding to at least one set of verification hash tables are all normal words, i.e., whitelisted words.

[0078] It is understandable that the process of matching the phrase to be verified is the same as the process of matching the characters in the file to be detected, and will not be repeated here.

[0079] In simple terms, if a sensitive word is matched, the characters before and after it are extracted and input into the verification state machine (i.e., at least one set of verification hash tables) for matching. If the match fails, it means that the current sensitive word is not in the verification word library, and there is no false alarm. Conversely, if the match succeeds, it means that the current sensitive word is a false alarm, and the matching result is ignored.

[0080] As a specific embodiment of this application, this application mainly constructs a finite state machine based on a sensitive word database, cyclically reads the matched content, and then sequentially inputs it into the constructed finite state machine. Based on the current state and the read characters, it transitions from the current state to the next state according to the state machine transition rules. When the entire string has been read, if the automaton is in a certain accepting state, it is determined that the monitored string contains the specified sensitive word; otherwise, it is determined that the string does not contain the specified keyword. Then, the length of the current matching state machine is recorded, and the characters at the current length are truncated to obtain the detected sensitive word in the current content. Simultaneously, the process returns to the state machine's entry point from the current position to continue matching.

[0081] For example, such as Figure 4 As shown, the initial state represents inputting a word A. If A matches at least one hash table, the user enters the first state; otherwise, they enter another state and return to the initial state. Then, the user gets the next word B. If B matches at least one hash table, the user enters the second state; otherwise, they enter another state and return to the initial state. Finally, the user gets the next word C. If C matches at least one hash table, the user enters the final state, indicating that the current word ABC is a sensitive word.

[0082] As a specific scenario of this application, the sensitive word detection function in the content monitoring of website monitoring needs to detect sensitive words in the web page content, thereby detecting the sensitive words appearing in the corresponding website, and then promptly notifying the customer to reduce potential website risks.

[0083] Therefore, compared with traditional website monitoring sensitive word detection methods, the method provided in this application can greatly improve the speed and efficiency of sensitive word detection because it only requires one matching of the detected content and adds a verification stage. At the same time, it improves the accuracy of sensitive word detection, greatly reduces or avoids the possibility of false alarms and misreports of sensitive words in website monitoring, and solves the problem of relatively low efficiency of sensitive word analysis process and the problem of easy false alarms of sensitive word alerts in the process of website sensitive word monitoring.

[0084] Therefore, this application uses a finite state machine for sensitive word matching. By constructing a finite state machine based on the sensitive word database and then using it for sensitive word matching, the efficiency of sensitive word detection in website monitoring is greatly improved. Simultaneously, a sensitive word verification and detection function is used, significantly improving the accuracy of sensitive word detection alerts in website monitoring.

[0085] The above describes a specific embodiment of a method for detecting sensitive words; the following describes an apparatus for detecting sensitive words.

[0086] like Figure 5 As shown, some embodiments of this application provide a sensitive word detection device 500, which includes a file acquisition module 510 and a sensitive word detection module 520.

[0087] The file acquisition module 510 is configured to acquire a file to be detected, wherein the file to be detected includes multiple words.

[0088] The sensitive word detection module 520 is configured to input each character in the file to be detected sequentially into the sensitive word detection model, and match each character through the sensitive word detection model to obtain the sensitive word detection result of the file to be detected. The sensitive word detection model includes at least one set of matching forms constructed from a sensitive word library.

[0089] In one embodiment of this application, the matching form is a hash table, and at least one set of hash tables includes a first set of hash tables, wherein the first set of hash tables is any one of the at least one set of hash tables; the sensitive word detection module 520 is further configured to: use the first character of the first sensitive word as the first index in the first set of hash tables, wherein the first sensitive word is any one of the sensitive word library, and one sensitive word corresponds to one set of hash tables; use the remaining characters of the first sensitive word other than the first character as the value corresponding to the first index; establish a nested index based on the remaining characters other than the first character in the value corresponding to the first index to obtain the first set of hash tables; and match each character through the at least one set of hash tables to obtain the sensitive word detection result of the file to be detected.

[0090] In one embodiment of this application, the sensitive word detection module 520 is further configured to: use the i-th character among the remaining characters excluding the first character as an index, and use the remaining characters excluding all characters before the i-th character as values, where i is an integer greater than or equal to 1; nest the index and value of the i-th character within the corresponding value of the (i-1)-th character; repeat the above steps until the i-th character is the last character in the first sensitive word, set the value of the i-th character to a preset identifier, and obtain the first set of hash tables.

[0091] In one embodiment of this application, the sensitive word detection result includes sensitive words in the file to be detected; the sensitive word detection module 520 is further configured to: find the index corresponding to the j-th character in at least one set of hash tables, obtain the value corresponding to the j-th character, where j is an integer greater than or equal to 1; repeat the above steps until the value corresponding to the j-th character is a preset identifier, then the word composed of multiple consecutively matched characters before the j-th character is taken as the sensitive word.

[0092] In one embodiment of this application, the sensitive word detection module 520 is further configured to: if it is confirmed that there are sensitive words in the file to be detected, obtain the position of the sensitive word in the file to be detected, and form a word group to be verified with the K characters before and after the position and the sensitive word; further verify the word group to be verified through a sensitive word verification model to obtain normal word groups in the word group to be verified; delete the sensitive words that appear in the normal word groups in the sensitive words of the file to be detected, and obtain the target sensitive word detection result of the file to be detected.

[0093] In one embodiment of this application, the sensitive word detection module 520 is further configured to: establish at least one set of verification hash tables based on the word group to be verified; the step of further verifying the word group to be verified through the sensitive word verification model to obtain normal word groups in the word group to be verified includes: matching the word group to be verified through the at least one set of verification hash tables to obtain normal word groups in the word group to be verified.

[0094] In the embodiments of this application, Figure 5 The module shown can achieve Figures 1 to 4 Each process in the method embodiment. Figure 5 The operations and / or functions of each module in the document are respectively designed to achieve... Figures 1 to 4 The corresponding processes in the method embodiments are described above. For details, please refer to the descriptions in the above method embodiments; to avoid repetition, detailed descriptions are omitted here.

[0095] like Figure 6 As shown, this application provides an electronic device 600, including: a processor 610, a memory 620 and a bus 630. The processor is connected to the memory via the bus. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the method as described in any one of the above embodiments. For details, please refer to the description in the above method embodiments. To avoid repetition, detailed descriptions are appropriately omitted here.

[0096] The bus is used to enable direct communication between these components. In this embodiment, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an On-Premises Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor.

[0097] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores computer-readable instructions, which, when executed by the processor, can perform the methods described in the above embodiments.

[0098] Understandable. Figure 6 The structure shown is for illustrative purposes only and may include structures larger than those shown. Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown. Figure 6 The components shown can be implemented using hardware, software, or a combination thereof.

[0099] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a server, it implements any of the methods described in all the above embodiments. For details, please refer to the descriptions in the above method embodiments. To avoid repetition, detailed descriptions are appropriately omitted here.

[0100] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for sensitive word detection, the method comprising: The method includes: Obtain the file to be detected, wherein the file to be detected includes multiple words; Each character in the file to be detected is sequentially input into the sensitive word detection model, and the sensitive word detection model is used to match each character to obtain the sensitive word detection result of the file to be detected. The sensitive word detection model contains at least one set of matching forms constructed from a sensitive word library. If it is confirmed that there are sensitive words in the file to be detected, the position of the sensitive words in the file to be detected is obtained, and the K characters before and after the position are combined with the sensitive words to form a word group to be verified; The sensitive word verification model is used to further verify the word group to be verified, and the normal word groups in the word group to be verified are obtained. The sensitive words appearing in the normal word groups are deleted from the sensitive words in the file to be detected, and the target sensitive word detection results of the file to be detected are obtained.

2. The method of claim 1, wherein, The matching form is a hash table, and at least one set of hash tables includes a first set of hash tables, wherein the first set of hash tables is any one of the at least one set of hash tables; Before sequentially inputting each character from the file to be detected into the sensitive word detection model, the method further includes: The first character in the first sensitive word is used as the first index in the first set of hash tables, where the first sensitive word is any one of the sensitive word library, and one sensitive word corresponds to one set of hash tables; The remaining characters in the first sensitive word, excluding the first character, are used as the value corresponding to the first index; In the value corresponding to the first index, a nested index is established based on the remaining characters other than the first character to obtain the first set of hash tables; The step of matching each character using the sensitive word detection model to obtain the sensitive word detection result of the file to be detected includes: By matching each character using at least one set of hash tables, the sensitive word detection result of the file to be detected is obtained.

3. The method of claim 2, wherein, The step of establishing a nested index based on the remaining characters other than the first character in the value corresponding to the first index to obtain the first hash table includes: Use the i-th character among the remaining characters excluding the first character as the index, and use the remaining characters excluding all characters before the i-th character as the value, where i is an integer greater than or equal to 1; The index and value of the i-th character are nested within the corresponding value of the (i-1)-th character; Repeat the above steps until the i-th character is the last character in the first sensitive word, then set the value of the i-th character to a preset identifier to obtain the first set of hash tables.

4. The method of claim 2, wherein, The sensitive word detection results include the sensitive words in the file to be detected; The step of matching each character using the at least one set of hash tables to obtain the sensitive word detection result of the file to be detected includes: The index corresponding to the j-th character is found in the at least one set of hash tables, and the value corresponding to the j-th character is obtained, where j is an integer greater than or equal to 1; Repeat the above steps until the value corresponding to the j-th character is a preset identifier. Then, the word composed of multiple consecutively matched characters before the j-th character is taken as the sensitive word.

5. The method of claim 1, wherein, Before further validating the word group to be verified using a sensitive word verification model to obtain normal word groups within the word group to be verified, the method further includes: Establish at least one set of verification hash tables based on the phrases to be verified; The step of further verifying the word group to be verified using a sensitive word verification model to obtain normal word groups within the word group to be verified includes: The normal word groups in the word groups to be verified are obtained by matching the word groups to be verified using at least one set of verification hash tables. 6.A system for sensitive word detection, characterized in that, The system includes: The client is configured to send the file to be tested; The server is configured to acquire the file to be detected and, based on the file to be detected, execute the method as described in any one of claims 1-5 to obtain the sensitive word detection result of the file to be detected.

7. An apparatus for sensitive word detection, the apparatus comprising: The device includes: The file acquisition module is configured to acquire a file to be detected, wherein the file to be detected includes multiple words; The sensitive word detection module is configured to input each character in the file to be detected sequentially into the sensitive word detection model, and match each character through the sensitive word detection model to obtain the sensitive word detection result of the file to be detected. The sensitive word detection model includes at least one set of matching forms constructed from a sensitive word library. If it is confirmed that there are sensitive words in the file to be detected, the position of the sensitive words in the file to be detected is obtained, and the K characters before and after the position are combined with the sensitive words to form a word group to be verified; The sensitive word verification model is used to further verify the word group to be verified, and the normal word groups in the word group to be verified are obtained. The sensitive words appearing in the normal word groups are deleted from the sensitive words in the file to be detected, and the target sensitive word detection results of the file to be detected are obtained.

8. An electronic device, comprising: include: Processor, memory, and bus; The processor is connected to the memory via the bus, and the memory stores a computer program that, when executed by the processor, can implement the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1-5.

Images