System and method for analyzing sensitive vocabularies in text in scene supporting low operational capability and low network

Abstract

The invention discloses a system and a method for analyzing sensitive vocabularies in a text in a scene supporting low operational capability and low network, and belongs to the technical field of text information processing, the system comprises a central processing unit, and the central processing unit is connected with an input and output module. An interaction module is arranged in the system, fuzzy vocabularies are synchronously extracted in the sensitive word extraction process, the fuzzy vocabularies are judged manually, it is guaranteed that when the sensitive vocabularies are rapidly analyzed through the system, the fuzzy vocabularies can be analyzed in a manual review mode, omission is avoided, and the sensitivity of the sensitive vocabularies is improved. According to the method, accurate recognition and analysis of texts are guaranteed, the word bank module and the cache module are arranged and are used in cooperation with the interaction module, sensitive vocabulary data caching can be carried out after each time of text analysis, manual sensitive vocabulary input can also be carried out, and a word bank can be constructed; and it is ensured that text sensitive vocabularies can be recognized and analyzed accurately in a low-computing-capacity and low-network scene.

Description

Technical Field

[0001] The present invention relates to the technical field of text information processing, and in particular to a system and method for analyzing sensitive words in texts in scenarios supporting low computing power and low network. Background Art

[0002] In the current information age, with the popularity of the Internet and social media, the scale and complexity of text data have increased dramatically. In order to maintain the health and safety of network scenarios and prevent the spread of bad information, it is necessary to screen the text content, identify sensitive words in the text and block and improve them. Traditional sensitive words are analyzed manually. Due to the different experiences of security operators, the probability of false alarms and missed alarms increases, which reduces the accuracy of determining the type of sensitive information, and then leads to errors in the judgment of threats in the text. Therefore, sensitive word analysis systems and methods are particularly important. This system automatically analyzes the text content and identifies words that may contain sensitive or inappropriate content, thereby helping regulators, companies or individuals take timely measures to prevent the spread of bad information.

[0003] In the existing sensitive word analysis system and method, there is no interactive module in the analysis system, and there is no manual judgment process in the analysis method. When relying on the system to screen sensitive words, it can only identify and screen sensitive words already stored in big data. For words at the boundary of sensitive words, the system cannot accurately identify them and is prone to omissions, which in turn affects the accurate analysis and processing of the text. In addition, in low computing power and low network scenarios, the use of this system and method will also affect the accurate analysis of the text, and the use effect is general. Summary of the invention

[0004] The main purpose of the present invention is to provide a system and method for analyzing sensitive words in texts that supports low computing power and low network scenarios, which can effectively solve the problems in the background technology.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A sensitive word analysis system in a text supporting low computing power and low network scenarios includes a central processing unit, an input and output module is connected to the central processing unit, and the input and output module is electrically connected to the central processing unit, a preprocessing module is connected to the central processing unit, and a vocabulary extraction module is connected to the central processing unit, a precision processing module is electrically connected to the central processing unit, and an interaction module is connected to the central processing unit, and the interaction module is bidirectionally electrically connected to the central processing unit. A word library module is electrically connected to the central processing unit, and the word library module is bidirectionally electrically connected to the central processing unit, and a cache module is bidirectionally electrically connected to the central processing unit, the word library module is used to store sensitive words, and the cache module is used to store matched results.

[0007] Preferably, a power module is connected to the central processing unit, and an output end of the power module is electrically connected to an input end of the central processing unit, the central processing unit is bidirectionally electrically connected to the preprocessing module, and the central processing unit is bidirectionally electrically connected to the vocabulary extraction module.

[0008] Preferably, the input-output module includes an input unit and an output unit, and the input-output module is bidirectionally electrically connected to the central processing unit, and the output unit is connected to the user.

[0009] Preferably, the precise processing module comprises an identification unit, a classification unit and a feedback unit, the output end of the identification unit is electrically connected to the input end of the classification unit, and the output end of the classification unit is electrically connected to the input end of the feedback unit.

[0010] The present invention also provides a method for analyzing sensitive words in texts in low computing power and low network scenarios, comprising the following steps:

[0011] S1, text input, collect original data from various text sources, and use input and output modules to input text data;

[0012] S2, data preprocessing, using the preprocessing module to preprocess the acquired text data, including denoising, word segmentation, part-of-speech tagging, etc., for subsequent analysis;

[0013] S3, building a model, building an initial sensitive word analysis model through the central processing unit to form a sensitive word database;

[0014] S4, sensitive word extraction and fuzzy word extraction, after step S3 is completed, the sensitive words and fuzzy words in the text are extracted using a word extraction module;

[0015] S5, sensitive word recognition and manual determination of fuzzy words, after step S4 is completed, the sensitive words are recognized by the recognition unit, and the fuzzy words are determined manually;

[0016] S6, sensitive word classification, after step S5 is completed, the sensitive words are classified by the classification unit and marked;

[0017] S7, evaluation and optimization. After classifying sensitive words, the model is evaluated through cross-validation, confusion matrix and other methods. The model is optimized based on the evaluation results to improve the generalization ability and robustness of the model.

[0018] S8. Result output and data caching. After the evaluation and optimization are completed, qualified text information will be published and unqualified text information will be blocked. At the same time, the results related to sensitive words will be cached.

[0019] Preferably, in step S5, when manual determination is performed, text information determined to be sensitive words is classified together with sensitive words detected by the initial system, and text information determined not to be sensitive words is directly output.

[0020] Preferably, in step S7, performing evaluation optimization can increase the sensitive vocabulary of the original database and improve the accuracy of subsequent text analysis.

[0021] Preferably, when the steps S6 and S7 are performed, corresponding data is retrieved from the sensitive word related data in the text cached after previous evaluation and optimization, so as to avoid repeated calculations, thereby speeding up the recognition and classification rate of the current sensitive words.

[0022] Compared with the prior art, the present invention has the following beneficial effects: the system and method for analyzing sensitive words in texts supporting low computing power and low network scenarios, an interactive module is set in the system, fuzzy words are extracted synchronously during the sensitive word extraction process, and fuzzy words are judged manually to ensure that when the system is used to quickly analyze sensitive words, fuzzy words can be analyzed by manual review to avoid omissions, thereby ensuring accurate recognition and analysis of texts; a classification unit is set in the precise processing module to classify sensitive words and improve system retrieval efficiency; through the set word library module and cache module, in conjunction with the interactive module, sensitive word data can be cached after each text analysis, and manual sensitive words can be input and a word library can be constructed, thereby ensuring that accurate recognition and analysis of text sensitive words can be performed even in low computing power and low network scenarios; evaluation and optimization are performed after each text analysis, and the evaluation and optimization results are cached, thereby increasing the amount of sensitive words in the database and improving the accuracy of subsequent analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A system block diagram of a sensitive word analysis system in a text supporting low computing power and low network scenarios according to the present invention;

[0024] Figure 2 The present invention is a flowchart of a method for analyzing sensitive words in texts that supports low computing power and low network scenarios. DETAILED DESCRIPTION

[0025] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0026] like Figure 1 As shown, a system for analyzing sensitive words in texts supporting low computing power and low network scenarios includes a central processing unit, an input and output module is connected to the central processing unit, and the input and output module is electrically connected to the central processing unit, a preprocessing module is connected to the central processing unit, and a vocabulary extraction module is connected to the central processing unit, a precise processing module is electrically connected to the central processing unit, and an interactive module is connected to the central processing unit, and the interactive module is bidirectionally electrically connected to the central processing unit;

[0027] The central processing unit is electrically connected to a word library module, and the word library module is bidirectionally electrically connected to the central processing unit. The central processing unit is bidirectionally electrically connected to a cache module. The word library module is used to store sensitive words, and the cache module is used to store matched results.

[0028] Through the above embodiments, a power module is connected to the central processing unit, and the output end of the power module is electrically connected to the input end of the central processing unit, the central processing unit is bidirectionally electrically connected to the preprocessing module, and the central processing unit is bidirectionally electrically connected to the vocabulary extraction module. The input-output module includes an input unit and an output unit, and the input-output module is bidirectionally electrically connected to the central processing unit, and the output unit is connected to the user. The precision processing module includes a recognition unit, a classification unit and a feedback unit, the output end of the recognition unit is electrically connected to the input end of the classification unit, and the output end of the classification unit is electrically connected to the input end of the feedback unit.

[0029] like Figure 2 As shown, a method for analyzing sensitive words in text in low computing power and low network scenarios includes the following steps:

[0030] S1, text input, collect original data from various text sources, and use input and output modules to input text data;

[0031] S2, data preprocessing, using the preprocessing module to preprocess the acquired text data, including denoising, word segmentation, part-of-speech tagging, etc., for subsequent analysis;

[0032] S3, building a model, building an initial sensitive word analysis model through the central processing unit to form a sensitive word database;

[0033] S4, sensitive word extraction and fuzzy word extraction, after step S3 is completed, the sensitive words and fuzzy words in the text are extracted using a word extraction module;

[0034] S5, sensitive word recognition and manual determination of fuzzy words, after step S4 is completed, the sensitive words are recognized by the recognition unit, and the fuzzy words are determined manually;

[0035] S6, sensitive word classification, after step S5 is completed, the sensitive words are classified by the classification unit and marked;

[0036] S7, evaluation and optimization. After classifying sensitive words, the model is evaluated through cross-validation, confusion matrix and other methods. The model is optimized based on the evaluation results to improve the generalization ability and robustness of the model.

[0037] S8. Result output and data caching. After the evaluation and optimization are completed, qualified text information will be published and unqualified text information will be blocked. At the same time, the results related to sensitive words will be cached.

[0038] Through the above embodiment, in step S5, when manual judgment is performed, the text information judged as sensitive words is classified together with the sensitive words detected by the initial system, and the text information judged not to be sensitive words is directly output. In step S7, evaluation optimization can increase the amount of sensitive words in the original database and improve the accuracy of subsequent text analysis. When the steps S6 and S7 are performed, the corresponding data is retrieved from the sensitive word-related data in the text cached after previous evaluation optimization to avoid repeated calculations, which can speed up the recognition and classification rate of the current sensitive words.

[0039] When analyzing and processing sensitive words in text information, the power module supplies power to the central processing unit in the system, and text input is performed through the input unit in the input and output module. Then the central processing unit builds a model, produces a sensitive word database, and uses the preprocessing module to preprocess the text information. The text information is processed by denoising, acne, and part-of-speech tagging. After the processing is completed, the vocabulary extraction module is used to extract sensitive words and fuzzy words. The extraction results are transmitted to the precise processing module, in which the recognition unit identifies sensitive words, and the fuzzy words are manually judged through the interaction module. If it is determined that it is not a sensitive word, it can be directly output. If it is determined to be a sensitive word, the fuzzy word is classified into sensitive words and uniformly classified by the classification module. After different types of sensitive words are classified, the feedback unit feeds back the relevant information to the central processing unit. The central processing unit evaluates and optimizes the sensitive words, supplements the database of sensitive words, and outputs information at the same time. Due to the settings of the vocabulary module and the cache module, the cache module will cache the relevant results of sensitive words after each evaluation and optimization. When the system is used in a scenario with low computing power and low network, the cached data in the cache module can be extracted and used to directly match the results without repeated calculations, thus reducing the difficulty of calculation. In the case of low network, the interactive module can be used in conjunction with the vocabulary module to manually build a database of sensitive words, and then perform recognition analysis after text input, ensuring that the recognition and analysis of sensitive words in the text can be accurately completed even in low network and low computing power scenarios.

[0040] It should be noted that an interactive module is set up in the system, and fuzzy words are extracted synchronously during the sensitive word extraction process. Fuzzy words are judged manually to ensure that when the system is used to quickly analyze sensitive words, fuzzy words can be analyzed by manual review to avoid omissions, thereby ensuring accurate recognition and analysis of texts. A classification unit is set up in the precise processing module to classify sensitive words and improve the system retrieval efficiency. Through the use of the set vocabulary module and cache module in conjunction with the interactive module, sensitive word data can be cached after each text analysis, and sensitive words can be manually input and a vocabulary can be built to ensure accurate recognition and analysis of text sensitive words even in low computing power and low network scenarios. Evaluation and optimization are performed after each text analysis, and the evaluation and optimization results are cached, which can increase the number of sensitive words in the database and improve the accuracy of subsequent analysis.

[0041] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A sensitive word analysis system in text that supports low computing power and low network scenarios, characterized by: The system comprises a central processing unit, an input / output module is connected to the central processing unit, and the input / output module is electrically connected to the central processing unit, a preprocessing module is connected to the central processing unit, and a vocabulary extraction module is connected to the central processing unit, a precision processing module is electrically connected to the central processing unit, and an interaction module is connected to the central processing unit, and the interaction module is bidirectionally electrically connected to the central processing unit; The central processing unit is electrically connected to a word library module, and the word library module is bidirectionally electrically connected to the central processing unit. The central processing unit is bidirectionally electrically connected to a cache module, the word library module is used to store sensitive words, and the cache module is used to store matched results.

2. According to claim 1, a system for analyzing sensitive words in texts that supports low computing power and low network scenarios, characterized in that: The central processor is connected with a power module, and the output end of the power module is electrically connected to the input end of the central processor. The central processor is bidirectionally electrically connected to the preprocessing module, and the central processor is bidirectionally electrically connected to the vocabulary extraction module.

3. The sensitive word analysis system in text supporting low computing power and low network scenarios according to claim 2, characterized in that: The input-output module includes an input unit and an output unit, and the input-output module is bidirectionally electrically connected to the central processing unit, and the output unit is connected to the user.

4. According to claim 3, a system for analyzing sensitive words in texts that supports low computing power and low network scenarios is characterized in that: The precise processing module comprises an identification unit, a classification unit and a feedback unit. The output end of the identification unit is electrically connected to the input end of the classification unit, and the output end of the classification unit is electrically connected to the input end of the feedback unit.

5. A method for analyzing sensitive words in texts in low computing power and low network scenarios, characterized in that: The following steps are involved: S1, text input, collect original data from various text sources, and use input and output modules to input text data; S2, data preprocessing, using the preprocessing module to preprocess the acquired text data, including denoising, word segmentation, part-of-speech tagging, etc., for subsequent analysis; S3, building a model, building an initial sensitive word analysis model through the central processing unit to form a sensitive word database; S4, sensitive word extraction and fuzzy word extraction, after step S3 is completed, the sensitive words and fuzzy words in the text are extracted using a word extraction module; S5, sensitive word recognition and manual determination of fuzzy words, after step S4 is completed, the sensitive words are recognized by the recognition unit, and the fuzzy words are determined manually; S6, sensitive word classification, after step S5 is completed, the sensitive words are classified by the classification unit and marked; S7, evaluation and optimization. After classifying sensitive words, the model is evaluated through cross-validation, confusion matrix and other methods. The model is optimized based on the evaluation results to improve the generalization ability and robustness of the model. S8. Result output and data caching. After the evaluation and optimization are completed, qualified text information will be published and unqualified text information will be blocked. At the same time, the results related to sensitive words will be cached.

6. According to claim 5, a method for analyzing sensitive words in texts in low computing power and low network scenarios is characterized in that: In step S5, when manual determination is performed, text information determined to be sensitive words is classified together with sensitive words detected by the initial system, and text information determined not to be sensitive words is directly output.

7. The method for analyzing sensitive words in texts in low computing power and low network scenarios according to claim 6, characterized in that: In step S7, performing evaluation optimization can increase the sensitive vocabulary of the original database and improve the accuracy of subsequent text analysis.

8. The method for analyzing sensitive words in texts in low computing power and low network scenarios according to claim 7, characterized in that: When the steps S6 and S7 are performed, corresponding data is retrieved from the sensitive word related data in the text cached after previous evaluation and optimization, so as to avoid repeated calculations, thereby speeding up the recognition and classification rate of the current sensitive words.

Images