Intelligent recognition method and system for website sensitive words and storage medium

Abstract

The invention relates to an intelligent recognition method and system for website sensitive words and a storage medium. The intelligent recognition method for the website sensitive words comprises the following steps: S1, website text data acquisition: in a website data acquisition stage, acquiring original data of a website, and fully acquiring source code data of an entered website in real time; s2, website text data preprocessing: after website data acquisition is completed, preprocessing the data; and S3, intelligent recognition of the sensitive words of the website text: an artificial neural network algorithm is realized by constructing a neuron model, and sensitive word information in the text is automatically and quickly recognized. Iterative self-organizing feature extraction is carried out by means of an artificial neural network algorithm, sensitive information is automatically recognized, high-precision recognition capability is achieved, and the detection rate and the accuracy rate of the sensitive words of the website are increased; high throughput and low delay are maintained, the detection efficiency is improved, and the timeliness is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and in particular relates to an intelligent recognition method, system and storage medium for sensitive words on a website. Background Art

[0002] Among the existing technologies for identifying sensitive words on websites, most traditional monitoring software is based on rule matching, based on a pre-set sensitive word library, and then crawling the website, and finally matching the words on the website;

[0003] This method generally has the following problems:

[0004] The generalization ability is insufficient. The recognition process cannot determine whether sensitive words are compliant based on the context, and the false alarm rate is high. It cannot perform self-optimization and self-learning, and requires manual intervention from experts in business and technical fields, which makes construction difficult. Summary of the invention

[0005] The purpose of the present invention is to provide a method, system and storage medium for intelligently identifying sensitive words on a website, so as to solve the problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for intelligently identifying sensitive words on a website, the method comprising:

[0008] S1: Website text data collection: During the website data collection phase, the original data of the website is collected and the source code data entered into the website is collected in full real-time;

[0009] S2: Preprocessing of website text data. After the website data collection is completed, the data is preprocessed;

[0010] S3: Intelligent identification of sensitive words in website texts. By building a neuron model to implement an artificial neural network algorithm, it can automatically and quickly identify sensitive word information in the text.

[0011] Furthermore, the source code data in S1 is obtained by setting the collection depth of the entered website data.

[0012] Furthermore, the setting of the data collection depth is performed by manual configuration during the collection process.

[0013] Furthermore, the data described in S2 is pre-processed to ensure data security and protect the privacy of website data in accordance with the law. At the same time, the data is sent to a message queue for decoupled caching and multi-level consumption processing.

[0014] Furthermore, the artificial neural network in S3 is composed of a neuron model; the neuron model includes:

[0015] Construct an operation unit, and use the operation unit to construct neurons. The information processing network composed of many neurons has a parallel distribution structure. Each neuron has a single output and can be connected to other neurons. There are multiple output connection methods, and each connection corresponds to a connection weight coefficient;

[0016] Connect multiple single neurons together to form a network model.

[0017] Furthermore, the output of the operation unit in S3.1 is the function hw,b(x):

[0018] hw,b(x) depends on two parameters w and b and the input x, w is a weight vector, w T is the transpose of the weight vector w, x is an input vector containing a series of weights w, w*x represents the dot product of the weight vector w and the input vector x, b is a constant term also called bias or intercept, and f is an activation function that applies the dot product w*x+b.

[0019] Furthermore, the multiple single neurons described in S3.2 are connected together to form a network model including three layers:

[0020] The first layer is the input layer. The number of nodes in the input layer is the number of sample features, where +1 is the bias node.

[0021] The second layer is the hidden layer, and the number of nodes is set manually or automatically selected by the system;

[0022] The third layer is the output layer, and the number of nodes is the number of features of the sample target.

[0023] Furthermore, the signal propagation process of the artificial neural network is:

[0024] The input layer inputs sample data;

[0025] The input layer is used as the input of the hidden layer nodes, and the output of the hidden layer nodes is calculated. If the hidden layer contains multiple layers, the input of the previous layer is used as the output of the next layer, and the output of the next layer is calculated;

[0026] The output of the hidden layer nodes is used as the input of the output layer to calculate the output of the output layer.

[0027] The present invention also provides an intelligent recognition system for website sensitive words, the detection system comprising:

[0028] Website text data full acquisition module: used in the website data collection stage to collect the original data of the website and collect the source code of the website in full real-time;

[0029] Website text data preprocessing module: used to preprocess the data after the website data collection is completed;

[0030] Website text sensitive word intelligent module: used to implement artificial neural network algorithm by building a neuron model to automatically and quickly identify sensitive word information in the text.

[0031] The present invention also provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a computer, the computer executes the steps of any of the above-mentioned methods for intelligently identifying sensitive words on a website.

[0032] By adopting the above technical solution, the present invention has the following beneficial effects:

[0033] The present invention uses an artificial neural network algorithm to perform iterative self-organizing feature extraction, can automatically identify sensitive information in combination with context, has high-precision recognition capabilities, and can automatically detect without human intervention, which is beneficial to improving the detection rate and accuracy of sensitive words on the website; using a streaming processing engine, it can maintain high throughput and low latency in the case of massive data, significantly improving detection efficiency and ensuring the timeliness of detection.

[0034] Send data distributed in different nodes to the same cache queue, realizing multi-point data collection and single-point data transmission.

[0035] According to data governance standards, reduce the proportion of dirty data in the entire data flow, maximize the accuracy rate, eliminate illegal information in the data, and ensure data security.

[0036] Replace the original sensitive word recognition method, abandon the text segmentation and phrase matching recognition method, and use artificial neural network modeling to recognize sensitive words to improve recognition accuracy and efficiency. At the same time, ensure that data is safe and reliable during the processing process to avoid security risks such as data leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of the intelligent identification method of website sensitive words of the present invention;

[0039] Figure 2A schematic diagram of signal propagation of a neural network of website sensitive words of the present invention;

[0040] Figure 3 Schematic diagram of the intelligent recognition system for website sensitive words of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Combination Figure 1 As shown, the present invention provides an intelligent recognition method for sensitive words on a website, the method comprising:

[0043] S1: Website text data collection: During the website data collection phase, the original data of the website is collected and the source code data entered into the website is collected in full real-time;

[0044] S2: Preprocessing of website text data. After the website data collection is completed, the data is preprocessed;

[0045] S3: Intelligent identification of sensitive words in website texts. By building a neuron model to implement an artificial neural network algorithm, it can automatically and quickly identify sensitive word information in the text.

[0046] Specifically, the source code data in S1 is obtained by setting the collection depth of the entered website data.

[0047] Specifically, the setting of the data collection depth is set by manual configuration during the collection process.

[0048] Specifically, the data described in S2 is pre-processed to ensure data security and protect the privacy of website data in accordance with the law. At the same time, the data is sent to a message queue for decoupled caching and multi-level consumption processing.

[0049] Legally, this means protecting the privacy of website data based on the "Cybersecurity Law", "Civil Code", "Data Security Law" (draft for comments), "Personal Information Protection Law" (draft for comments), etc., which have been intensively promulgated by the country in recent years.

[0050] Specifically, the artificial neural network in S3 is composed of a neuron model; the neuron model includes:

[0051] Construct an operation unit, and use the operation unit to construct neurons. The information processing network composed of many neurons has a parallel distribution structure. Each neuron has a single output and can be connected to other neurons. There are multiple output connection methods, and each connection corresponds to a connection weight coefficient;

[0052] Connect multiple single neurons together to form a network model.

[0053] Specifically, the output of the operation unit described in S3.1 is the function hw,b(x): hw,b(x) depends on two parameters w and b and the input x, w is a weight vector, w T is the transpose of the weight vector w, x is an input vector containing a series of weights w, w*x represents the dot product of the weight vector w and the input vector x, b is a constant term also called bias or intercept, and f is an activation function that applies the dot product w*x+b.

[0054] Specifically, the multiple single neurons described in S3.2 are connected together to form a network model including three layers:

[0055] The first layer is the input layer. The number of nodes in the input layer is the number of sample features, where +1 is the bias node.

[0056] The second layer is the hidden layer, and the number of nodes is set manually or automatically selected by the system;

[0057] The third layer is the output layer, and the number of nodes is the number of features of the sample target.

[0058] Specifically, the signal propagation process of the artificial neural network is:

[0059] The input layer inputs sample data;

[0060] The input layer is used as the input of the hidden layer nodes, and the output of the hidden layer nodes is calculated. If the hidden layer contains multiple layers, the input of the previous layer is used as the output of the next layer, and the output of the next layer is calculated;

[0061] The output of the hidden layer nodes is used as the input of the output layer to calculate the output of the output layer.

[0062] See also Figure 2 , the signal propagation process of the neural network is as follows:

[0063] 1) Input sample data.

[0064] 2) The input layer is used as the input of the hidden layer nodes, and the output of the hidden layer nodes is calculated. If the hidden layer contains multiple layers, the input of the previous layer is used as the output of the next layer, and the output of the next layer is calculated, and so on.

[0065] 3) Use the output of the hidden layer nodes as the input of the output layer and calculate the output of the output layer.

[0066] The above neural network has three layers, the first layer is denoted as L1, the second layer is denoted as L2, and the third layer is denoted as L3. The parameters of the neural network include the weights and biases of the first and second layers.

[0067] a i (k) represents the output value of the i-th unit in the k-th layer. When k = 1, a i (1) =x i , which is the i-th input value (i-th eigenvalue). Based on the known w and b parameters, the neural network can calculate the output result according to the function hw,b(x). The calculation of the neural network is as follows:

[0068] Input of the first layer nodes:

[0069]

[0070] Output of the second layer nodes:

[0071]

[0072] Output of the third layer node:

[0073]

[0074] Simplify the above formula in vector form and use z i (k) represents the weighted sum of the inputs of the i-th unit in the k-th layer, then a i (k) =f(z i (k) ), where z i (k) It can be expressed by a vector. The above formula can be expressed more concisely as:

[0075] z (2) =w (1) x+b (1)

[0076] a (2) =f(z (2) )

[0077] z (3) =w (2) a (2) +b (2)

[0078] h w，b (x) = a (3) =f(z (3) )

[0079] Similarly, the activation value a of the k+1th layer is (k+1) , you can get it by following the steps below:

[0080] z( k+1) )=w (k) a (k) +b (i)

[0081] a (k+1) =f(z (k+1) )

[0082] The sample error training function is defined as:

[0083]

[0084] For a data set of m solar calendars, the overall error function is defined as:

[0085]

[0086] The first term is the mean square error, and the second term is L2 regularization, which aims to reduce the magnitude of the weights and prevent overfitting.

[0087] To solve (w, b) so that the overall error J(w, b) is minimized, the commonly used method is the gradient descent method. Each iteration in the gradient descent method updates w and b according to the following formula:

[0088]

[0089] Where a is the learning efficiency. The key step is to calculate the partial derivatives, which are calculated using the back propagation algorithm.

[0090] in, and is the partial derivative of the error function J(w, b; x, y) of a single sample (x, y). Then the partial derivative of the overall error is:

[0091]

[0092] 1) For a single sample (x, y), according to the forward propagation of the signal, the output of each layer of the neural network is calculated, including the output value of hw,b(x), and finally L2, L3, ... until the end of the output layer.

[0093] 2) For the nth k For each output unit i of the layer (output layer), the residual value is calculated according to the following formula:

[0094]

[0095] 3) For k = n k-1 , n k-2 , ..., the residual calculation method of the i-th node in the k-th layer is as follows:

[0096]

[0097] 4) The partial derivative calculation method is as follows:

[0098]

[0099] In summary, the present invention uses an artificial neural network algorithm to perform iterative self-organizing feature extraction, can automatically identify sensitive information in combination with the context, has high-precision recognition capabilities, and can automatically detect without human intervention, which is beneficial to improving the detection rate and accuracy of sensitive words on the website; using a streaming processing engine, it can maintain high throughput and low latency in the case of massive data, significantly improves detection efficiency, and ensures the timeliness of detection.

[0100] In another embodiment, see Figure 3 , an intelligent recognition system for sensitive words on a website, wherein the detection system comprises:

[0101] Website text data full acquisition module: used in the website data collection stage to collect the original data of the website and collect the source code of the website in full real-time;

[0102] Website text data preprocessing module: used to preprocess the data after the website data collection is completed;

[0103] Website text sensitive word intelligent module: used to implement artificial neural network algorithm by building a neuron model to automatically and quickly identify sensitive word information in the text.

[0104] In yet another embodiment, a computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a computer, the computer executes the steps of the method for intelligently identifying sensitive words on a website as described above.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligently identifying sensitive words on a website, characterized in that: The method comprises: S1: Website text data collection: During the website data collection phase, the original website data is collected and the source code data entered into the website is collected in full real-time; S2: Preprocessing of website text data. After the website data collection is completed, the data is preprocessed; S3: Intelligent identification of sensitive words in website texts. By building a neuron model to implement an artificial neural network algorithm, it can automatically and quickly identify sensitive word information in the text.

2. The intelligent identification method of website sensitive words according to claim 1 is characterized in that: The source code data in S1 is obtained by setting the collection depth of the entered website data.

3. The intelligent identification method of website sensitive words according to claim 2 is characterized in that: The data collection depth is set by manual configuration during the collection process.

4. The intelligent identification method of website sensitive words according to claim 1 is characterized in that: The data preprocessing described in S2 is centered around data security, and the privacy of website data is protected in accordance with the law. At the same time, the data is sent to the message queue for decoupled caching and multi-level consumption processing.

5. The intelligent identification method of website sensitive words according to claim 1 is characterized in that: The artificial neural network in S3 is composed of a neuron model; the neuron model includes: Construct an operation unit, and use the operation unit to construct neurons. The information processing network composed of many neurons has a parallel distribution structure. Each neuron has a single output and can be connected to other neurons. There are multiple output connection methods, and each connection corresponds to a connection weight coefficient; Connect multiple single neurons together to form a network model.

6. The intelligent identification method of website sensitive words according to claim 5, characterized in that: The output of the operation unit described in S3.1 is the function hw , b(x): hw , b(x) depends on two parameters w and b and the input x, w is a weight vector, w T is the transpose of the weight vector w, x is an input vector containing a series of weights w, w*x represents the dot product of the weight vector w and the input vector x, b is a constant term also called bias or intercept, and f is an activation function applied to the result of the dot product w*x+b.

7. The intelligent identification method of website sensitive words according to claim 5, characterized in that: The multiple single neurons described in S3.2 are connected together to form a network model consisting of three layers: The first layer is the input layer. The number of nodes in the input layer is the number of sample features, where +1 is the bias node. The second layer is the hidden layer, and the number of nodes is set manually or automatically selected by the system; The third layer is the output layer, and the number of nodes is the number of features of the sample target.

8. The intelligent identification method of website sensitive words according to claim 7, characterized in that: The signal propagation process of the artificial neural network is: The input layer inputs sample data; The input layer is used as the input of the hidden layer nodes, and the output of the hidden layer nodes is calculated. If the hidden layer contains multiple layers, the input of the previous layer is used as the output of the next layer, and the output of the next layer is calculated; The output of the hidden layer nodes is used as the input of the output layer to calculate the output of the output layer.

9. An intelligent recognition system for sensitive words on a website, characterized in that: The detection system comprises: Website text data full acquisition module: used in the website data collection stage to collect the original data of the website and collect the source code of the website in full real-time; Website text data preprocessing module: used to preprocess the data after the website data collection is completed; Website text sensitive word intelligent module: used to implement artificial neural network algorithm by building a neuron model to automatically and quickly identify sensitive word information in the text.

10. A computer-readable storage medium storing computer program instructions, wherein when the computer program instructions are executed by a computer, the computer executes the steps of the method for intelligently identifying sensitive words on a website as described in any one of claims 1 to 8.

Images