A rule-based website detection method and system

Abstract

The present application belongs to the technical field of network security, and provides a website detection method and system based on a rule base. The method controls a browser to render a website, injects a piece of JavaScript script into the rendered website, detects and identifies harmful features exhibited by the website based on a rule base, pre-processes the website that hits a rule through a strategy in the rule base, and reports the detection result to a harmful information detection and perception platform for further processing by an operation and maintenance personnel, so as to solve the problem that a harmful website can easily escape content review due to the incomplete sensitive word library and text confusion in sensitive word detection technology.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a rule-based website detection method and system. Background Technology

[0002] Current technologies generally rely on sensitive word databases to detect the presence of sensitive words such as pornography, gambling, politically sensitive, and terroristic terms in website content. This requires a comprehensive sensitive word database. On one hand, techniques for detecting and matching harmful content using sensitive words suffer from lag and passivity, often collecting information only after it has been generated. After some common sensitive words are effectively blocked, new terms corresponding to them become new code words. Only after extensive media exposure are these new terms widely collected into sensitive word databases by the industry. On the other hand, websites often easily evade content review by using complex variations of text, such as meaningless characters or garbled text. For example, variations of "add WeChat" include "add WeChat" and "╇v".

[0003] Network operators must fulfill their network security protection obligations when conducting business and service activities. Value-added telecommunications service providers need to monitor and identify harmful information on websites accessed by their customers, promptly detect security risks, and take corresponding measures to ensure compliant website access. However, technologies that use sensitive word detection to match harmful content suffer from incomplete sensitive word databases and textual obfuscation, allowing harmful websites to easily evade content review and making it impossible to detect security risks and take appropriate measures in a timely manner. Summary of the Invention

[0004] This invention provides a website detection method and system based on a rule base, aiming to solve the problem that harmful websites can easily evade content review due to incomplete sensitive word databases and textual confusion in sensitive word detection technology.

[0005] To achieve the above objectives, the present invention provides a website detection method based on a rule base, the method comprising:

[0006] The program controls the browser to render the website;

[0007] The program injects a JavaScript script into the rendered website and detects and identifies harmful features displayed by the website based on a rule base.

[0008] The program preprocesses websites that match the rules using policies from the rule base;

[0009] The program reports the detection results to the harmful information detection and perception platform.

[0010] Furthermore, the program controls the browser to render the website, including:

[0011] The program uses an open-source browser testing framework to automatically launch a headless browser to render the website.

[0012] Furthermore, the program, by using an open-source browser testing framework, automatically launches a headless browser to render the website, including:

[0013] After rendering the website, the program first obtains the website content and uses sensitive word detection technology to identify harmful information.

[0014] Furthermore, the program injects a JavaScript script into the rendered website, which includes:

[0015] The JavaScript script contains a rule base and execution engine for detecting harmful content, and is capable of detecting harmful characteristics exhibited by the website.

[0016] Furthermore, the rule base for detecting harmful content includes:

[0017] Detect and identify text variations in website title and meta tags;

[0018] Detect and identify features of websites that display harmful information using only image pages;

[0019] Detect and identify features of websites that use iframes to evade content censorship.

[0020] It detects and identifies features of frequent changes in the content of website title and meta tags.

[0021] Furthermore, the program preprocesses the websites that match the rules using policies in the rule base, including:

[0022] The program obtains the corresponding text content from pure image websites by calling OCR image recognition and then uses sensitive words for detection.

[0023] For rules with an extremely low false positive rate, the program directly shuts down websites that violate the rule and sends an email notification to the security manager.

[0024] To achieve the above objectives, the present invention also provides a rule-based website detection system, the system comprising:

[0025] The rendering module is used to control the browser to render the website;

[0026] The detection module is used to inject a JavaScript script into the rendered website and detect and identify harmful features of the website based on a rule base.

[0027] The preprocessing module is used to preprocess websites that match the rules using policies from the rule base.

[0028] The reporting module is used to summarize the detection results and report them to the harmful information detection and perception platform.

[0029] Furthermore, the JavaScript script includes a rule base and execution engine for detecting harmful content, used to detect harmful characteristics exhibited by the website.

[0030] Furthermore, the rule base for detecting harmful content is used to detect and identify features of text variations in the website's title and meta tags;

[0031] Detect and identify features of websites that display harmful information using only image pages;

[0032] Detect and identify features of websites that use iframes to evade content censorship.

[0033] It detects and identifies features of frequent changes in the content of website title and meta tags.

[0034] Furthermore, the policies in the rule base include:

[0035] For websites that are purely image-based, the corresponding text content is obtained by calling OCR image recognition and then sensitive words are detected.

[0036] For rules with an extremely low false positive rate, the program directly shuts down the websites that violate the rule and sends an email notification to the security manager.

[0037] This invention provides a website detection method and system based on a rule-based database. The method controls the browser to render a website, injects a JavaScript script into the rendered website, detects and identifies harmful features displayed by the website based on a rule-based database, preprocesses websites that match the rules using strategies in the rule-based database, and summarizes and reports the detection results to a harmful information detection and awareness platform for further processing by maintenance personnel. This addresses the problem in sensitive word detection technology where incomplete sensitive word databases and word obfuscation allow harmful websites to easily evade content review. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a rule-based website detection method provided in an embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and effects of this invention clearer, specific embodiments of the invention are described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0042] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0043] It should also be understood that the term "and / or" as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0044] Embodiments of this application provide a rule-based website detection method, the method comprising:

[0045] The program controls the browser to render the website;

[0046] The program injects a JavaScript script into the rendered website and detects and identifies harmful features displayed by the website based on a rule base.

[0047] The program preprocesses websites that match the rules using policies from the rule base;

[0048] The program reports the detection results to the harmful information detection and perception platform.

[0049] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0050] Please see Figure 1 , Figure 1 This is a flowchart illustrating a rule-based website detection method provided in an embodiment of this application.

[0051] S10, the program controls the browser to render the website;

[0052] Furthermore, the program uses an open-source browser testing framework to automatically launch a headless browser to render the website;

[0053] After rendering the website, the program first obtains the website content and uses sensitive word detection technology to identify harmful information.

[0054] In this embodiment, the browser is automatically controlled by a program; in this example, Chrome is used to simulate website rendering. The program can directly use open-source browser testing frameworks such as Selenium or Puppeteer to automatically launch a headless Chrome browser and automatically open the specified website for rendering. After the website rendering is complete, a notification event is sent to prompt the program for further processing. There are two reasons for using Chrome to render the website instead of the program directly sending a native GET request:

[0055] First: After some websites have their webpage code tampered with by hackers, they will use JavaScript to redirect visitors to pornographic and gambling websites when they browse the website. Therefore, it is necessary to use headless Chrome (a headless browser) to simulate the process of users browsing webpages.

[0056] Second: The rule base in this case is implemented using JavaScript, so it needs to run in headless Chrome.

[0057] S20, the program injects a JavaScript script into the rendered website and detects and identifies harmful features displayed by the website based on the rule base;

[0058] The JavaScript script contains a rule base and execution engine for detecting harmful content, and is capable of detecting harmful characteristics exhibited by the website.

[0059] Furthermore, the rule base for detecting harmful content includes: detecting and identifying features of text variations in website title tags and meta tags.

[0060] In this embodiment, because hackers often use HTML symbol entities for obfuscation to evade sensitive word detection when tampering with webpage code, such as the HTML symbol entity for "gambling":

[0061] "Lottery;" or "Lottery;", after being rendered by the browser, what the user finally sees is the two characters "Lottery". After analyzing a large amount of website construction data, normal website codes rarely use HTML symbol entities as the title of the website. Therefore, the existence of HTML symbol entities in the title and meta tags of the website can be used as a harmful feature. Similarly, when there are confusions between full-width and half-width characters in the title and meta tags of the website (such as variants of "av": "a∨", "av"), etc., it can also be used as a harmful feature. Based on this feature, relevant rules are formulated. By executing the engine to detect the web page content, it gets rid of the dependence on the sensitive word library and avoids the problem of incomplete collection of sensitive words.

[0062] Furthermore, the rule library for detecting harmful content includes: detecting and identifying the feature of using pure picture pages to display harmful information on the website.

[0063] In this embodiment, in order to avoid sensitive word detection, some harmful websites display harmful information in the form of pictures. Considering the problem of execution speed, traditional sensitive word detection technologies are difficult to perform OCR image recognition on all web pages to convert them into text and then detect. Through big data analysis of the discovered harmful websites, we found that most harmful websites that display in the form of pure pictures will fill the visible area of the browser window with pictures. It is difficult to distinguish whether the web page is a pure picture or a normal html page in appearance, which easily makes users relax their vigilance and actively click on the fake button on the picture, achieving the purpose of jumping to pornographic and gambling websites. We can formulate relevant rules based on the features of "pure picture" and "filling the visible area of the browser window", according to the number and size of the pictures in the web page and their proportion to the visible area of the browser window, combined with counting the number of words in the web page. For web pages that hit the rules, the OCR image recognition interface will be further called to identify the words and patterns on the pictures, so as to more accurately judge harmful information. This solves the problem that traditional sensitive word detection technologies call OCR image recognition for all web pages without discrimination, resulting in slow execution speed.

[0064] Furthermore, the rule library for detecting harmful content includes: detecting and identifying the feature of using the iframe framework to load content to evade content censorship on the website.

[0065] In this embodiment, if a native GET request is used directly to obtain the webpage content without browser rendering, the content within the iframe will not be retrieved. Even if browser rendering is used, if only the entire webpage data is retrieved, the content within the iframe embedded in the webpage will not be retrieved. Hackers exploit this principle to embed iframes into compromised websites, filling the visible area of ​​the browser window and directing the iframes to a malicious website. We can formulate relevant rules based on the characteristics of "the presence of iframes" and "filling the visible area of ​​the browser window," combined with the number of iframes and their proportion of the visible area of ​​the browser window. For webpages that match the rules, we can delve deeper to obtain the content within the iframes, or even have the program automatically take a screenshot of the entire webpage and call OCR image recognition.

[0066] Furthermore, the rule base for detecting harmful content includes: detecting and identifying features of frequent changes in the content of website title tags and meta tags.

[0067] In this embodiment, hackers continuously change the specific names of pornographic and gambling platforms in the website's title and meta tags to attract search engines like Baidu and Google, thereby increasing the search popularity and ranking of these platforms. Normal websites do not frequently change their title and meta tags. Therefore, we can formulate relevant rules based on the frequency of title and meta tag changes to detect the presence of harmful information on a website.

[0068] S30, the program preprocesses the websites that hit the rules using policies in the rule base;

[0069] Furthermore, the program preprocesses the websites that match the rules using policies in the rule base, including:

[0070] The program obtains the corresponding text content from pure image websites by calling OCR image recognition and then uses sensitive words for detection.

[0071] For rules with an extremely low false positive rate, the program directly shuts down the websites that violate the rule and sends an email notification to the security manager.

[0072] S40, the program summarizes the detection results and reports them to the harmful information detection and perception platform.

[0073] In this embodiment, the program reports and summarizes the detection results of harmful information to the "Sanwu Host Harmful Information Monitoring and Sensing Platform," where on-duty personnel further analyze and process the information, providing data support for the formulation of new detection rules.

[0074] To achieve the above objectives, the present invention also provides a rule-based website detection system, the system comprising:

[0075] The rendering module is used to control the browser to render the website;

[0076] The detection module is used to inject a JavaScript script into the rendered website and detect and identify harmful features of the website based on a rule base.

[0077] The preprocessing module is used to preprocess websites that match the rules using policies from the rule base.

[0078] The reporting module is used to summarize the detection results and report them to the harmful information detection and perception platform.

[0079] Furthermore, the JavaScript script includes a rule base and execution engine for detecting harmful content, used to detect harmful characteristics exhibited by the website.

[0080] Furthermore, the rule base for detecting harmful content is used to detect and identify features of text variations in the website's title and meta tags;

[0081] Detect and identify features of websites that display harmful information using only image pages;

[0082] Detect and identify features of websites that use iframes to evade content censorship.

[0083] It detects and identifies features of frequent changes in the content of website title and meta tags.

[0084] Furthermore, the policies in the rule base include:

[0085] For websites that are purely image-based, the corresponding text content is obtained by calling OCR image recognition and then sensitive words are detected.

[0086] For rules with an extremely low false positive rate, the program directly shuts down the websites that violate the rule and sends an email notification to the security manager.

[0087] This invention provides a website detection method and system based on a rule-based database. The method controls the browser to render a website, injects a JavaScript script into the rendered website, detects and identifies harmful features displayed by the website based on a rule-based database, preprocesses websites that match the rules using strategies in the rule-based database, and summarizes and reports the detection results to a harmful information detection and awareness platform for further processing by maintenance personnel. This addresses the problem in sensitive word detection technology where incomplete sensitive word databases and word obfuscation allow harmful websites to easily evade content review.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A website detection method based on a rule base, characterized in that, The method includes: The program controls the browser to render the website; The program injects a JavaScript script into the rendered website and detects and identifies harmful features displayed by the website based on a rule base; wherein, the JavaScript script injected into the rendered website includes: The JavaScript script contains a rule base and execution engine for detecting harmful content, and is capable of detecting harmful characteristics exhibited by the website; The rule base for detecting harmful content includes: Detect and identify text variations in website title and meta tags; Detect and identify features of websites that display harmful information using only image pages; Detect and identify features of websites that use iframes to evade content censorship. Detect and identify features that frequently change the content in the website's title and meta tags; The program preprocesses websites that match the rules using policies from the rule base; The program reports the detection results to the harmful information detection and perception platform.

2. The method as described in claim 1, characterized in that, The program controls the browser to render the website, including: The program uses an open-source browser testing framework to automatically launch a headless browser to render the website.

3. The method as described in claim 2, characterized in that, The program, using an open-source browser testing framework, automatically launches a headless browser to render the website, including: After rendering the website, the program first obtains the website content and uses sensitive word detection technology to identify harmful information.

4. The method as described in claim 1, characterized in that, The program preprocesses websites that match the rules using policies from the rule base, including: The program obtains the corresponding text content from pure image websites by calling OCR image recognition and then uses sensitive words for detection. For rules with an extremely low false positive rate, the program directly shuts down websites that violate the rule and sends an email notification to the security manager.

5. A rule-based website detection system, characterized in that, The system includes: The rendering module is used to control the browser to render the website; A detection module is used to inject a JavaScript script into the rendered website and detect and identify harmful features displayed by the website based on a rule base. The detection module includes a rule base for detecting harmful content and an execution script within the JavaScript script, used to detect harmful features displayed by the website. The rule base for detecting harmful content is used to detect and identify features of text variations in website title tags and meta tags; Detect and identify features of websites that display harmful information using only image pages; Detect and identify features of websites that use iframes to evade content censorship. Detect and identify features that frequently change the content in the website's title and meta tags; The preprocessing module is used to preprocess websites that match the rules using policies from the rule base. The reporting module is used to report the detection results to the harmful information detection and perception platform.

6. The system as described in claim 5, characterized in that, The policies in the rule base include: For websites that are purely image-based, the corresponding text content is obtained by calling OCR image recognition and then sensitive words are detected. For rules with an extremely low false positive rate, the program directly shuts down the websites that violate the rule and sends an email notification to the security manager.

Images