Intelligent mining method for webpage invaded and tampered by hacker

Abstract

The invention relates to an intelligent mining method for a webpage invaded and tampered by a hacker, which belongs to the field of network and information security, and comprises the processes of data preprocessing, domain name verification, keyword comparison, screening verification and the like. The intelligent mining method solves the problems that manual mining is low in efficiency, small in effect, large in workload, prone to errors and the like, manual mining is replaced with artificial intelligence, and manpower, material resources and financial resources are effectively saved; meanwhile, the method not only can be used for tampering detection of a single domain name, but also can realize intelligent mining of batch domain names and automatic report generation, and actual application results prove that the method can screen out 97% or above of normal domain names and position about 3% of abnormal domain names tampered by hackers, so that the method is suitable for large-scale popularization and application. The method has the characteristics of high excavation precision, good practicability, intelligence, high efficiency and the like.

Description

Technical Field

[0001] The invention belongs to the field of network and information security, and in particular relates to an intelligent mining method for web pages invaded and tampered by hackers. Background Art

[0002] In recent years, with the rapid development of new technologies and applications such as Internet technology, AI artificial intelligence, and ChatGPT, various websites, information systems, and servers are facing severe network security threats. Network security incidents involving counter-labeling, tampering, and implantation of dark links have occurred frequently, causing serious impacts on the unit's portal website.

[0003] Most of the mining for websites that have been invaded and tampered with by hackers is still mainly done manually, with the help of search platforms for auxiliary mining. Overall, it is still inefficient and ineffective, and there are problems such as high duplication of work, repeated errors and inability to automatically generate reports.

[0004] At present, in practice, human mining cannot solve the problem of locating abnormal website domain names that account for less than 3% from a large number of website domain names. The data generated by the search platform is delayed, the results are uncertain, and only certain categories of keywords can be searched. All of the above reasons lead to low efficiency and few results, and it is also impossible to achieve intelligent and efficient mining effects. Summary of the invention

[0005] In order to solve the above-mentioned problems, a convenient, practical, efficient, intelligent and reliable intelligent mining method is urgently needed. Therefore, the present invention proposes an intelligent mining method for web pages invaded and tampered by hackers; the method includes data preprocessing, domain name verification, keyword comparison, screening and verification and other processes; different from traditional manual mining, the present invention can screen out more than 97% of normal websites, locate the remaining approximately 3% of abnormal websites, and automatically generate reports at the same time, which is far more efficient than human mining, greatly improves mining efficiency, realizes the use of artificial intelligence instead of human mining, and has the advantages of saving manpower, material and financial resources.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] An intelligent mining method for hacked and tampered web pages, characterized by comprising the following steps:

[0008] Step S1: Data preprocessing: cleaning and reducing the acquired batch domain name data, removing empty or non-domain name data, and ensuring that the input data meets the requirements;

[0009] Step S2: Domain name verification, by simulating a Chrome browser to access the domain name website, obtain the parts of the webpage source code that are prone to hacker intrusion and tampering, and summarize and organize them;

[0010] Step S3: keyword comparison, the sorted and accumulated various black and gray industry keywords are formed into a library, and the weight is calculated based on the number of hit keywords. The higher the weight, the greater the possibility that the webpage has been tampered with. The website domain name, keyword, ICP registration number and other data that exceed the threshold will be retained;

[0011] Step S4: Screening and verification: The websites that exceed the threshold are further generated into intrusion and tampering clue reports through the automation module. The domain name reports of the websites with suspected problems are screened out, supplemented by brief manual confirmation, to form a set of tampering data clues and reports.

[0012] According to the above-mentioned intelligent mining method for hacked and tampered web pages, the data preprocessing process described in step S1 is: first, the data fields, formats, etc. in the data sets obtained from different sources are adjusted, and then the data is cleaned and standardized to ensure that the website domain name is alive and valid, and at the same time, important fields such as IP addresses, contacts, and contact information are supplemented.

[0013] According to the above-mentioned intelligent mining method for hacked and tampered web pages, the domain name verification process described in step S2 is as follows:

[0014] S2.1 must first simulate the Chrome browser, ensure the header structure is complete, set a time limit, and improve efficiency;

[0015] S2.2 Then, the source code obtained by simulating the Chrome browser for the surviving website domain name is adjusted to adjust the character encoding inconsistency problem, and the header tags, web page titles, descriptions and other parts of the source code that are prone to hacker intrusion and tampering are summarized and sorted out;

[0016] According to the above-mentioned intelligent mining method for hacked and tampered web pages, the keyword comparison process described in step S3 is as follows:

[0017] S3.1 first collides the statistically compiled high-frequency keyword library with the summarized and compiled data;

[0018] S3.2 Then the frequency and relevance of the keywords are analyzed and compared with the results, and the weight is calculated based on the number of keywords hit. The higher the weight, the greater the possibility that the webpage has been tampered with.

[0019] S3.3 The website domain names, keywords, ICP registration numbers and other data that exceed the threshold will be retained for subsequent clue report generation;

[0020] S3.4 If the verification result of S3.3 is normal, it only proves that there is no problem with the key parts that are easily invaded and tampered by hackers. It is necessary to check and verify the entire content of the webpage and repeat the above S3.1-3 to improve time efficiency.

[0021] According to the above-mentioned intelligent mining method for hacked and tampered web pages, the screening and verification process described in step S4 is as follows:

[0022] S4.1 First, prepare the corresponding automation module according to the requirements for generating tampering reports to ensure that the final report meets the requirements;

[0023] S4.2 Secondly, the website domain names that exceed the threshold are used by the automation module to generate intrusion and tampering clue reports to form a report set and data set suspected of being tampered with;

[0024] S4.3 Finally, for the report set of suspected problems that have been screened out, brief manual verification and confirmation are carried out, data is supplemented, and a small number of domain name websites that have been rectified are deleted to form clues and report sets of data tampering. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of an intelligent mining method for web pages invaded and tampered by hackers.

[0026] Figure 2 This is a structural diagram of an algorithm model of an intelligent mining method for web pages invaded and tampered by hackers according to the present invention.

[0027] Figure 3 This is a result diagram of an intelligent mining method for web pages invaded and tampered by hackers according to the present invention.

[0028] Figure 4 This is a diagram of an intelligently generated report of an intelligent mining method for web pages invaded and tampered by hackers according to the present invention. DETAILED DESCRIPTION

[0029] In order to enable ordinary researchers in the field to better understand the solution of the present invention, the solution of the present invention will be further described below in conjunction with the accompanying drawings and implementation cases.

[0030] Refer to the attached Figure 1-4 An intelligent mining method for hacked and tampered web pages is shown.

[0031] Figure 1 It is a flow chart of the method of the present invention, which generally introduces the two functions of single domain name verification and batch data intrusion and tampering mining, which has a wider range of applications and more comprehensive functions; it also shows the steps of data preprocessing, domain name verification, keyword comparison, screening and verification, as well as the steps involved in each function; Figure 2 It is an algorithm model structure diagram, which describes the function of the present invention in detail, as well as the specific steps:

[0032] Step S1: Correspondence Figure 2Data preprocessing: cleaning and reducing the acquired batch domain name data, that is, removing the non-compliant domain name data that is empty or does not end with .com, .cn, .top, .xyz, etc., to improve time efficiency;

[0033] Step S2: Correspondence Figure 2 Domain name verification in China, access the domain name webpage by simulating Chrome browser, headers = {'User-Agent':'Mozilla / 5.0(Windows NT 10.0;Win64;x64)AppleWebKit / 537.36(KHTML,like Gecko)Chrome / 91.0.4472.106Safari / 537.36'}, ensure the integrity of headers structure, use requests.get, set timeout = 30, response.status_code = 200, adjust the character encoding inconsistency problem, summarize and organize the web page tags, header tags, descriptions and other parts in the source code that are prone to hacker intrusion and tampering;

[0034] Step S3: Correspondence Figure 2 For keyword comparison, we first collide the sorted and accumulated high-frequency keyword library of black and gray industries with the high-incidence parts of intrusion and tampering, and calculate the weight according to the number of hit keywords. The higher the weight, the greater the possibility that the webpage has been tampered with; we remove the hit keywords and count them, and assign 10% weight to 1 hit, 30% weight to 2 hits, 50% weight to 3 hits, 80% weight to 4 hits, 90% weight to 5 hits, and 99% weight to no less than 6 hits. Finally, we adjust the weight threshold to control the accuracy of the hit. The experimental results show that the threshold of 30% can balance the number of suspected problematic domain names and the hit accuracy. The website domain names, keywords, ICP registration numbers and other data that exceed the threshold are retained for subsequent clue report generation;

[0035] Step S4: Screening and verification. The websites that exceed the threshold are further generated into intrusion and tampering clue reports through the automation module. Since the intelligent report generation method is adopted, the use of Python's docx library and selenium library is involved. The docx library is mainly used to generate docx file templates and adjust the generated report format; the selenium library is used to pull down the scroll bar of key parts by 50 pixels to take screenshots and adjust the screenshot position, etc., to form tampering data clues and report sets.

[0036] Figure 3 This is the result of the method of the present invention. At present, in the application verification, a total of about 260,000 domain name websites have been detected and mined to generate more than 1,800 reports on hacker tampering with websites, including Figure 3It mines 31,800 batch domain name website data in a certain area, and intelligently generates 196 clue report results charts through screening and verification.

[0037] Figure 4 The method of the present invention intelligently generates report diagrams, that is, intelligently generates reports in an automated manner to form a report set, which requires only a small amount of manual verification, avoids repeated report writing work, greatly reduces the consumption of manpower, material and financial resources, and improves the accuracy of generated reports.

Claims

1. An intelligent mining method for hacked and tampered web pages. Its characteristics include the following steps: Step S1: Data preprocessing, cleaning and reducing the acquired batch domain name data, removing empty or non-domain name data, and ensuring that the input data meets the requirements; Step S2: Domain name verification, by simulating a Chrome browser to access the domain name webpage, obtain the parts of the webpage source code that are prone to hacker intrusion and tampering, and summarize and organize them; Step S3: keyword comparison, the sorted and accumulated various black and gray industry keywords are formed into a library, and the weight is calculated based on the number of hit keywords. The higher the weight, the greater the possibility that the webpage has been tampered with. The website domain name, keyword, ICP registration number and other data that exceed the threshold will be retained; Step S4: Screening and verification: The websites that exceed the threshold are further generated into intrusion and tampering clue reports through the automation module. The website domain name reports with suspected problems are screened out, supplemented by brief manual verification, to form a set of tampering data clues and reports.

2. The intelligent mining method for hacked and tampered web pages according to claim 1, characterized in that: The data preprocessing process described in step S1 adjusts the data fields and formats obtained from different sources, and then cleans and regulates the data to ensure that the website domain name is alive and valid, while supplementing the missing important fields such as IP address, contact person, and contact information.

3. The intelligent mining method for hacked and tampered web pages according to claim 2, characterized in that: The domain name verification process described in step S2 is as follows: S2.1 must first simulate the Chrome browser, ensure the header structure is complete, set a time limit, and improve efficiency; S2.2 then uses a simulated Chrome browser to obtain the source code of the surviving website domain name, adjusts the character encoding inconsistency problem, and summarizes and organizes the source code header tags, web page titles, descriptions and other parts that are prone to hacker intrusion and tampering.

4. The intelligent mining method for hacked and tampered web pages according to claim 3 is characterized by: The keyword comparison process described in step S3 is as follows: S3.1 First, collate the statistically compiled high-frequency keyword library with the data compiled and organized in requirement 3; S3.2 Then analyze the frequency and relevance of the keywords, compare with the keyword library, and calculate the weight according to the number of hit keywords. The higher the weight, the greater the possibility that the webpage has been tampered with; S3.3 The website domain names, keywords, ICP registration numbers and other data that exceed the threshold will be retained for subsequent clue report generation; S3.4 If the verification result of S3.3 is normal, it only proves that there is no problem with the key parts that are easily invaded and tampered by hackers. It is necessary to check and verify the entire content of the webpage and repeat the above S3.1-3 to improve time efficiency.

5. The intelligent mining method for hacked and tampered web pages according to claim 4 is characterized by: The screening and verification process described in step S4 is as follows: S4.1 First, prepare the corresponding automation module according to the requirements for generating tampering reports to ensure that the final report meets the requirements; S4.2 Secondly, the domain names exceeding the threshold are used to generate intrusion and tampering clue reports through the automation module to form a report set and data set suspected of tampering; S4.3 Finally, for the report set of suspected problems that have been screened out, brief manual verification and confirmation are carried out, data is supplemented, and a small number of website domain names that have been rectified are deleted to form clues and report sets of data tampering.

Images