CMS identification method based on limited request packet detection and multi-feature detection

Abstract

The CMS identification method based on limited request packet detection and multi-feature detection belongs to the technical field of network security. The CMS identification method based on limited request packet detection and multi-feature detection solves the problems of reduced scanning speed, inaccurate detection results and existing vulnerabilities in the CMS identification process as the feature library increases. The CMS identification method based on limited request packet detection and multi-feature detection comprises the following steps: obtaining the homepage content of a detection target through an http request sent to the homepage url; crawling the homepage content and constructing multiple scanning content regions and feature regions, and constructing a complete scanning context; using the added CMS fingerprint library record file to scan and match the context through a rule record mode, and generating an accurate scanning identification result. The scanning range is standardized, and the scanning is performed within a limited range, so that the scanning speed will not decrease as the feature library increases.

Description

Technical Field

[0001] The CMS identification method based on limited request packet detection and multi-feature detection of this invention belongs to the field of network security technology. Background Technology

[0002] A Content Management System (CMS) is a server-side programming language used to manage and maintain a website's sections, content, and templates. A typical CMS website comprises multiple CMS programs implementing different functions. Current CMS identification primarily relies on fingerprint databases, which store a large number of CMS fingerprints. By extracting the CMS fingerprint of a webpage to be identified and comparing it with fingerprints in the database, the CMS category of that webpage can be determined.

[0003] Currently, most methods for CMS identification in the cybersecurity industry use single-feature or limited-feature detection techniques, such as homepage source code, homepage header, static file content, and static file MD5 hashes. Feature matching uses regular expressions or strict equality checks. However, existing detection methods still have the following problems:

[0004] 1. As the number of feature records in the feature library increases, the number of probe data packets sent by the detection engine will also increase. When the number of feature values ​​is too large, on the one hand, the scanning and recognition speed will be reduced, and on the other hand, excessive packet sending will cause excessive pressure on the detected website, affecting the normal operation of business. At the same time, if the detected website has security equipment protection, it will increase the probability of being blocked by security equipment, resulting in inaccurate detection results.

[0005] 2. The feature-based equality or regular expression matching methods have a high false negative rate and cannot effectively detect secondary development or feature-modified CMS systems. Summary of the Invention

[0006] To address the shortcomings of existing technologies and avoid issues such as reduced scanning speed, inaccurate detection results, and high vulnerability rates during feature matching as the feature library grows during CMS recognition, a CMS recognition method based on limited request packet probing and multi-feature detection is proposed.

[0007] A CMS identification method based on finite request packet detection and multi-feature detection is characterized by the following steps:

[0008] S1. Obtain homepage parsing: Obtain the homepage content of the target by sending an HTTP request to the homepage URL, and parse out all elements and referenced files contained in the homepage;

[0009] S2, crawl all contents and resource files parsed from the homepage content, sequentially grab, download and save, calculate the MD5 and MinHash of the files, and build multiple scanning content regions and feature regions;

[0010] S3, build a scanning context, combine the multiple scanning content regions and feature regions in S2 to build a complete scanning context;

[0011] S4, use the added CMS fingerprint library record file to scan and match the context by rule recording, and generate accurate scanning recognition results.

[0012] Further, in the S1 step, the homepage content is the original data of the obtained homepage, and these data are usually html format content containing various elements and reference files required for building the homepage;

[0013] The obtained homepage content is parsed to identify and extract various elements and reference files contained therein, wherein the various elements include but are not limited to icons (favicon), titles, paragraphs, links and picture tags in the webpage; the reference files include but are not limited to resource files, JavaScript (.js) files, Cascading Style Sheets (.css) files and image files;

[0014] The image files include but are not limited to files with suffixes.jpg,.jpeg,.png,.ico,.gif.

[0015] Further, in the S2 step, the parsed html content is crawled to build multiple content regions and feature regions through content acquisition and feature calculation, which are as follows:

[0016] (1) Extract html tags to form a lable region;

[0017] (2) Extract the js, css, icon, jpg, jpeg, png, ico, gif file paths contained in the html to form a filelist region;

[0018] (3) Download all files in the filelist region to form a file region;

[0019] (4) Calculate the md5 of the file content in the filelist region to form an md5 region;

[0020] (5) The minihash of the file content in the filelist region is calculated to form a minihash region;

[0021] (6) The external link website in the html is extracted to form a link region;

[0022] Further, in the S3 step, a scanning context is constructed, which specifically includes: creating an initial scanning context framework, incorporating the multiple scanning content regions and feature regions constructed in the above step S2 into the scanning context, and then explicitly specifying that the scanning operation is specifically performed on the incorporated multiple scanning content regions and feature regions, and then constructing a scanning context associated with the S2 step.

[0023] Further, in the S4 step, the rule recording mode adopts a point system, and the matching is successful when the specified points are met; wherein the rule recording specifies the required points for a hit, which is specified by the @Match field in the record, the @Pattern in the record specifies a detection rule, a hit of a detection rule obtains the specified points, and no hit does not obtain points, the points of all hit detection rules are added, and the total points are greater than or equal to the points of @Match4, then the record hits and generates an accurate scanning recognition result;

[0024] Wherein, @Match 4 is the number of points in the specified hit rule, which is greater than or equal to 4.

[0025] The detection rule specified by @Pattern includes:

[0026] @Pattern 2 region: lable;

[0027] @Pattern 1 region: file;

[0028] @Pattern 1 region: filelist;

[0029] @Pattern 1 region: md5;

[0030] @Pattern 1 region: minihash;

[0031] @Pattern 1 region: link;

[0032] Wherein, @Pattern 2 means that the content that hits this detection rule can obtain 2 points, and @Pattern 1 means that the content that hits this detection rule can obtain 1 point.

[0033] The beneficial effects of the present application are: 1. First, by analyzing the html content, a plurality of scanning content regions and feature regions are constructed, and then a plurality of scanning content regions and feature regions are included in the scanning context framework to establish the relevant detection environment, and the scanning range is standardized, and then the scanning is performed within a limited range, which will not cause the increase of scanning elements with the increase of scanning records, and the possibility of reducing the scanning speed, on the other hand, the number of data packet transmissions when scanning the target is effectively controlled, the impact on the target is reduced, the risk of inaccurate detection results is reduced, and the protection measures of the target can also be effectively bypassed.

[0034] 2. A plurality of scanning content regions and feature regions are constructed, the scanning range is clear, a multi-feature multi-region scanning matching mode is formed, and the probability of missing reports is reduced, so that the detection result is more accurate. Compared with the prior art, the flexibility of the scanning record is improved, and the detection rate is improved. DETAILED DESCRIPTION

[0035] The embodiments of the present application will be described in detail below. The embodiments are implemented on the premise of the technical scheme of the present application, and detailed implementation methods and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0036] The present embodiment provides a technical scheme: a CMS identification method based on limited request packet detection and multi-feature detection includes the following steps:

[0037] S1, obtain the homepage analysis, obtain the homepage content of the detection target by sending the http request of the homepage url, and analyze all elements and referenced files contained in the homepage;

[0038] The homepage content is the original data of the obtained homepage, and these data are usually html format content, which contains various elements and referenced files required for constructing the homepage;

[0039] The obtained homepage content is analyzed, and various elements and referenced files contained therein are identified and extracted, wherein the various elements include but are not limited to icons (favicon), titles, paragraphs, links and picture tags in the webpage; the referenced files include but are not limited to resource files, JavaScript (.js) files, Cascading Style Sheets (.css) files and image files;

[0040] The image files include but are not limited to files with suffixes.jpg,.jpeg,.png,.ico,.gif.

[0041] Specifically, for https: / / www.example.comMonitoring detailed steps: send the scanning device to the url of the homepage content through the url of the homepage content, and obtain the homepage content, usually in html format; https: / / www.example.com's http Request, and obtain the homepage content, usually in html format;

[0042] S2, all contents and resource files parsed in the homepage content are crawled, downloaded and saved in sequence, MD5 and MinHash of the files are calculated, and multiple scanning content regions and feature regions are constructed;

[0043] The parsed html content is crawled and multiple scanning content regions and feature regions are constructed. Multiple content regions and feature regions are constructed through content acquisition and feature calculation, which are as follows:

[0044] (1) Extract the html label to form a lable region;

[0045] (2) Extract the js, css, icon, jpg, jpeg, png, ico, gif file path contained in the html to form a filelist region;

[0046] (3) Download all files in the filelist region to form a file region;

[0047] (4) Calculate the md5 of the file content in the filelist region to form a md5 region;

[0048] (5) Calculate the minihash of the file content in the filelist region to form a minihash region;

[0049] (6) Extract the external link website in the html to form a link region;

[0050] Specifically, the html content is parsed, and multiple scanning content regions and feature regions are constructed, which include:

[0051] (1) Extract the html label to form a lable region;

[0052] (2) Extract the js, css, icon, jpg, jpeg, png, ico, gif file path contained in the html to form a filelist region;

[0053] (3) Download all files in the filelist region to form a file region;

[0054] (4) Calculate the md5 of the file content in the filelist region to form the md5 region;

[0055] (5) Calculate the minihash of the file content in the filelist region to form the minihash region;

[0056] (6) Extract the external link website in the html to form the link region.

[0057] By constructing a plurality of scanning content regions and feature region combinations with a limited number of http probe packets, additional request packets are not required in the scanning and matching process, the scanning range in the CMS fingerprint identification process is regulated, and the scanning is performed within a limited range.

[0058] S3, construct a scanning context, combine the plurality of scanning content regions and feature regions in S2 to construct a complete scanning context;

[0059] Construct a scanning context, which specifically includes: creating an initial scanning context framework, incorporating the plurality of scanning content regions and feature regions constructed in the above step S2 into the scanning context, and then explicitly performing the scanning operation on the incorporated plurality of scanning regions, and then constructing a scanning context associated with the S2 step.

[0060] Specifically, an initial scanning context framework is created, and the plurality of scanning content regions and feature regions constructed in the above step S2, including the lable region, the filelist region, the file region, the md5 region, the minihash region and the link region, are incorporated into the initial scanning context framework, and then the scanning operation is explicitly performed on the incorporated plurality of scanning content regions and feature regions, and then a scanning context associated with the S2 step is constructed, so that the scanning speed will not decrease with the increase of the feature records in the feature library, and the problem of inaccurate detection results and false negatives is avoided.

[0061] S4, use the added CMS fingerprint library record file to scan and match the context by a rule record method, and generate an accurate scanning and identification result.

[0062] The rule record mode adopts the integral system, and the specified integral is met to match successfully; wherein the record specifies the required integral of hitting, which is specified by the @Match field in the record, the @Pattern in the record specifies a detection rule, and the integral of hitting a detection rule is obtained, and no integral is obtained without hitting, the integral of all hit detection rules is added, and the total integral is greater than or equal to the integral of @Match 4, then the record hits and generates an accurate scanning recognition result;

[0063] Wherein, @Match 4 is the integral number in the specified hit rule, which is greater than or equal to 4.

[0064] Wherein, the detection rule specified by @Pattern includes:

[0065] @Pattern 2 region: lable;

[0066] @Pattern 1 region: file;

[0067] @Pattern 1 region: filelist;

[0068] @Pattern 1 region: md5;

[0069] @Pattern 1 region: minihash;

[0070] @Pattern 1 region: link;

[0071] Wherein, @Pattern 2 means that the content hitting this detection rule can obtain 2 integrals, and @Pattern 1 means that the content hitting this detection rule can obtain 1 integral.

[0072] The record specifies the required integral of hitting, which is specified by the @Match field in the record, the @Pattern in the record specifies a detection rule, and the integral of hitting the rule is obtained, and no integral is obtained without hitting, the integral of all hit rules is added, and the total integral is greater than or equal to the integral in @Match, then the record hits, and an accurate scanning recognition result is generated.

[0073] Specifically, the record example of the embodiment is:

[0074] @Match 4

[0075] @Pattern 2" "region:lable;

[0076] @Pattern 1 "xxxs"!path "xxx.js"!region:file;

[0077] @Pattern 1 "ruoyi.css"!region:filelist;

[0078] @Pattern 1 "2221f431e845d608b9bb71c07c7a6a18"!path ". / js / index.js"!region:md5;

[0079] @Pattern 1 "11111"!path ". / js / index.js"!similarity 0.8!region:minihash;

[0080] @Pattern 1 "http: / / xxx.com"!region:link;

[0081] Using the added CMS fingerprint library record file to scan and compare with the above built context, in the record, @Pattern2 " "!region:lable

[0082] matching " in html lable tag If hit, get 2 points, if not hit, get no points.

[0083] @Pattern 1 "xxxs"!path "xxx.js"!region:file

[0084] In the xxx.js content in the file region, match "xxxs", hit to get 1 point;

[0085] @Pattern 1 "ruoyi.css"!region:filelist

[0086] In the filelist region, match whether it contains the ruoyi.css file, hit to get 1 point;

[0087] @Pattern 1 "2221f431e845d608b9bb71c07c7a6a18"!path ". / js / index.js"!region:md5

[0088] In the md5 region, match the md5 of ". / js / index.js" file, if it is

[0089] "2221f431e845d608b9bb71c07c7a6a18", hit to get 1 point;

[0090] @Pattern 1 "11111"!path ". / js / index.js"!similarity 0.8!region:minihash

[0091] In the minhash region, match the minihash value of ". / js / index.js" and "11111", if the similarity is greater than or equal to 0.8, hit to get 1 point;

[0092] @Pattern 1 "http: / / xxx.com"!region:link

[0093] In the link region, match whether it contains the "http: / / xxx.com" external link, hit to get 1 point; Add all the points of the above hits, if it is greater than or equal to the point number 4 specified in @Match, the rule hits, and then output the accurate scanning recognition result.

[0094] The above embodiments are only illustrative of the principles of the present application and its efficacy, and are not intended to limit the present application. Any modification or change made by any person skilled in the art without departing from the spirit and scope of the present application shall be covered by the claims of the present application.

Claims

1. A CMS identification method based on finite request packet detection and multi-feature detection, characterized in that, Includes the following steps: S1. Obtain homepage parsing: Obtain the homepage content of the target by sending an HTTP request to the homepage URL, and parse out all elements and referenced files contained in the homepage; S2. Crawl all content and resource files parsed from the homepage, download and save them sequentially, calculate the MD5 and MinHash of the files, and construct multiple scan content regions and feature regions; S3. Construct the scanning context, specifically by combining the multiple scanning regions and feature regions from S2 to construct a complete scanning context; S4. Use the added CMS fingerprint database record file to scan and match the context using rule-based recording, and generate accurate scan and recognition results.

2. The CMS identification method based on finite request packet detection and multi-feature detection according to claim 1, characterized in that, In step S1, the homepage content is the raw data of the obtained homepage, which is usually in HTML format and contains various elements and referenced files required to build the homepage. The obtained homepage content is parsed to identify and extract various elements and referenced files. The various elements include icons, titles, paragraphs, links, and image tags in the webpage; the referenced files include resource files, JavaScript files, cascading style sheet files, and image files. The image files include files with the extensions .jpg, .jpeg, .png, .ico, and .gif.

3. The CMS identification method based on finite request packet detection and multi-feature detection according to claim 1, characterized in that, In step S2, the parsed HTML content is crawled, and multiple content regions and feature regions are constructed through content acquisition and feature calculation, as detailed below: (1) Extract HTML tags to form a label region; (2) Extract the file paths of js, css, icon, jpg, jpeg, png, ico, and gif contained in the html and construct a filelistregion; (3) Download all files in the filelist region to form a file region; (4) Calculate the MD5 hash of the file contents in the filelist region to form an MD5 region; (5) Calculate the minihash of the file content in the filelist region to form a minihash region; (6) Extract the URLs of external links in the HTML to form a link region.

4. The CMS identification method based on finite request packet detection and multi-feature detection according to claim 1, characterized in that, In step S3, constructing the scanning context specifically includes: creating an initial scanning context framework, incorporating the multiple scanning regions constructed in step S2 into the scanning context, thereby clarifying that the scanning operation is specifically performed on the multiple scan regions included, and then incorporating the calculated features into the scanning context based on the already included scan regions, thereby constructing a scanning context associated with step S2.

5. The CMS identification method based on finite request packet detection and multi-feature detection according to claim 1, characterized in that, In step S4, the rule recording method adopts an points system. A successful match is achieved if the specified points are met. The points required for a match are specified in the @Match field of the record. The @Pattern field of the record specifies a detection rule. A specified points are obtained for each detection rule matched. No points are obtained for each rule matched. The points of all matched detection rules are added together. If the total points are greater than or equal to the points of @Match 4, then a match is recorded and an accurate scanning recognition result is generated. In this context, @Match 4 specifies that the sum of the points in the hit rule must be greater than or equal to 4.

6. The CMS identification method based on finite request packet detection and multi-feature detection according to claim 5, characterized in that, The detection rules specified by @Pattern include: @Pattern 2 region:label; @Pattern 1 region:file; @Pattern 1 region:filelist; @Pattern 1 region:md5; @Pattern 1 region:minihash; @Pattern 1 region:link; Here, @Pattern 2 means that matching the content of this detection rule will earn 2 points, and @Pattern 1 means that matching the content of this detection rule will earn 1 point.