Intelligent classification method of webpage and validity verification method of security equipment of website

Abstract

The invention provides an intelligent classification method and device for webpages, a validity verification method and system for security equipment of websites and electronic equipment, and relates to the technical field of information security. The intelligent webpage classification method comprises the following steps: acquiring test content for testing a website, sending one or more webpage requests for testing to a server of the website according to the test content, and receiving response content which is returned by the server and corresponds to each webpage request for testing, according to the method, multi-dimensional feature extraction and multi-level intelligent classification of a static feature classification process, a preliminary classification process and a deep classification process are carried out on response content corresponding to each webpage request for testing, so that more accurate and more efficient response analysis and webpage classification can be provided to meet high-standard requirements of modern website testing and verification.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a method and device for intelligently classifying web pages, a method and system for validating the security equipment of a website, and electronic equipment. Background Art

[0002] As website complexity and security requirements continue to increase, maintenance personnel and security testers need to conduct extensive and in-depth testing of websites to ensure that they can correctly handle various types of requests, including normal user access, illegal and malformed requests, and potentially offensive requests. Traditional testing methods mainly rely on manually sending website requests and manually analyzing the web pages returned by website requests, which is not only inefficient but also prone to omissions and misjudgments. Therefore, this technical problem needs to be solved urgently. Summary of the invention

[0003] In view of the above problems, the present application is proposed to provide a method and device for intelligently classifying web pages, a method and system for validating the security equipment of a website, and an electronic device that overcomes the above problems or at least partially solves the above problems. The technical solution is as follows:

[0004] In a first aspect, a method for intelligently classifying web pages is provided, comprising:

[0005] Acquire test content for testing the website, wherein the test content includes one or more web page requests for testing;

[0006] According to the test content, one or more web page requests for testing are sent to the server of the website, and response content corresponding to each web page request for testing is returned by the server;

[0007] For the response content corresponding to each web page request for testing, a static feature classification process is executed, and the content in the response content indicating the request response status is used as a fingerprint to be matched with a preset fingerprint library. If a fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and web page types corresponding to each fingerprint;

[0008] If no fingerprint in the preset fingerprint library is matched, a current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and an initial fuzzy hash value calculated and stored by a preset fuzzy hash algorithm after the web page is accessed for the first time is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, it is determined that the actual web page type of the web page data in the response content is a first category page;

[0009] If the similarity is less than a first preset threshold, it is determined that the preliminary web page type of the web page data in the response content is a second type of page, and a preliminary classification process is further performed to extract a response status code, a response status code title, a response header, a response header length, a response body, and a response body length from the response content, and the extracted response status code, response status code title, response header, response header length, response body, and response body length are input into a trained preliminary classification model, and the trained preliminary classification model is used to predict the web page type of the web page data in the response content, and probability values ​​of multiple predicted web page types are output, and the probability values ​​of the multiple predicted web page types are sorted from large to small, and the two predicted web page types with the highest sorting are selected. If the difference between the probability values ​​of the two predicted web page types with the highest sorting is greater than or equal to a second preset threshold, the predicted web page type with the largest probability value is used as the actual web page type of the web page data in the response content;

[0010] If the difference between the probability values ​​of the top two predicted web page types is less than a second preset threshold, the deep classification process is further executed to extract the page loading order, loading time, loading content, and dynamic cookies from the response content, and the extracted page loading order, loading time, loading content, and dynamic cookies are input into the trained deep classification model. The trained deep classification model is used to predict the web page type of the web page data in the response content, and the predicted result of the web page type is output, and the web page type of the predicted result is used as the actual web page type of the web page data in the response content.

[0011] In a possible implementation, the web page types corresponding to each fingerprint in the preset fingerprint library include not found pages and server error pages; the first type of pages are normal pages, and the second type of pages are undetermined pages.

[0012] In one possible implementation, the preliminary classification model is trained by the following steps:

[0013] Build a preliminary classification model using the machine learning model random forest algorithm;

[0014] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a response status code of a response content corresponding to the web page request for the sample, a response status code title, a response header, a response header length, a response body, and a response body length;

[0015] The sample data is used to train the preliminary classification model to obtain a trained preliminary classification model.

[0016] In one possible implementation, the deep classification model is trained by the following steps:

[0017] Build a deep classification model through long short-term memory network LSTM;

[0018] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a page loading order, a loading time, a loading content, and a dynamic cookie of a response content corresponding to the web page request for the sample;

[0019] The deep classification model is trained using the sample data to obtain a trained deep classification model.

[0020] In a possible implementation, the test content also includes theoretical web page types corresponding to each web page request for testing;

[0021] After determining the actual web page type of the web page data in the response content, the method further includes:

[0022] From the test content, according to the theoretical web page types corresponding to the web page requests for each test, searching for the theoretical web page types of the web page data in the response content;

[0023] The actual web page type of the web page data in the response content is compared with the theoretical web page type, and a test report for testing the website is generated.

[0024] In a possible implementation, the theoretical web page type corresponding to each web page request for testing includes one or more of a normal page, a security device intercepted page, a server error page, a not found page, and an undetermined page.

[0025] In a second aspect, a method for verifying the validity of a security device of a website is provided, comprising:

[0026] Using any of the above-mentioned intelligent classification methods for web pages to obtain the actual web page type of the website web page;

[0027] If the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page, it is determined that the security device passes the validity verification.

[0028] In a third aspect, a device for intelligently classifying web pages is provided, comprising:

[0029] An acquisition unit, used to acquire test content for testing a website, wherein the test content includes one or more web page requests for testing;

[0030] A simulation test unit, used to send one or more web page requests for testing to a server of a website according to the test content, and receive response content corresponding to each web page request for testing returned by the server;

[0031] A static feature classification unit is used to perform a static feature classification process for the response content corresponding to each web page request for testing, and use the content in the response content indicating the request response status as a fingerprint to match it with a preset fingerprint library. If a fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and the web page type corresponding to each fingerprint; and

[0032] If no fingerprint in the preset fingerprint library is matched, the current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and the initial fuzzy hash value calculated and stored by the preset fuzzy hash algorithm after the first visit to the web page is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, the actual web page type of the web page data in the response content is determined to be a first-category page; if the similarity is less than the first preset threshold, the preliminary web page type of the web page data in the response content is determined to be a second-category page;

[0033] a preliminary classification unit, for further executing a preliminary classification process when the static feature classification unit determines that the preliminary web page type of the web page data in the response content is a second-category page, extracting a response status code, a response status code title, a response header, a response header length, a response body, and a response body length from the response content, inputting the extracted response status code, response status code title, response header, response header length, response body, and response body length into a trained preliminary classification model, predicting the web page type of the web page data in the response content using the trained preliminary classification model, outputting probability values ​​of a plurality of predicted web page types, sorting the probability values ​​of the plurality of predicted web page types from large to small, selecting the top two predicted web page types, and if the difference between the probability values ​​of the top two predicted web page types is greater than or equal to a second preset threshold, taking the predicted web page type with the largest probability value as the actual web page type of the web page data in the response content;

[0034] The deep classification unit is used to further execute the deep classification process when the difference between the probability values ​​of the two predicted web page types ranked first by the preliminary classification unit is determined to be less than a second preset threshold value, extract the page loading sequence, loading time, loading content, and dynamic cookies from the response content, input the extracted page loading sequence, loading time, loading content, and dynamic cookies into the trained deep classification model, use the trained deep classification model to predict the web page type of the web page data in the response content, output the predicted result of the web page type, and use the web page type of the predicted result as the actual web page type of the web page data in the response content.

[0035] In a possible implementation, the web page types corresponding to each fingerprint in the preset fingerprint library include not found pages and server error pages; the first type of pages are normal pages, and the second type of pages are undetermined pages.

[0036] In a possible implementation manner, the apparatus further includes a first training unit, configured to:

[0037] Train a preliminary classification model by following these steps:

[0038] Build a preliminary classification model using the machine learning model random forest algorithm;

[0039] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a response status code of a response content corresponding to the web page request for the sample, a response status code title, a response header, a response header length, a response body, and a response body length;

[0040] The sample data is used to train the preliminary classification model to obtain a trained preliminary classification model.

[0041] In a possible implementation manner, the apparatus further includes a second training unit, configured to:

[0042] Build a deep classification model through long short-term memory network LSTM;

[0043] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a page loading order, a loading time, a loading content, and a dynamic cookie of a response content corresponding to the web page request for the sample;

[0044] The deep classification model is trained using the sample data to obtain a trained deep classification model.

[0045] In a possible implementation, the test content further includes theoretical web page types corresponding to each web page request for testing; and the simulation test unit is further used to:

[0046] After the static feature classification unit, the preliminary classification unit or the deep classification unit determines the actual web page type of the web page data in the response content, searching the theoretical web page type of the web page data in the response content from the test content according to the theoretical web page type corresponding to each web page request for testing;

[0047] The actual web page type of the web page data in the response content is compared with the theoretical web page type, and a test report for testing the website is generated.

[0048] In a possible implementation, the theoretical web page type corresponding to each web page request for testing includes one or more of a normal page, a security device intercepted page, a server error page, a not found page, and an undetermined page.

[0049] In a fourth aspect, a validity verification system for a website security device is provided, comprising the intelligent classification device for web pages as described in any one of the above items, for intelligently classifying and obtaining the actual web page type of the website web pages;

[0050] The verification unit is used to determine that the security device passes the validity verification if the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page.

[0051] In a fifth aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above-mentioned methods for intelligent classification of web pages or the above-mentioned methods for verifying the validity of the security device of the website.

[0052] By means of the above-mentioned technical scheme, the embodiments of the present application provide a method and device for intelligently classifying web pages, a method and system for verifying the validity of a website's security device, and an electronic device. The method for intelligently classifying web pages can obtain test content for testing a website, and according to the test content, send one or more web page requests for testing to the website's server, and receive response content corresponding to each web page request for testing returned by the server. For the response content corresponding to each web page request for testing, multi-dimensional feature extraction and multi-level intelligent classification of static feature classification process, preliminary classification process, and deep classification process are performed, which can provide more accurate and efficient response analysis and web page classification to meet the high standards of modern website testing and verification.

[0053] In addition, the present embodiment can adopt an intelligent classification method of web pages to obtain the actual web page type of the website web page. If the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page, then it can be determined that the security device has passed the validity verification. In this way, through continuous monitoring and testing, the deficiencies of the security device in actual operation can be discovered in time, and the security policies and equipment can be adjusted and optimized accordingly, thereby enhancing the defense capabilities against network security threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0055] Figure 1A flow chart of a method for intelligently classifying web pages provided in an embodiment of the present application is shown;

[0056] Figure 2 A structural diagram of an intelligent classification device for web pages provided in an embodiment of the present application is shown;

[0057] Figure 3 A structural diagram of an intelligent classification device for web pages provided in another embodiment of the present application is shown;

[0058] Figure 4 A structural diagram of a security device validity verification system for a website provided by an embodiment of the present application is shown;

[0059] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0060] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such use is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to".

[0062] In order to solve the above technical problems, the present application embodiment provides an intelligent classification method for web pages, such as Figure 1 As shown, the intelligent classification method of web pages may include the following steps S101 to S106:

[0063] Step S101, obtaining test content for testing a website, wherein the test content includes one or more web page requests for testing.

[0064] In this step, a web page request for testing may include one of multiple types of requests, and multiple web page requests for testing may include one or more of multiple types of requests. The multiple types of requests here include, for example, requests without a path, deformed requests, normal requests, malicious requests, etc. This embodiment does not limit this.

[0065] Step S102, according to the test content, one or more web page requests for testing are sent to the server of the website, and response content corresponding to each web page request for testing returned by the server is received.

[0066] In this step, any web page request for testing may be sent to the server of the website, and response content corresponding to the any web page request for testing returned by the server may be received.

[0067] Step S103, for the response content corresponding to each web page request for testing, a static feature classification process is executed, and the content representing the request response status in the response content is used as a fingerprint to be matched with a preset fingerprint library. If a fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and the web page type corresponding to each fingerprint.

[0068] For example, "404 Not Found" is a fingerprint in the preset fingerprint library, and its corresponding web page type is a not found page; "502 Bad Gateway" is a fingerprint in the preset fingerprint library, and its corresponding web page type is a server error page; "503 Service Unavailable" is a fingerprint in the preset fingerprint library, and its corresponding web page type is a server error page, and so on. This implementation does not impose any restrictions on this.

[0069] Step S104: if no fingerprint in the preset fingerprint library is matched, the current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and the initial fuzzy hash value calculated and stored by the preset fuzzy hash algorithm after the first visit to the web page is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, the actual web page type of the web page data in the response content is determined to be a first category page; if the similarity is less than the first preset threshold, the preliminary web page type of the web page data in the response content is determined to be a second category page.

[0070] In this step, the fuzzy hash algorithm is a technology used for file similarity comparison, which can still identify the similarity between files when the file content changes. This algorithm determines whether the files are similar by dividing the file into small blocks, generating hash values ​​for these blocks, and then comparing these hash values. The preset fuzzy hash algorithm can be ssdeep, which can hash the input file and generate a hash value that can match the input with homology. Even if the file content and length are different between these sequences, ssdeep can match the files by identifying byte sequences of the same order. It should be noted that the ssdeep example used here is only illustrative and does not limit this embodiment.

[0071] In addition, the first visit to a web page means a normal visit to the site without any violation, and the web page data in the returned response content is calculated through a preset fuzzy hash algorithm to obtain an initial fuzzy hash value, which is then stored.

[0072] In addition, the first preset threshold can be set according to actual needs. For example, the first preset threshold is 0.8 or 0.85, etc. This embodiment does not limit this.

[0073] Step S105, execute the preliminary classification process, extract the response status code, response status code title, response header, response header length, response body, and response body length from the response content, input the extracted response status code, response status code title, response header, response header length, response body, and response body length into the trained preliminary classification model, use the trained preliminary classification model to predict the web page type of the web page data in the response content, output probability values ​​of multiple predicted web page types, sort the probability values ​​of the multiple predicted web page types from large to small, select the top two predicted web page types, if the difference between the probability values ​​of the top two predicted web page types is greater than or equal to the second preset threshold, then use the predicted web page type with the largest probability value as the actual web page type of the web page data in the response content.

[0074] In this step, the second preset threshold can be set according to actual needs. For example, the second preset threshold is 0.3 or 0.4, etc. This embodiment does not limit this.

[0075] Step S106: If the difference between the probability values ​​of the top two predicted web page types is less than a second preset threshold, the deep classification process is further executed to extract the page loading order, loading time, loading content, and dynamic cookies from the response content, and the extracted page loading order, loading time, loading content, and dynamic cookies are input into the trained deep classification model. The trained deep classification model is used to predict the web page type of the web page data in the response content, and the predicted result of the web page type is output, and the web page type of the predicted result is used as the actual web page type of the web page data in the response content.

[0076] In this step, a cookie is a small data file created by a website and stored on the user's device to record the user's browsing habits, personal settings, session information, etc.

[0077] Dynamic cookies are cookies that are dynamically generated by the server and sent to the user when the user interacts with the server through the user's device. Compared with static cookies, the content of dynamic cookies is not pre-defined, but is generated in real time based on the user's specific session or behavior. The purpose of this is to improve security and personalized experience.

[0078] This embodiment obtains test content for testing a website, sends one or more web page requests for testing to the website's server according to the test content, and receives response content corresponding to each web page request for testing returned by the server. For the response content corresponding to each web page request for testing, a static feature classification process, a preliminary classification process, and a deep classification process are performed for multi-dimensional feature extraction and multi-level intelligent classification, which can provide more accurate and efficient response analysis and web page classification to meet the high standards of modern website testing and verification.

[0079] A possible implementation method is provided in an embodiment of the present application. The web page types corresponding to each fingerprint in the preset fingerprint library mentioned in step S103 above may include not found pages, server error pages, etc.; the first type of pages mentioned in step S104 are normal pages, and the second type of pages are undetermined pages.

[0080] That is to say, for the response content corresponding to a web page request for testing, the first level of classification is first executed, that is, the static feature classification process of step S103 and step S104, and the content representing the request response status in the response content is used as a fingerprint to be matched with a preset fingerprint library. If the fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content, such as a page not found or a server error page, and subsequent hierarchical classification is not continued. If no fingerprint in the preset fingerprint library is matched, the current fuzzy hash value of the web page data in the response content is calculated by the preset fuzzy hash algorithm, and the initial fuzzy hash value calculated and stored by the preset fuzzy hash algorithm after the first visit to the web page is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to the first preset threshold, the actual web page type of the web page data in the response content is determined to be a first-category page, that is, a normal page, and subsequent hierarchical classification is not continued; if the similarity is less than the first preset threshold, the preliminary web page type of the web page data in the response content is determined to be a second-category page, that is, an undetermined page. Since only the preliminary web page type of the web page data in the response content is determined here, and the actual web page type of the web page data in the response content is not determined, it is necessary to continue the subsequent hierarchical classification. Here, the second-level classification is continued, that is, the preliminary classification process of step S105.

[0081] From the above embodiments, it can be seen that for the response content corresponding to each web page request for testing, the actual web page type of the web page data in a part of the response content can be obtained through the first level classification, that is, the static feature classification process of step S103 and step S104; for the remaining uncertain web page types, the second level classification, that is, the preliminary classification process of step S105, can be continued to be judged. If the preliminary classification process of step S105 still cannot determine the web page type, it can continue to be judged through the third level classification, that is, the deep classification process of step S106, and finally the actual web page type is obtained, thereby providing more accurate and efficient response analysis and web page classification, meeting the high standard requirements of modern website testing and verification.

[0082] A possible implementation method is provided in the embodiment of the present application. The preliminary classification model mentioned in step S105 above can be trained by following the steps A1 to A3:

[0083] Step A1, constructing a preliminary classification model through a machine learning model random forest algorithm;

[0084] Step A2, obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a response status code of a response content corresponding to the web page request for the sample, a response status code title, a response header, a response header length, a response body, and a response body length;

[0085] Step A3: Use sample data to train the preliminary classification model to obtain a trained preliminary classification model.

[0086] This embodiment uses a machine learning model random forest algorithm to perform primary classification on the static features of the web page (such as response status code, response status code title, response header, response header length, response body, response body length, etc.). Random forest integrates multiple decision trees and uses the static features of the web page for fast and effective classification. Here, the implementation of the preliminary classification model relies on the random forest classifier in the scikit-learn library. The scikit-learn library is an open source machine learning library that contains many algorithms for machine learning tasks such as classification, regression, clustering and dimensionality reduction, such as support vector machines, decision trees, random forests, gradient boosting, principal component analysis, etc., which can be used for data mining and data analysis.

[0087] A possible implementation method is provided in the embodiment of the present application. The deep classification model mentioned in step S106 above can be trained by following steps B1 to B3:

[0088] Step B1, building a deep classification model through LSTM;

[0089] Step B2, obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a page loading order, a loading time, a loading content, and a dynamic cookie of a response content corresponding to the web page request for the sample;

[0090] Step B3: Use sample data to train the deep classification model to obtain a trained deep classification model.

[0091] In this embodiment, LSTM (Long Short Term Memory) can capture the time series dependencies between features, thereby achieving more refined classification. Here, the deep classification model is implemented and trained using the PyTorch framework, which is an open source machine learning library widely used in applications such as computer vision and natural language processing.

[0092] The present application provides a possible implementation method. The test content in the above step S101 may also include theoretical web page types corresponding to each web page request for testing. Therefore, after determining the actual web page type of the web page data in the response content, the following steps C1 and C2 may be further included:

[0093] Step C1, searching the test content for the theoretical web page type of the web page data in the response content according to the theoretical web page type corresponding to each web page request for the test;

[0094] Step C2, comparing the actual web page type of the web page data in the response content with the theoretical web page type, and generating a test report for testing the website.

[0095] In this embodiment, the theoretical web page types corresponding to each web page request for testing may include one or more of a normal page, a security device intercepted page, a server error page, a not found page, and an undetermined page.

[0096] This embodiment compares the actual web page type of the web page data in the response content with the theoretical web page type, and automatically generates a test report for testing the website, thereby providing strong technical support for website operation and maintenance and security detection.

[0097] The above introduces Figure 1 There are multiple implementation methods for each link of the illustrated embodiment. The intelligent classification method for web pages of the embodiment of the present application will be further explained below through specific embodiments.

[0098] In a specific embodiment, the intelligent classification method of web pages may include data collection, feature extraction and selection, classification model construction and training, multi-level classification, real-time updating and adaptation, which will be described in detail below.

[0099] (1) Data Collection

[0100] Collect and store various types of data of web pages to be classified, and conduct comprehensive data collection on web pages from multiple dimensions, including but not limited to the following:

[0101] a) Web page data: By using crawler tools to capture the text, HTML (HyperText Markup Language) structure, CSS (Cascading Style Sheets) style, JavaScript code, etc. of the web page, the core content features of the page are extracted. The specific implementation includes parsing the HTML DOM (Document Object Model) structure, extracting the content of each tag, analyzing the application of CSS rules, and capturing JavaScript execution results to generate the overall feature vector of the page.

[0102] b) HTTP Headers: Use crawler tools to capture and analyze header information in HTTP (HyperText TransferProtocol) requests and responses, including set-cookie (the server sends one or more cookies to the client browser through this header field), cache-control (this header field is used to specify the cache instructions of the response, telling the browser or intermediate proxy server how to control the cache of the response. It can control the validity period of the cache, whether caching is allowed, whether the cache must be verified, etc.), content-type (this header field is used to specify the media type of the returned resource, such as text, HTML, image, video, etc.), etc., and further parse the header information to extract the server configuration, caching policy and security settings, and store them as structured data.

[0103] c) Page hash: The fuzzy hash value of each web page is calculated through a fuzzy hash algorithm (such as ssdeep) to quickly detect the uniqueness and similarity of page content. Fuzzy hash technology can identify pages with similar content and mark potential duplicate or variant pages.

[0104] d) Page loading order: When using crawler tools, the loading order of various resources (such as images, scripts, style sheets, etc.) when accessing the web page is recorded at the same time. By analyzing the loading order of resources, the loading mode and dependency of the page can be inferred.

[0105] e) Loading time: When using the crawler tool, the time from the page request to the completion of each resource loading is measured and recorded. The collected data includes the total page loading time, the loading time of each resource, etc. This information can be used to evaluate page performance and response time.

[0106] f) Cookies and Web Storage (a technology for storing data in the user's browser, which allows web pages to store large amounts of data, and the life cycle of the data can be persistent or temporary): When using crawler tools, analyze the cookie information generated after the page is loaded, and collect detailed information including the cookie name, value, validity period, path, domain name, etc. At the same time, collect and analyze data in Web Storage. This information helps understand the user status, personalized settings, and dynamic behavior of the page.

[0107] (2) Feature extraction and selection

[0108] Extract key features from data collection and select the most recognizable features through feature selection algorithm. The specific implementation is as follows:

[0109] a) Static feature extraction

[0110] i. Content: Extract static features by analyzing the fingerprint, page content, and fuzzy hash values ​​of web pages. Static features include the HTML structure, CSS style, and page hash values ​​of the page, which are used for static feature classification. Specific methods include extracting tag structures through regular expressions, calculating the TF-IDF (Term Frequency-Inverse Document Frequency) value of the text using word frequency analysis, and extracting the visual features of the page through image processing technology.

[0111] ii. Implementation: Generate feature vectors describing the static properties of the web page based on the structural features, content word frequency, and tag usage of the web page. These feature vectors are used for static feature classification to help distinguish different types of web pages.

[0112] b) Dynamic feature extraction

[0113] i. Extract dynamic features from page loading order, loading time, cookie generation content, etc. Dynamic features are used to reflect the behavior pattern of the page under different requests. Use time series analysis and statistical modeling methods to record and analyze the loading order of the page and the time interval between resource requests.

[0114] ii. Implementation: Capture the time series data of page loading and use tools to record the order in which resources are loaded. Further analyze the dynamic changes of cookies and Web Storage obtained from accessing web pages, and generate feature vectors that describe page interaction patterns and performance characteristics to support more accurate classification.

[0115] c) Feature Selection

[0116] i. Content: Use LASSO (Least Absolute Shrinkage and Selection Operator) regression and PCA (Principal Component Analysis) algorithms to select the features that have the greatest impact on the classification results. LASSO regression is used for feature selection, and by introducing regularization terms, features with larger coefficients are selected. PCA is used for data dimensionality reduction, retaining the most informative features in the data and reducing the complexity of the data.

[0117] Implementation: LASSO regression is used to automatically select important features and exclude redundant features; PCA reduces high-dimensional feature data to a lower dimension while retaining key information in the data. The final feature selection results will be input into the subsequent classification model to optimize the classification performance of the model.

[0118] (3) Classification model construction and training

[0119] Classification model construction and training is the core part, which is responsible for building a multi-level classification model based on the extracted features, including:

[0120] a) Static feature matching model: Directly identify web pages based on fingerprints and page hash values. Static feature matching uses pre-calculated hash values ​​and fingerprint feature libraries for comparison.

[0121] b) Primary classification model: Use the machine learning model Random Forest algorithm to perform primary classification on the static features of web pages. Random Forest integrates multiple decision trees and uses the static features of web pages for fast and effective classification. Here, the implementation of the preliminary classification model relies on the random forest classifier in the scikit-learn library. The scikit-learn library is an open source machine learning library that contains many algorithms for machine learning tasks such as classification, regression, clustering, and dimensionality reduction, such as support vector machines, decision trees, random forests, gradient boosting, principal component analysis, etc., which can be used for data mining and data analysis.

[0122] c) Deep classification model: For dynamic features (page loading order, loading time, dynamic cookies, etc.), a deep learning model LSTM is used for further classification. The LSTM network can capture the time series dependencies between features, thereby achieving more refined classification. Here, the deep classification model is implemented and trained using the PyTorch framework, which is an open source machine learning library widely used in applications such as computer vision and natural language processing.

[0123] Model training: Use cross-validation to train and validate the classification model to ensure that the model can be generalized to different web page data. Use annotated datasets to train the model, evaluate the model performance through cross-validation, and finally adjust the model parameters to improve the classification accuracy.

[0124] (4) Multi-level classification

[0125] The final web page classification is achieved through multi-level classification, that is, the results of static feature matching, primary and deep classification models are integrated, combined with multi-level feature analysis, to make the final classification. Specifically, it can include:

[0126] a) Static feature matching: Based on the static feature matching results of fingerprints and page hash values, known normal pages and abnormal pages are quickly identified. This step processes web pages that can be accurately classified by static features and classifies them into normal pages, security device intercepted pages, server error pages, not found pages, and undetermined pages.

[0127] b) Primary classification: For web pages that cannot be clearly classified through static feature matching (i.e., the first-level classification is undetermined), the primary classification model is used for further classification. This layer uses the random forest model to make preliminary judgments on web pages that cannot be processed by the initial classification, and uses ensemble learning of multiple decision trees to further classify the categories of undetermined pages into: normal pages, security device intercepted pages, server error pages, not found pages, and pages with small differences (i.e., pages that are very close to pages of other categories in the feature space, making it difficult for the model to determine their exact categories).

[0128] c) Deep classification: For pages with small differences that cannot be clearly distinguished by primary classification, a deep classification model is used for detailed analysis. This step is based on dynamic features and further classifies the categories of pages with small differences into: normal pages, and pages blocked by security devices.

[0129] (5) Real-time update and adaptation

[0130] In order to adapt to the ever-changing web content and network environment, the model parameters are dynamically adjusted and the classification rules are updated through real-time updating and adaptation. The specific implementation is as follows:

[0131] a) Online learning: During system operation, new data is collected in real time and incremental training is performed to continuously optimize and adjust the classification model. The online learning algorithm enables the model to continuously improve classification accuracy over time. For example, in the machine learning library scikit-learn, the partial_fit method is used in incremental learning or online learning scenarios, allowing the model to gradually update model parameters when new data arrives, rather than training from scratch each time.

[0132] b) Adaptive adjustment: By continuously monitoring the performance indicators of the classification model (such as accuracy, recall, etc.), the model parameters are automatically adjusted to ensure the accuracy and stability of the classification. This includes adjusting the learning rate, updating the algorithm parameters, etc., to cope with the performance degradation or overfitting problems that may occur in the model in actual applications, and to ensure the efficiency and reliability of the model in a dynamic environment.

[0133] Through the above multi-level implementation, this embodiment can effectively and accurately classify web pages of a single site or multiple sites, thereby providing strong technical support for website operation and security detection. This multi-dimensional feature extraction and multi-level classification method ensures that the system can still maintain efficient and accurate classification capabilities when processing complex web page structures and dynamic content.

[0134] Based on the same inventive concept, an embodiment of the present application also provides a method for verifying the validity of a website's security device. The method for verifying the validity of a website's security device first uses the above-mentioned intelligent classification method to obtain the actual web page type of the website's web page; then, if the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website's web page is a security device intercepted page, it is determined that the security device has passed the validity verification; if the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website's web page is not a security device intercepted page, but a normal page or a page not found, etc., it is determined that the security device cannot pass the validity verification.

[0135] The security device of this embodiment may be a firewall, an intrusion detection system, an intrusion prevention system, a security operation center, a unified threat management system, etc., which is not limited in this embodiment.

[0136] Through continuous monitoring and testing, this embodiment can timely discover deficiencies in the actual operation of security devices, and then make corresponding adjustments and optimizations to security policies and devices, thereby enhancing the defense capability against network security threats.

[0137] It should be noted that the size of the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In practical applications, all the above possible implementation methods can be combined in any way to form possible embodiments of the present application, which will not be described one by one here.

[0138] Based on the intelligent classification methods for web pages provided in the above embodiments, and based on the same inventive concept, an embodiment of the present application further provides an intelligent classification device for web pages.

[0139] Figure 2is a structural diagram of the intelligent classification device for web pages provided in the embodiment of the present application. Figure 2 As shown, the intelligent classification device for web pages may specifically include an acquisition unit 210 , a simulation test unit 220 , a static feature classification unit 230 , a preliminary classification unit 240 and a deep classification unit 250 .

[0140] The acquisition unit 210 is used to acquire test content for testing the website, wherein the test content includes one or more web page requests for testing;

[0141] The simulation test unit 220 is used to send one or more web page requests for testing to the server of the website according to the test content, and receive response content corresponding to each web page request for testing returned by the server;

[0142] The static feature classification unit 230 is used to perform a static feature classification process for the response content corresponding to each web page request for testing, and use the content in the response content indicating the request response status as a fingerprint to match it with a preset fingerprint library. If the fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and the web page type corresponding to each fingerprint; and

[0143] If no fingerprint in the preset fingerprint library is matched, the current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and the initial fuzzy hash value calculated and stored by the preset fuzzy hash algorithm after the first visit to the web page is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, the actual web page type of the web page data in the response content is determined to be a first-category page; if the similarity is less than the first preset threshold, the preliminary web page type of the web page data in the response content is determined to be a second-category page;

[0144] The preliminary classification unit 240 is used to further perform the preliminary classification process when the static feature classification unit 230 determines that the preliminary web page type of the web page data in the response content is a second-category page, extract the response status code, response status code title, response header, response header length, response body, and response body length from the response content, input the extracted response status code, response status code title, response header, response header length, response body, and response body length into the trained preliminary classification model, predict the web page type of the web page data in the response content using the trained preliminary classification model, output probability values ​​of multiple predicted web page types, sort the probability values ​​of the multiple predicted web page types from large to small, select the top two predicted web page types, and if the difference between the probability values ​​of the top two predicted web page types is greater than or equal to the second preset threshold, use the predicted web page type with the largest probability value as the actual web page type of the web page data in the response content;

[0145] The deep classification unit 250 is used to further execute the deep classification process when the preliminary classification unit 240 determines that the difference between the probability values ​​of the two predicted web page types ranked first is less than a second preset threshold value, extract the page loading order, loading time, loading content, and dynamic cookies from the response content, input the extracted page loading order, loading time, loading content, and dynamic cookies into the trained deep classification model, use the trained deep classification model to predict the web page type of the web page data in the response content, output the predicted result of the web page type, and use the web page type of the predicted result as the actual web page type of the web page data in the response content.

[0146] A possible implementation method is provided in an embodiment of the present application. The web page types corresponding to each fingerprint in the preset fingerprint library include not found pages and server error pages; the first type of page is a normal page, and the second type of page is an undetermined page.

[0147] A possible implementation method is provided in the embodiment of the present application, such as Figure 3 As shown above Figure 2 The device shown may also include a first training unit 310, configured to:

[0148] Train a preliminary classification model by following these steps:

[0149] Build a preliminary classification model using the machine learning model random forest algorithm;

[0150] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a response status code of a response content corresponding to the web page request for the sample, a response status code title, a response header, a response header length, a response body, and a response body length;

[0151] The sample data is used to train the preliminary classification model to obtain a trained preliminary classification model.

[0152] A possible implementation method is provided in the embodiment of the present application, such as Figure 3 As shown above Figure 2 The device shown may also include a second training unit 320, which is used to:

[0153] Build a deep classification model through long short-term memory network LSTM;

[0154] Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a page loading order, a loading time, a loading content, and a dynamic cookie of a response content corresponding to the web page request for the sample;

[0155] The deep classification model is trained using the sample data to obtain a trained deep classification model.

[0156] In an embodiment of the present application, a possible implementation is provided, wherein the test content further includes theoretical web page types corresponding to each web page request for testing; and the simulation test unit 220 is further used to:

[0157] After the static feature classification unit, the preliminary classification unit or the deep classification unit determines the actual web page type of the web page data in the response content, searching the theoretical web page type of the web page data in the response content from the test content according to the theoretical web page type corresponding to each web page request for testing;

[0158] The actual web page type of the web page data in the response content is compared with the theoretical web page type, and a test report for testing the website is generated.

[0159] A possible implementation method is provided in an embodiment of the present application, and the theoretical web page types corresponding to each web page request for testing include one or more of a normal page, a security device interception page, a server error page, a not found page, and an undetermined page.

[0160] Based on the validity verification methods of the security devices of a website provided in the above embodiments, and based on the same inventive concept, an embodiment of the present application also provides a validity verification system for the security devices of a website.

[0161] Figure 4 1 is a structural diagram of a validity verification system for a website security device provided by an embodiment of the present application. Figure 4 As shown, the validity verification system of the website's security device may specifically include the above-mentioned intelligent classification device and verification unit for web pages;

[0162] An intelligent classification device for web pages, used for intelligent classification to obtain the actual web page type of the website web pages;

[0163] The verification unit is used to determine that the security device passes the validity verification if the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page.

[0164] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned embodiments of the intelligent classification method of web pages or the validity verification method of the security device of a website.

[0165] In an exemplary embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The electronic device 500 shown includes: a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, such as through a bus 502. Optionally, the electronic device 500 may also include a transceiver 504. It should be noted that in actual applications, the transceiver 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0166] Processor 501 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0167] The bus 502 may include a path to transmit information between the above components. The bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0168] The memory 503 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0169] The memory 503 is used to store the computer program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the computer program code stored in the memory 503 to implement the contents shown in the above method embodiment.

[0170] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0171] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.

[0172] Those skilled in the art can understand that the technical solution of the present application can be essentially or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, which includes a number of program instructions, so that an electronic device (such as a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application when running the program instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0173] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as electronic devices such as personal computers, servers, or network devices), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of an electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of the present application.

[0174] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present application, they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate from the protection scope of the present application.

Claims

1. A method for intelligent classification of web pages, characterized in that: include: Acquire test content for testing the website, wherein the test content includes one or more web page requests for testing; According to the test content, one or more web page requests for testing are sent to the server of the website, and response content corresponding to each web page request for testing is returned by the server; For the response content corresponding to each web page request for testing, a static feature classification process is executed, and the content in the response content indicating the request response status is used as a fingerprint to be matched with a preset fingerprint library. If a fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and web page types corresponding to each fingerprint; If no fingerprint in the preset fingerprint library is matched, a current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and an initial fuzzy hash value calculated and stored by a preset fuzzy hash algorithm after the web page is accessed for the first time is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, it is determined that the actual web page type of the web page data in the response content is a first category page; If the similarity is less than a first preset threshold, it is determined that the preliminary web page type of the web page data in the response content is a second type of page, and a preliminary classification process is further performed to extract a response status code, a response status code title, a response header, a response header length, a response body, and a response body length from the response content, and the extracted response status code, response status code title, response header, response header length, response body, and response body length are input into a trained preliminary classification model, and the trained preliminary classification model is used to predict the web page type of the web page data in the response content, and probability values ​​of multiple predicted web page types are output, and the probability values ​​of the multiple predicted web page types are sorted from large to small, and the two predicted web page types with the highest sorting are selected. If the difference between the probability values ​​of the two predicted web page types with the highest sorting is greater than or equal to a second preset threshold, the predicted web page type with the largest probability value is used as the actual web page type of the web page data in the response content; If the difference between the probability values ​​of the top two predicted web page types is less than a second preset threshold, the deep classification process is further executed to extract the page loading order, loading time, loading content, and dynamic cookies from the response content, and the extracted page loading order, loading time, loading content, and dynamic cookies are input into the trained deep classification model. The trained deep classification model is used to predict the web page type of the web page data in the response content, and the predicted result of the web page type is output, and the web page type of the predicted result is used as the actual web page type of the web page data in the response content.

2. The method according to claim 1, characterized in that The web page types corresponding to each fingerprint in the preset fingerprint library include not found pages and server error pages; the first type of pages are normal pages, and the second type of pages are undetermined pages.

3. The method according to claim 1 or 2, characterized in that: Train a preliminary classification model by following these steps: Build a preliminary classification model using the machine learning model random forest algorithm; Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a response status code of a response content corresponding to the web page request for the sample, a response status code title, a response header, a response header length, a response body, and a response body length; The sample data is used to train the preliminary classification model to obtain a trained preliminary classification model.

4. The method according to claim 1 or 2, characterized in that: Train the deep classification model by following these steps: Build a deep classification model through long short-term memory network LSTM; Obtaining sample data, wherein the sample data includes a web page request for the sample, a sample web page type corresponding to the web page request for the sample, a page loading order, a loading time, a loading content, and a dynamic cookie of a response content corresponding to the web page request for the sample; The deep classification model is trained using the sample data to obtain a trained deep classification model.

5. The method according to claim 1 or 2, characterized in that: The test content also includes theoretical web page types corresponding to each web page request for testing; After determining the actual web page type of the web page data in the response content, the method further includes: From the test content, according to the theoretical web page types corresponding to the web page requests for each test, searching for the theoretical web page types of the web page data in the response content; The actual web page type of the web page data in the response content is compared with the theoretical web page type, and a test report for testing the website is generated.

6. The method according to claim 5, characterized in that The theoretical web page types corresponding to each web page request for testing include one or more of a normal page, a security device intercepted page, a server error page, a not found page, and an undetermined page.

7. A method for verifying the validity of a website's security device, characterized in that: include: Using the intelligent classification method of web pages according to any one of claims 1 to 6 to obtain the actual web page type of the website web page; If the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page, it is determined that the security device passes the validity verification.

8. An intelligent classification device for web pages, characterized in that: include: An acquisition unit, used to acquire test content for testing a website, wherein the test content includes one or more web page requests for testing; A simulation test unit, used to send one or more web page requests for testing to a server of a website according to the test content, and receive response content corresponding to each web page request for testing returned by the server; A static feature classification unit is used to perform a static feature classification process for the response content corresponding to each web page request for testing, and use the content in the response content indicating the request response status as a fingerprint to match it with a preset fingerprint library. If a fingerprint in the preset fingerprint library is matched, the web page type corresponding to the matched fingerprint in the preset fingerprint library is used as the actual web page type of the web page data in the response content; wherein the preset fingerprint library includes multiple fingerprints and the web page type corresponding to each fingerprint; and If no fingerprint in the preset fingerprint library is matched, the current fuzzy hash value of the web page data in the response content is calculated by a preset fuzzy hash algorithm, and the initial fuzzy hash value calculated and stored by the preset fuzzy hash algorithm after the first visit to the web page is obtained, and the similarity between the current fuzzy hash value and the initial fuzzy hash value is calculated. If the similarity is greater than or equal to a first preset threshold, the actual web page type of the web page data in the response content is determined to be a first-category page; if the similarity is less than the first preset threshold, the preliminary web page type of the web page data in the response content is determined to be a second-category page; a preliminary classification unit, for further executing a preliminary classification process when the static feature classification unit determines that the preliminary web page type of the web page data in the response content is a second-category page, extracting a response status code, a response status code title, a response header, a response header length, a response body, and a response body length from the response content, inputting the extracted response status code, response status code title, response header, response header length, response body, and response body length into a trained preliminary classification model, predicting the web page type of the web page data in the response content using the trained preliminary classification model, outputting probability values ​​of a plurality of predicted web page types, sorting the probability values ​​of the plurality of predicted web page types from large to small, selecting the top two predicted web page types, and if the difference between the probability values ​​of the top two predicted web page types is greater than or equal to a second preset threshold, taking the predicted web page type with the largest probability value as the actual web page type of the web page data in the response content; The deep classification unit is used to further execute the deep classification process when the difference between the probability values ​​of the two predicted web page types ranked first by the preliminary classification unit is determined to be less than a second preset threshold value, extract the page loading sequence, loading time, loading content, and dynamic cookies from the response content, input the extracted page loading sequence, loading time, loading content, and dynamic cookies into the trained deep classification model, use the trained deep classification model to predict the web page type of the web page data in the response content, output the predicted result of the web page type, and use the web page type of the predicted result as the actual web page type of the web page data in the response content.

9. A validity verification system for a website's security device, characterized in that: include: The intelligent classification device for web pages as claimed in claim 8 is used for intelligent classification to obtain the actual web page types of website web pages; The verification unit is used to determine that the security device passes the validity verification if the web page request used for testing is a malicious request that needs to be intercepted by the security device, and the actual web page type of the website web page is a security device interception page.

10. An electronic device, characterized in that: including a processor and a memory; Wherein, the memory stores a computer program; The processor is configured to run the computer program to execute the method for intelligently classifying web pages according to any one of claims 1 to 6 or the method for validating the security device of a website according to claim 7.

Images