A method and apparatus for classifying similar websites

Abstract

This invention provides a method and apparatus for classifying similar websites. It involves clustering all target websites based on their corresponding directory hierarchy data to obtain a first group of data; then clustering all target websites based on their corresponding user feature data and website fingerprint data to obtain a second group of data; and finally, using the first and second group data, obtaining a third group of data. The method and apparatus for classifying similar websites provided by this invention achieve high accuracy in determining website similarity, and grouping highly similar websites into a single category facilitates similar-category management.

Description

Technical Field

[0001] This application relates to the field of network security technology, specifically to a method and apparatus for classifying similar websites. Background Technology

[0002] With the continuous development of internet technology, the number of internet platforms and websites has surged in recent years, leading to a proliferation of attack methods. Consequently, more and more websites are using Web Application Firewalls (WAFs) for security protection. Cloud WAFs offer a cloud-based model of Web Application Firewalls, allowing website operators to implement security protection without installing software or deploying hardware on their own networks. Its primary implementation method utilizes DNS technology, transferring domain name resolution rights to achieve security protection.

[0003] For cloud-based web protection servers, a large number of websites typically use the protection services they provide. Since each website uses a specific development framework, specific external resources, and a specific operating environment, existing cloud-based web protection servers do not classify and manage multiple websites, which to some extent affects the protection efficiency and effectiveness. Summary of the Invention

[0004] In view of this, one of the technical problems solved by the embodiments of the present invention is to provide a method and apparatus for classifying similar websites, so as to solve the problem that there is no classification and management of similar websites in the prior art.

[0005] The first aspect of this application discloses a method for classifying similar websites. The method includes: clustering all target websites based on directory hierarchy data corresponding to multiple target websites to obtain first grouping data; wherein the directory hierarchy data is used to characterize the directory hierarchy structure of the websites; and the first grouping data is used to characterize a first category label corresponding to each target website.

[0006] Furthermore, based on the user feature data and website fingerprint data corresponding to all the target websites, clustering is performed on all the target websites to obtain second grouping data; wherein, the user feature data is used to characterize the characteristics of the users visiting the website; the website fingerprint data is used to characterize the response content characteristics of the website; and the second grouping data is used to characterize the second category label corresponding to each target website;

[0007] Based on the first group data and the second group data, a third group data is obtained; wherein, the third group data is used to characterize the third category tag corresponding to each target website.

[0008] A second aspect of this application discloses an apparatus for classifying similar websites. The apparatus includes: a first classification module, configured to cluster all target websites based on directory hierarchy data corresponding to multiple target websites to obtain first grouping data; wherein the directory hierarchy data is used to characterize the directory hierarchy structure of the websites; and the first grouping data is used to characterize a first category label corresponding to each target website.

[0009] The second classification module is used to cluster all the target websites based on the user feature data and website fingerprint data corresponding to all the target websites to obtain second group data; wherein, the user feature data is used to characterize the characteristics of the users visiting the website; the website fingerprint data is used to characterize the response content characteristics of the website; and the second group data is used to characterize the second category label corresponding to each target website.

[0010] The third classification module is used to obtain third group data based on the first group data and the second group data; wherein the third group data is used to characterize the third category tag corresponding to each target website.

[0011] This invention clusters all target websites based on their directory hierarchy data to obtain a first group of data; then clusters them again based on user characteristic data and website fingerprint data to obtain a second group of data; and finally, a third group of data is obtained based on the first and second group data. As can be seen, this embodiment of the invention utilizes both directory hierarchy data and user characteristic data and website fingerprint data corresponding to multiple target websites for two clustering groupings, and combines the results of these two clustering groupings to classify all target websites. This results in high accuracy in judging the similarity of multiple websites, and grouping highly similar websites into one category facilitates similar-category management operations. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a method for classifying similar websites disclosed in Embodiment 1 of this application;

[0014] Figure 2 This is a flowchart illustrating a method for classifying similar websites as disclosed in Embodiment 2 of this application;

[0015] Figure 3 This is a schematic block diagram of the structure of a similar website classification device disclosed in Embodiment 3 of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] It should be noted that the terms "first," "second," "third," and "fourth," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0018] Example 1

[0019] like Figure 1 As shown, Figure 1 This is a schematic flowchart illustrating a method for classifying similar websites disclosed in Embodiment 1 of this application. The method for classifying similar websites includes:

[0020] Step S101: Cluster all target websites based on the directory hierarchy data corresponding to multiple target websites to obtain the first group data.

[0021] In this embodiment, the directory hierarchy data is used to represent the directory hierarchy of the website. The specific representation method of the website's directory hierarchy and the number of directory levels are not limited and can be reasonably selected according to actual application needs.

[0022] In this embodiment, the method of clustering all target websites using the directory hierarchy data corresponding to all target websites is not limited and can be reasonably selected according to actual application needs. For example, any clustering algorithm such as K-Means clustering, hierarchical clustering, t-SNE clustering, or DBSCAN clustering can be used.

[0023] Optionally, the clustering method used to cluster all target websites is unsupervised clustering, which allows all target websites to be clustered into a specified number of categories.

[0024] In this embodiment, the first group of data is used to represent the first category tag corresponding to each target website. The specific number and identification method of the first category tags are not limited, but each target website corresponds to only one first category tag.

[0025] For example, after clustering all target websites based on the directory hierarchy data corresponding to all target websites, all target websites can be divided into five groups, and the first category labels corresponding to the target websites in the five groups can be identified by A, B, C, D, and E respectively.

[0026] Step S102: Cluster all target websites based on user feature data and website fingerprint data corresponding to all target websites to obtain the second group data.

[0027] In this embodiment, user feature data is used to characterize the features of website visitors. The specific method of characterizing the website visitor's features and the number of features are not limited and can be reasonably selected according to actual application needs. For example, the features of website visitors can be all or part of the user's access behavior parameters.

[0028] In this embodiment, website fingerprint data is used to characterize the response content features of the website. The specific characterization method and the number of response content features are not limited and can be reasonably selected according to actual application needs.

[0029] In this embodiment, the second group data is used to characterize the second category tag corresponding to each target website. The specific number and identification method of the second category tags are not limited, but each target website corresponds to only one second category tag.

[0030] In this embodiment, the order of implementation of steps S101 and S102 is not limited and can be reasonably selected according to actual application needs.

[0031] Step S103: Obtain the third group data based on the first group data and the second group data.

[0032] In this embodiment, the third group data is used to characterize the third category tag corresponding to each target website. The specific number and identification method of the third category tags are not limited, but each target website corresponds to only one third category tag.

[0033] In this embodiment, the method of obtaining the third group data is not limited and can be reasonably selected according to actual application needs. For example, the third category tags of target websites with the same first category tag and / or the same second category tag can be set to be the same, that is, they can be classified into one category.

[0034] Optionally, in order to classify patents with high similarity into the same category, it is preferable that multiple target websites with the same third category label also have the same first category label and second category label. That is, the target websites classified into the same category in step S103 are the target websites classified into the same category in both steps S101 and S102.

[0035] In this embodiment, after the cloud Waf protection server obtains the third group data, it can use the third group data to perform similar management operations on target websites with the same third category label. For example, if a target website is attacked, other target websites with the same third category label may also be at relatively high risk of being attacked. The cloud Waf protection server can preemptively perform targeted protection processing on these target websites.

[0036] As can be seen from the above embodiments of the present invention, the embodiments of the present invention cluster all target websites according to the directory hierarchy data corresponding to multiple target websites to obtain first group data; and cluster all target websites according to the user feature data and website fingerprint data corresponding to all target websites to obtain second group data; and then obtain third group data based on the first group data and the second group data. It is evident that the embodiments of the present invention utilize the directory hierarchy data corresponding to multiple target websites, as well as the user feature data and website fingerprint data corresponding to multiple target websites, to perform two clustering groupings, and combine the results of the two clustering groupings to classify all target websites. This results in high accuracy in judging the similarity of multiple websites, and classifying highly similar websites into one category facilitates similar management operations.

[0037] Example 2

[0038] like Figure 2 As shown, Figure 2 This is a schematic flowchart illustrating a method for classifying similar websites disclosed in Embodiment 2 of this application. The method for classifying similar websites includes:

[0039] Step S201: Based on the first request log data corresponding to each target website, obtain the directory hierarchy structure data and user feature data corresponding to each target website.

[0040] In this embodiment, the first request log data is used at least to characterize the website's URL and the access behavior parameters of the visiting user, and the specific method of obtaining it is not limited. In practical applications, the first request log data can be log data recorded and stored by the protection server or the target website, or it can be data obtained after processing log data.

[0041] In this embodiment, the directory hierarchy data corresponding to each target website can be obtained by analyzing the URL of each target website. Since the URL of a target website represents its directory hierarchy and corresponding attribute information, the directory information of the target website can be constructed by parsing the URL, and further processed by merging and resolving data at the same directory level to finally obtain the directory hierarchy data corresponding to the target website. The specific analysis method for analyzing the URL of each target website is not limited and can be reasonably selected according to actual application needs.

[0042] Optionally, to improve data processing efficiency and effectiveness, invalid request URLs may be filtered in step S201. Specifically, step S201 may include: preprocessing the first request log data corresponding to each target website according to the first data filtering rules to obtain the second request log data corresponding to each target website, and obtaining the directory hierarchy structure data corresponding to each target website based on the second request log data corresponding to each target website.

[0043] The first data filtering rule is used to filter the URLs corresponding to invalid requests represented in the first request log data, so that the second request log data is only used to represent the URLs corresponding to valid requests.

[0044] The method for determining the first data filtering rule is not limited and can be reasonably selected according to actual application needs. For example, if a request is blocked by the protection server, it can be determined that the request is invalid, and the URL corresponding to the request can be filtered using the first data filtering rule. Alternatively, the first data filtering rule can include a preset URL blacklist, so that after obtaining the first request log data, it can be processed using the first data filtering rule to obtain the second request log data.

[0045] In this embodiment, user characteristic data for each target website can be obtained by analyzing the access behavior parameters of users visiting each target website. The user characteristic data for each target website can be determined based on the access behavior parameters of some or all users visiting the target website; the specific determination method is not limited and can be reasonably selected according to actual application needs. For example, the user characteristic data for the target website can be determined based on the access behavior parameters of the user who sent the most requests; alternatively, it can be determined based on the statistical results of the access behavior parameters of all users visiting the target website.

[0046] Furthermore, there are no restrictions on the types of access behavior parameters for users, and they can be reasonably selected according to actual application needs. For example, access behavior parameters may include at least one of the following: the type of website resource accessed, the number of user agents, the source address of the requested resource, and the URL depth.

[0047] Optionally, in order to characterize the features of users visiting the target website more reasonably and objectively, step S201 may include the following sub-steps:

[0048] Sub-step S201a: Based on the first request log data corresponding to the target website, cluster all users accessing the target website to obtain at least one user category and access behavior parameters corresponding to each user category.

[0049] Sub-step S201b: Obtain user feature data corresponding to the target website based on the access behavior parameters corresponding to the user category with the largest number of users.

[0050] In sub-step S201a, the method for clustering all users accessing the target website is not limited and can be selected appropriately according to actual application needs. For example, any clustering algorithm such as K-Means clustering, hierarchical clustering, t-SNE clustering, or DBSCAN clustering can be used.

[0051] Furthermore, it is preferable to use unsupervised clustering in sub-step S201a, so that all users accessing the target website can be clustered into a specified number of user categories.

[0052] Furthermore, in order to more comprehensively characterize the features of users accessing the target website, the preferred access behavior parameters can include at least the type of website resource accessed, the number of user agents, the source address of the requested resource, and the URL depth.

[0053] The source address of the requested resource can be determined by the referer (also known as referrer). The referer is a request header in HTTP that specifies the source address of the requested resource. It is generally used in communication between a web browser and a server to inform the server of the source of the requested resource.

[0054] Optionally, to improve the efficiency and effectiveness of subsequent data processing, the access behavior parameters of users corresponding to invalid requests can be filtered in step S201. Specifically, step S201 may include: preprocessing the first request log data corresponding to each target website according to the second data filtering rules to obtain the third request log data corresponding to each target website, and obtaining the user feature data corresponding to each target website based on the third request log data corresponding to each target website.

[0055] The second data filtering rule is used to filter the access behavior parameters of users corresponding to invalid requests represented in the first request log data, so that the third request log data is only used to represent the access behavior parameters of users corresponding to valid requests.

[0056] Step S202: Obtain website fingerprint data for each target website based on the first access response data for each target website.

[0057] In this embodiment, the first access response data is used at least to characterize the website's response to the access request, and the specific method of obtaining it is not limited. In practical applications, unlike the first request log data, the first access response data usually needs to be obtained through an active request, that is, by sending a specific access request to the target website's page to obtain the data responded by the page.

[0058] Optionally, since target websites typically consist of multiple pages, the homepages of multiple highly similar target websites are likely to also be highly similar. Therefore, to more accurately classify highly similar websites subsequently, the first access data can be preferred to characterize the content of the website's homepage in response to the access request.

[0059] Optionally, to improve data processing efficiency and effectiveness, step S202 may filter the response content corresponding to failed access requests. Specifically, step S202 may include: preprocessing the first access response data corresponding to each target website according to a third data filtering rule to obtain the second access response data corresponding to each target website, and obtaining the website fingerprint data corresponding to each target website based on the second access response data corresponding to each target website.

[0060] The third data filtering rule is used to filter the response content corresponding to the failed access requests represented in the first access data, so that the second access data is only used to represent the response content corresponding to the successful access requests.

[0061] Optionally, in order to accurately characterize the response content features of the target website and quickly obtain the website fingerprint data corresponding to each target website, the website fingerprint data corresponding to each target website can be obtained only based on the response header sub-data and response page sub-data corresponding to the target website.

[0062] Specifically, the first access response data includes response header data and response page data. The response header data represents the website's response to the access request, and the response page data represents the content of the website's response page. Correspondingly, step S202 may include: using preset feature detection rules to detect and match the response header data and response page data corresponding to each target website to obtain website fingerprint data corresponding to each target website.

[0063] Some fields in the response header represent certain information about the website, such as the Server header. By using preset feature detection rules, the response headers of the target website can be detected and matched. If a match is successful, the corresponding website fingerprint data can be obtained based on the characteristics of the successfully matched response header.

[0064] The content of the response page may contain information such as third-party resources that the website depends on. By using preset feature detection rules, the content of the response page corresponding to the target website can be detected and matched. If the match is successful, the corresponding website fingerprint data can be obtained based on the content features of the successfully matched response page.

[0065] In this embodiment, the order of implementation of steps S201 and S202 is not limited and can be reasonably selected according to actual application needs.

[0066] Step S203: Cluster all target websites based on the directory hierarchy data corresponding to multiple target websites to obtain the first group data.

[0067] Optionally, since the website's directory structure resembles a tree structure, to more accurately cluster multiple similar target websites, the similarity of the tree structures corresponding to the multiple target websites can be calculated to obtain the first group data. Specifically, step S203 may include:

[0068] Sub-step S203a: Based on the directory hierarchy data corresponding to each target website, construct the tree structure data corresponding to each target website.

[0069] Sub-step S203b: According to the preset edit distance calculation rules, obtain the edit distance between the tree structure data corresponding to all target websites.

[0070] Sub-step S203c: Based on the edit distance between the tree structure data corresponding to all target websites, cluster all target websites to obtain the first group data.

[0071] Furthermore, to ensure the reliability of the grouping results and improve computational efficiency, the maximum depth of the tree structure corresponding to the target website can be preferably set to 4.

[0072] Step S204: Cluster all target websites based on user feature data and website fingerprint data corresponding to all target websites to obtain the second group data.

[0073] Optionally, considering that in practical applications, the user feature data and website fingerprint data corresponding to each target website include data from multiple dimensions, and the influence of different dimensions on similarity judgment will vary, step S204 may include the following to obtain more accurate and objective clustering results:

[0074] Sub-step S204a involves dimensionality reduction processing of the user feature data and website fingerprint data corresponding to each target website to obtain K-dimensional feature data corresponding to each target website.

[0075] Sub-step S204b calculates the similarity values ​​between the K-dimensional feature data corresponding to all target websites according to the preset similarity calculation rules.

[0076] Sub-step S204c involves clustering all target websites based on the similarity values ​​between the K-dimensional feature data corresponding to all target websites to obtain the second group data.

[0077] The dimensionality reduction method used in sub-step S204a is not limited, nor is the preset similarity calculation rule used in step 204b. They can be reasonably selected according to actual application needs.

[0078] Furthermore, to reduce data complexity and identify the most important features, PCA (Principal Component Analysis) is a preferred method for dimensionality reduction. In PCA, data is transformed from the original coordinate system to a new coordinate system, the choice of which is determined by the data itself.

[0079] Optionally, the pairwise similarity values ​​between all target websites can be calculated using the cosine similarity metric, and then unsupervised clustering can be used to obtain the second group data.

[0080] In this embodiment, the order of implementation of steps S203 and S204 is not limited and can be reasonably selected according to actual application needs.

[0081] Step S205: Obtain third group data based on the first group data and the second group data.

[0082] In this embodiment, step S205 is basically the same as or similar to step S103 in the aforementioned embodiment one, and will not be described again here.

[0083] As can be seen from the above embodiments of the present invention, by filtering the first request log data and the first access response data corresponding to each target website, the present invention can improve data processing efficiency and effectiveness; by clustering all users accessing the target website and obtaining user feature data corresponding to the target website based on the access behavior parameters corresponding to the user category with the largest number of users, the characteristics of users accessing the target website can be characterized in a more reasonable and objective way; by performing dimensionality reduction processing on the user feature data and website fingerprint data corresponding to each target website, a more accurate website classification result can be obtained.

[0084] Example 3

[0085] Embodiment 5 of this application provides a similar website classification device. Figure 3 This is a schematic diagram of a similar website classification device disclosed in Embodiment 3 of this application. The device includes:

[0086] The first classification module is used to cluster all target websites based on the directory hierarchy data corresponding to multiple target websites to obtain the first group data; wherein, the directory hierarchy data is used to represent the directory hierarchy structure of the website; the first group data is used to represent the first category label corresponding to each target website.

[0087] The second classification module is used to cluster all target websites based on user feature data and website fingerprint data corresponding to all target websites to obtain second group data; wherein, user feature data is used to characterize the characteristics of users visiting the website; website fingerprint data is used to characterize the response content characteristics of the website; and the second group data is used to characterize the second category label corresponding to each target website.

[0088] The third classification module is used to obtain third group data based on the first group data and the second group data; wherein, the third group data is used to characterize the third category tag corresponding to each target website.

[0089] Optionally, the device further includes:

[0090] The first data acquisition module is used to obtain the directory hierarchy structure data and user characteristic data corresponding to each target website based on the first request log data corresponding to each target website. The first request log data is used to characterize the URL of the website and the access behavior parameters of the accessing user.

[0091] The second data acquisition module is used to obtain website fingerprint data corresponding to each target website based on the first access response data corresponding to each target website, wherein the first access response data is used to characterize the website's response content to the access request.

[0092] Optionally, the first classification module is further configured to preprocess the first request log data corresponding to each target website according to the first data filtering rules to obtain the second request log data corresponding to each target website, and obtain the directory hierarchy structure data corresponding to each target website based on the second request log data corresponding to each target website.

[0093] Optionally, the second classification module is further configured to preprocess the first request log data corresponding to each target website according to the second data filtering rules to obtain the third request log data corresponding to each target website, and obtain the user feature data corresponding to each target website based on the third request log data corresponding to each target website.

[0094] Optionally, the second classification module is also used to preprocess the first access response data corresponding to each target website according to the third data filtering rules to obtain the second access response data corresponding to each target website, and to obtain the website fingerprint data corresponding to each target website based on the second access response data corresponding to each target website.

[0095] Optionally, the first data acquisition module is further configured to cluster all users accessing the target website based on the first request log data corresponding to the target website, to obtain at least one user category and access behavior parameters corresponding to each user category; and to obtain user feature data corresponding to the target website based on the access behavior parameters corresponding to the user category with the largest number of users.

[0096] Furthermore, the access behavior parameters include at least the type of website resource accessed, the number of user agents, the source address of the requested resource, and the URL depth.

[0097] Optionally, the first access response data includes response header data and response page data. The response header data represents the website's response headers to the access request, and the response page data represents the content of the website's response page to the access request. Correspondingly, the second data acquisition module is further used to detect and match the response header data and response page data corresponding to each target website using preset feature detection rules to obtain website fingerprint data corresponding to each target website.

[0098] Optionally, the first classification module is also used to construct tree structure data corresponding to each target website based on the directory hierarchy structure data corresponding to each target website;

[0099] According to the preset edit distance calculation rules, obtain the edit distance between the tree structure data corresponding to all target websites;

[0100] Based on the edit distance between the tree structure data corresponding to all target websites, cluster all target websites to obtain the first group of data.

[0101] Optionally, the second classification module is also used to perform dimensionality reduction processing on the user feature data and website fingerprint data corresponding to each target website to obtain K-dimensional feature data corresponding to each target website;

[0102] According to the preset similarity calculation rules, calculate the similarity values ​​between the K-dimensional feature data corresponding to all target websites;

[0103] Based on the similarity values ​​between the K-dimensional feature data corresponding to all target websites, cluster all target websites to obtain the second group data.

[0104] Optionally, multiple target websites with the same third category label may also have the same first category label and second category label.

[0105] The similar website classification device of this embodiment can realize the corresponding similar website classification methods in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0106] Specific embodiments of this application have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0107] This application is described with reference to flowchart illustrations and / or block diagrams of methods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0109] Those skilled in the art will understand that the embodiments of this application can be provided as methods or apparatus. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0111] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method of classifying similar websites, characterized by, The method comprises: According to the directory hierarchy data corresponding to a plurality of target websites, the target websites are clustered to obtain first grouping data; wherein the directory hierarchy data is used to represent the directory hierarchy of a website; the first grouping data is used to represent the first category label corresponding to each target website; further comprising: constructing the tree structure data corresponding to each target website according to the directory hierarchy data corresponding to each target website; obtaining the edit distance between the tree structure data corresponding to all target websites according to the preset edit distance calculation rule; clustering all target websites according to the edit distance between the tree structure data corresponding to all target websites to obtain the first grouping data; And, according to the user feature data and website fingerprint data corresponding to all target websites, all target websites are clustered to obtain second grouping data; wherein the user feature data is used to represent the characteristics of the access user of a website; the website fingerprint data is used to represent the response content characteristics of a website; the second grouping data is used to represent the second category label corresponding to each target website; further comprising: dimension reduction processing the user feature data and the website fingerprint data corresponding to each target website to obtain the K-dimensional feature data corresponding to each target website; calculating the similarity value between the K-dimensional feature data corresponding to all target websites according to the preset similarity calculation rule; clustering all target websites according to the similarity value between the K-dimensional feature data corresponding to all target websites to obtain the second grouping data; According to the first grouping data and the second grouping data, the third grouping data is obtained; wherein the third grouping data is used to represent the third category label corresponding to each target website.

2. The method of claim 1, wherein, The method further comprises: According to the first request log data corresponding to each target website, the directory hierarchy data and the user feature data corresponding to each target website are obtained, wherein the first request log data is used to represent at least the URL of a website and the access behavior parameter of the access user; And, according to the first access response data corresponding to each target website, the website fingerprint data corresponding to each target website is obtained, wherein the first access response data is used to represent at least the response content of a website to an access request.

3. The method of claim 2, wherein, According to the first request log data corresponding to each target website, the directory hierarchy data and the user feature data corresponding to each target website are obtained, and according to the first access response data corresponding to each target website, the website fingerprint data corresponding to each target website is obtained, comprising at least one of the following steps: According to a first data filtering rule, the first request log data corresponding to each target website is preprocessed to obtain second request log data corresponding to each target website, and the directory hierarchy data corresponding to each target website is obtained according to the second request log data corresponding to each target website; According to a second data filtering rule, the first request log data corresponding to each target website is preprocessed to obtain third request log data corresponding to each target website, and the user feature data corresponding to each target website is obtained according to the third request log data corresponding to each target website; According to a third data filtering rule, the first access response data corresponding to each target website is preprocessed to obtain second access response data corresponding to each target website, and the website fingerprint data corresponding to each target website is obtained according to the second access response data corresponding to each target website.

4. The method of claim 2, wherein, The directory hierarchy data and the user feature data corresponding to each target website are obtained according to the first request log data corresponding to each target website, including: According to the first request log data corresponding to the target website, all access users corresponding to the target website are clustered to obtain at least one user category and access behavior parameters corresponding to each user category; According to the access behavior parameters corresponding to the user category with the largest number of users, the user feature data corresponding to the target website is obtained.

5. The method of claim 4, wherein, The access behavior parameters at least include accessed website resource types, user agent numbers, request resource source addresses, and URL depths.

6. The method of claim 2, wherein, The first access response data includes response header sub-data and response page sub-data, the response header sub-data is used to represent the response header of the website to the access request, and the response page sub-data is used to represent the content of the response page of the website to the access request; Correspondingly, the website fingerprint data corresponding to each target website is obtained according to the first access response data corresponding to each target website, including: According to a third data filtering rule, the first access response data corresponding to each target website is preprocessed to obtain second access response data corresponding to each target website, and the website fingerprint data corresponding to each target website is obtained according to the second access response data corresponding to each target website.

7. The method of claim 1, wherein, The first category label and the second category label corresponding to the plurality of target websites of the same third category label are all the same.

8. An apparatus for classifying similar websites, characterized by The apparatus includes: The first classification module is configured to cluster all the target websites according to directory hierarchy data corresponding to the target websites, to obtain first grouping data; the directory hierarchy data is used to represent a directory hierarchy of a website; the first grouping data is used to represent a first category label corresponding to each target website; the first classification module is further configured to construct tree structure data corresponding to each target website according to the directory hierarchy data corresponding to each target website; to obtain an edit distance between the tree structure data corresponding to all the target websites according to a preset edit distance calculation rule; and to cluster all the target websites according to the edit distance between the tree structure data corresponding to all the target websites, to obtain the first grouping data; The second classification module is configured to cluster all the target websites according to user feature data and website fingerprint data corresponding to the target websites, to obtain second grouping data; the user feature data is used to represent a feature of a user accessing a website; the website fingerprint data is used to represent a response content feature of a website; the second grouping data is used to represent a second category label corresponding to each target website; the second classification module is further configured to perform dimension reduction processing on the user feature data and the website fingerprint data corresponding to each target website, to obtain K-dimensional feature data corresponding to each target website; to calculate a similarity value between the K-dimensional feature data corresponding to all the target websites according to a preset similarity calculation rule; and to cluster all the target websites according to the similarity value between the K-dimensional feature data corresponding to all the target websites, to obtain the second grouping data; The third classification module is configured to obtain third grouping data according to the first grouping data and the second grouping data; the third grouping data is used to represent a third category label corresponding to each target website.

Images