A website matching method, apparatus, device, and storage medium

Abstract

This application provides a website matching method, apparatus, device, and storage medium to address the problem of significant limitations in website matching. The method includes at least: extracting features from multiple sub-websites belonging to a main website to obtain a URL feature set corresponding to each of the sub-websites; fusing and encoding the URL feature sets to obtain a website fingerprint of a first website, where the first website includes the main website and the web pages corresponding to each of the multiple sub-websites; and generating a website matching result based on the fingerprint similarity between the website fingerprint of the first website and the reference website fingerprints of multiple pre-stored reference websites. The website fingerprint generation process does not involve the acquisition of web page content, thus avoiding the significant limitations in website matching caused by the acquisition of web page content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a website matching method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of technology, more and more client or server devices can generate website fingerprints. These fingerprints are used as unique identifiers to match websites, thereby enabling intelligent website analysis.

[0003] For example, in the case of abnormal website identification, after receiving an identification request for the website to be matched, a website fingerprint of the website to be matched can be generated based on the web page content involved in the website to be matched. The website fingerprint of each abnormal website can be obtained from the stored abnormal website database. Based on the obtained website fingerprints, the website to be matched and each abnormal website can be matched to obtain the identification result of whether the website to be identified is an abnormal website.

[0004] For example, in a website recommendation scenario, when accessing a website to be matched, a recommendation request can be generated for the website to be matched. Based on the web page content involved in the website to be matched, a website fingerprint of the website to be matched can be generated. The website fingerprints of each candidate website can be obtained from the stored candidate website library. Based on the obtained website fingerprints, the website to be matched can be matched with each candidate website to determine the recommended website as recommendation feedback, thereby achieving the purpose of website promotion.

[0005] In related technologies, website matching methods typically involve using a web crawler to traverse a webpage within a website and obtain its content; then, encoding the obtained webpage content to generate a website fingerprint, which can then be used to match websites.

[0006] However, using web scraping programs for website matching has significant limitations, for example:

[0007] First, it can easily lead to the leakage of some important information in the webpage content, which can bring some unnecessary security risks to the website matching process;

[0008] Second, for websites that have set up anti-scraping mechanisms, it will be impossible to obtain the corresponding web page content, and thus impossible to generate the website fingerprint. Therefore, it will be impossible to match websites based on the website fingerprint.

[0009] It is evident that the matching process of websites using the relevant technologies has significant limitations. Summary of the Invention

[0010] This application provides a website matching method, apparatus, device, and storage medium to address the problem of significant limitations in website matching.

[0011] Firstly, a website matching method is provided, including:

[0012] Feature extraction is performed on multiple sub-websites belonging to a main website to obtain the website feature set corresponding to each of the multiple sub-websites;

[0013] The feature sets of each URL are fused and encoded to obtain the website fingerprint of the first website, which includes the main URL and the web pages corresponding to each of the multiple sub-URLs;

[0014] Based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of the reference websites of multiple pre-stored reference websites, a website matching result is generated.

[0015] Secondly, a website matching device is provided, comprising:

[0016] Processing module: used to extract features from multiple sub-websites belonging to a main website, and obtain the website feature set corresponding to each of the multiple sub-websites;

[0017] The processing module is further configured to: perform fusion encoding on each URL feature set to obtain the website fingerprint of the first website, wherein the first website includes the main URL and the web pages corresponding to each of the multiple sub-URLs;

[0018] The processing module is further configured to: generate a website matching result based on the fingerprint similarity between the website fingerprint of the first website and the reference website fingerprints of each of the pre-stored multiple reference websites.

[0019] Optionally, the apparatus further includes an acquisition module, the acquisition module being used for:

[0020] Receive a website matching request for the first website;

[0021] From the access database, query the multiple sub-websites belonging to the main website, the access database being used to store each website accessed in historical time;

[0022] The processing module is specifically used for:

[0023] Feature extraction is performed on each of the multiple sub-URLs to obtain the URL feature set corresponding to each of the multiple sub-URLs.

[0024] Optionally, the processing module is specifically used for:

[0025] For each of the multiple sub-URLs, perform the following operations:

[0026] Extract the access path from a sub-URL;

[0027] When it is determined that the access path contains resource path information representing the address of a web page resource, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

[0028] Based on the at least one resource sub-information, establish a set of URL features corresponding to the sub-URL.

[0029] Optionally, the processing module is specifically used for:

[0030] Replace the path characters in the access path that match the character format of the preset character with the preset character to obtain the access cleaning path;

[0031] Based on a preset URL format, at least one path information is extracted from the access cleaning path, wherein the path information represents: the webpage resource address or resource query parameters;

[0032] When the at least one path information includes resource path information representing the webpage resource address, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

[0033] Optionally, the processing module is specifically used for:

[0034] When the access path includes query path information representing resource query parameters, the query path information and the at least one resource sub-information are each used as URL features to obtain the URL feature set corresponding to the sub-URL.

[0035] Optionally, the processing module is specifically used for:

[0036] For each set of URL features, perform the following operations:

[0037] Based on the corresponding position of each URL feature in the corresponding sub-URL in a set of URL features, the directory hierarchy of each URL feature is determined;

[0038] Based on each directory hierarchy, the URL features are concatenated to obtain the URL features of each hierarchy.

[0039] Each hierarchical URL feature is encoded separately to obtain each encoded feature, and the encoded features are then fused to obtain the website fingerprint of the first website.

[0040] Optionally, the processing module is specifically used for:

[0041] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features of each hierarchy.

[0042] Optionally, the processing module is specifically used for:

[0043] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features of each directory.

[0044] By concatenating every two adjacent directory URL features in each directory hierarchy, we obtain the URL features for each hierarchy.

[0045] Optionally, each of the multiple sub-URLs is associated with a number of visits over a historical period; then the processing module is specifically used for:

[0046] Based on the number of visits associated with each of the multiple sub-URLs, the frequency of each feature of each hierarchical URL is statistically analyzed;

[0047] The hash value of each hierarchical URL feature is determined to obtain each encoding feature;

[0048] Based on the frequency of each feature, the website fingerprint of the first website is obtained by fusing the encoded features.

[0049] Optionally, the processing module is specifically used for:

[0050] Based on the frequency of each feature, the weighted sum of each encoded feature is performed to obtain the comprehensive feature;

[0051] Based on a preset binary mapping strategy, the comprehensive features are mapped into a binary sequence to obtain the website fingerprint of the first website.

[0052] Optionally, the processing module is specifically used for:

[0053] Retrieve the reference website fingerprints and website anomaly types for each of the multiple reference websites from the pre-stored fingerprint blacklist;

[0054] The Hamming distance between the website fingerprint of the first website and the reference website fingerprint of each of the plurality of reference websites is determined, and the fingerprint similarity of each of the plurality of reference websites is obtained.

[0055] When at least one of the multiple reference websites has a fingerprint similarity greater than a preset similarity threshold, the website anomaly type of the first website is determined based on the website anomaly type of each of the at least one reference website, and a website matching result is obtained.

[0056] Optionally, the processing module is specifically used for:

[0057] The fingerprint of the first website is determined, as well as the fingerprint similarity between the first website and the reference website fingerprints of each of the multiple reference websites obtained from the pre-stored set of fingerprints to be matched, and the fingerprint similarity between any two of the reference website fingerprints is determined.

[0058] Based on the fingerprint similarity, the first website and the plurality of reference websites are clustered to obtain at least one website cluster. The website cluster contains at least one website to be classified, which is either the first website or the reference website. The fingerprint similarity of the websites to be classified in the same website cluster is greater than the fingerprint similarity of the websites to be classified in different website clusters.

[0059] Based on the at least one website cluster, the website categories of the first website and the plurality of reference websites are identified respectively to obtain website matching results.

[0060] Thirdly, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0061] Fourthly, a computer device is provided, comprising:

[0062] Memory, used to store program instructions;

[0063] A processor is configured to invoke program instructions stored in the memory and execute the method described in the first aspect according to the obtained program instructions.

[0064] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method as described in the first aspect.

[0065] In this embodiment, a website fingerprint of the first website is generated based on the URL feature sets extracted from multiple sub-URLs belonging to a main URL. This process only involves data processing for the sub-URLs; it does not require obtaining webpage content or processing related data within the webpage content. Since the sub-URLs are completely public, no additional program for crawling webpage content is needed. Similarly, since the URL feature sets can be extracted from the URLs of the sub-URLs themselves, without processing the webpage content linked to by the sub-URLs, no additional program for crawling webpage content is required.

[0066] This application's embodiments avoid the leakage of important information in web page content due to web page crawling programs, thus improving data security during the website matching process. Simultaneously, it prevents situations where websites with anti-crawling mechanisms cannot obtain the corresponding web page content, thereby preventing the generation of the first website's website fingerprint. This increases the success rate of generating website fingerprints during the website matching process, thereby solving the matching limitations in the website matching process.

[0067] Furthermore, by fusing and encoding the feature sets of each URL into a website fingerprint, which integrates the website characteristics reflected by multiple sub-URLs, the accuracy of the website fingerprint can be improved, thereby improving the accuracy of website matching. Attached Figure Description

[0068] Figure 1A This is a schematic diagram illustrating an application field of the website matching method provided in the embodiments of this application;

[0069] Figure 1B This is a schematic diagram illustrating the principle of a website matching method based on related technologies.

[0070] Figure 1C This is one application scenario of the website matching method provided in the embodiments of this application;

[0071] Figure 2 A schematic flowchart of a website matching method provided in this application embodiment;

[0072] Figure 3A A schematic diagram illustrating the principle of the website matching method provided in this application embodiment;

[0073] Figure 3B A schematic diagram of the principle of the website matching method provided in the embodiments of this application. Figure 2 ;

[0074] Figure 4A A schematic diagram three illustrating the principle of the website matching method provided in this application embodiment;

[0075] Figure 4B A schematic diagram four illustrating the principle of the website matching method provided in this application embodiment;

[0076] Figure 4C A schematic diagram five illustrating the principle of the website matching method provided in this application embodiment;

[0077] Figure 5A A schematic diagram six illustrating the principle of the website matching method provided in this application embodiment;

[0078] Figure 5B A schematic diagram seven illustrating the principle of the website matching method provided in this application embodiment;

[0079] Figure 5C A schematic diagram of the principle of the website matching method provided in the embodiments of this application. Figure 8 ;

[0080] Figure 6A A schematic diagram of the principle of the website matching method provided in the embodiments of this application. Figure 9 ;

[0081] Figure 6B A schematic diagram ten illustrating the principle of the website matching method provided in this application embodiment;

[0082] Figure 7A A flowchart illustrating a website matching method provided in this application embodiment. Figure 2 ;

[0083] Figure 7B A flowchart illustrating the website matching method provided in this application embodiment is shown in Figure 3.

[0084] Figure 8 A schematic diagram of the structure of a website matching device provided in an embodiment of this application;

[0085] Figure 9 A schematic diagram of the structure of the website matching device provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0087] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0088] (1) Website fingerprint:

[0089] Website fingerprints can be used to uniquely identify a website, to identify or detect a website, help determine the website's identity, and also to discover website security vulnerabilities.

[0090] (2) Hamming distance:

[0091] Hamming distance is a metric for measuring the difference between two strings of equal length. Specifically, it is defined as the minimum number of character substitutions required to transform one string into another. In other words, Hamming distance is the number of different characters at corresponding positions in two strings.

[0092] (3) Website:

[0093] A website is a collection of interconnected web pages accessed via the internet. Each web page has a unique address called a Uniform Resource Locator (URL), which is referred to as a website address in this embodiment. Website addresses typically begin with "http: / / " or "https: / / " followed by a domain name (e.g., www.example.com).

[0094] (4) Domain Name:

[0095] A domain name is a unique name used to identify and locate resources on the Internet. A domain name consists of multiple parts, typically including a top-level domain (TLD), second-level domains, etc. The format of a domain name is from right to left, with each part separated by a period (.).

[0096] Top-level domains include .com, .org, .net, .edu, etc., while second-level domains are the parts that immediately follow the top-level domain. For example, in example.com, the top-level domain is .com and the second-level domain is example.

[0097] A primary domain refers to the main identifier of an organization or individual on the internet. It is usually represented by a second-level domain followed by a top-level domain. Any of these levels of domains can be called subdomains. The primary domain is used to manage and organize other subdomains. For example, in www.example.com, the primary domain is example.com.

[0098] This application relates to Artificial Intelligence (AI) technology and cloud computing. AI technology and cloud computing can be applied to many fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, smart transportation, smart maps, driver assistance systems, vehicle terminals, aircraft, digital twins, virtual humans, robots, AI-generated content (AIGC), conversational interaction, smart healthcare, smart customer service, game AI, etc. It is believed that with the development of technology, AI technology and cloud computing will be applied in more fields and play an increasingly important role.

[0099] It should be noted that the embodiments of this application involve operations such as obtaining sub-URLs. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0100] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0101] The application areas of the website matching method provided in the embodiments of this application will be briefly introduced below.

[0102] With the continuous development of technology, more and more client or server devices can generate website fingerprints. These fingerprints are used as unique identifiers to match websites, thereby enabling intelligent website analysis.

[0103] For example, in the scenario of identifying abnormal websites, please refer to... Figure 1A After receiving an identification request for a website to be matched, a website fingerprint of the website to be matched can be generated based on the web page content of each web page involved in the website to be matched. The website fingerprint of each abnormal website can be obtained from the stored abnormal website database. Based on the obtained website fingerprints, the website to be matched and each abnormal website can be matched to obtain the identification result of whether the website to be identified is an abnormal website.

[0104] For example, in a website recommendation scenario, when accessing a website to be matched, a recommendation request can be generated for the website to be matched. Based on the web page content involved in the website to be matched, a website fingerprint of the website to be matched can be generated. The website fingerprints of each candidate website can be obtained from the stored candidate website library. Based on the obtained website fingerprints, the website to be matched can be matched with each candidate website to determine the recommended website as recommendation feedback, thereby achieving the purpose of website promotion.

[0105] In related technologies, website matching methods typically involve using a web crawler to traverse a webpage within a website and obtain its content; then, encoding the obtained webpage content to generate a website fingerprint, which can then be used to match websites.

[0106] Please refer to Figure 1BHere is a method for generating a website fingerprint: S101, receive the target URL of the website, such as www.xxx.com; S102, use a web page content crawling program to extract the standard markup language (HyperText Markup Language, HTML) file of the target URL, such as the HTML file name is index.html; S103, parse the Document Object Model (DOM) tree of the HTML file to obtain the DOM sequence; S104, encode the DOM sequence to obtain the website fingerprint of the website.

[0107] However, using web scraping programs for website matching has significant limitations, for example:

[0108] First, it can easily lead to the leakage of some important information on the website, causing unnecessary security risks in the website matching process;

[0109] Second, for websites that have set up anti-scraping mechanisms, it will be impossible to obtain the corresponding web page content, and thus impossible to generate the website fingerprint. Therefore, it will be impossible to match websites based on the website fingerprint.

[0110] It is evident that the matching process of websites using the relevant technologies has significant limitations.

[0111] To address the limitations of website matching, this application proposes a website matching method. This method involves extracting features from multiple sub-websites belonging to a main URL, obtaining a feature set for each sub-website. These feature sets are then fused and encoded to obtain the website fingerprint of a first website, which includes the main URL and the web pages corresponding to each of the sub-websites. Based on the fingerprint similarity between the first website's fingerprint and the fingerprints of multiple pre-stored reference websites, a website matching result is generated.

[0112] In this embodiment, a website fingerprint of the first website is generated based on the URL feature sets extracted from multiple sub-URLs belonging to a main URL. This process only involves data processing for the sub-URLs; it does not require obtaining webpage content or processing related data within the webpage content. Since the sub-URLs are completely public, no additional program for crawling webpage content is needed. Similarly, since the URL feature sets can be extracted from the URLs of the sub-URLs themselves, without processing the webpage content linked to by the sub-URLs, no additional program for crawling webpage content is required.

[0113] This application's embodiments avoid the leakage of important information in web page content due to web page crawling programs, thus improving data security during the website matching process. Simultaneously, it prevents situations where websites with anti-crawling mechanisms cannot obtain the corresponding web page content, thereby preventing the generation of the first website's website fingerprint. This increases the success rate of generating website fingerprints during the website matching process, thereby solving the matching limitations in the website matching process.

[0114] Furthermore, by fusing and encoding the feature sets of each URL into a website fingerprint, which integrates the website characteristics reflected by multiple sub-URLs, the accuracy of the website fingerprint can be improved, thereby improving the accuracy of website matching.

[0115] The application scenarios of the website matching method provided in this application are described below.

[0116] Please refer to Figure 1C This is a schematic diagram illustrating an application scenario of the website matching method provided in this application. The application scenario includes a server 101 and a client 102, which can communicate with each other. The communication method can be wired, such as through a network cable or serial cable; or wireless, such as through Bluetooth or Wi-Fi. No specific limitation is imposed.

[0117] Client 102 generally refers to a device that can indicate a website to be matched or display website matching results, such as a terminal device, a third-party application accessible by the terminal device, or a webpage accessible by the terminal device. Server 101 generally refers to a device that can perform website matching, such as a terminal device or a server.

[0118] Terminal devices include, but are not limited to, mobile phones, computers, smart medical devices, smart home appliances, vehicle terminals, or aircraft. Servers include, but are not limited to, cloud servers, local servers, or associated third-party servers. Both client 101 and server 102 can use cloud computing to reduce the use of local computing resources; similarly, they can also use cloud storage to reduce the use of local storage resources.

[0119] As one embodiment, the server 101 and the client 102 can be the same device, or they can be different devices, or they can be different devices with some modules shared, etc., and there are no specific restrictions.

[0120] In response to a trigger operation targeting a URL, client 102 sends a website matching instruction to server 101 for the first website corresponding to that URL. Server 101 receives the website matching instruction from client 102, performs website matching for the first website, and sends the website matching result back to client 102. Client 102 receives and displays the website matching result sent by server 101.

[0121] The following is based on Figure 1C This document provides a detailed description of the website matching method provided in the embodiments of this application. Please refer to [link / reference]. Figure 2 This is a flowchart illustrating a website matching method provided in an embodiment of this application.

[0122] S201, extract features from multiple sub-websites belonging to a main website to obtain the URL feature sets corresponding to each sub-website.

[0123] Multiple sub-websites belonging to a main website represent multiple sub-websites that contain the same domain name as the main website, such as the same top-level domain, main domain, or second-level domain. In this embodiment, we will take the example of multiple sub-websites containing the same main domain name as the main website.

[0124] The webpages linked by the main URL and the webpages linked by each of the multiple sub-URLs are contained in the corresponding set of webpages on the first website. For example, the webpage linked by the main URL is the homepage of the first website, and the webpages linked by the sub-URLs are other webpages on the first website.

[0125] In addition to the domain name, sub-URLs can also contain access paths, which link to the corresponding web pages. If multiple sub-URLs represent URLs accessed when visiting the first website at a specific historical time, then the access paths contained within the sub-URLs can be called historical access paths. Historical time refers to a period of time prior to the current moment; the duration and start and end times of the historical time can be set according to the specific scenario and are not limited here.

[0126] For example, the homepage of the first website corresponds to the main URL, which is http: / / xxxxx.com, and the webpage providing the map function on the first website corresponds to the sub-URL, which is "http: / / map.xxxxx.com". Then, in the sub-URL http: / / map.xxxxx.com, the main domain is "xxxxx.com", and the access path is "map". Through this access path, one can link to the webpage providing the map function on the first website.

[0127] By identifying multiple sub-websites belonging to a main website, features can be extracted from each sub-website to obtain a set of website features for each sub-website. These sub-websites can reflect access characteristics specific to the main website, such as the protocol used, the hostname used, the service port, the resource path, resource query parameters, or anchor points. Therefore, the set of website features for each sub-website can be used as the basis for generating the website fingerprint of the main website, achieving the goal of uniquely representing the website.

[0128] As one embodiment, there are several methods for obtaining multiple sub-websites belonging to a main website. Two of these methods are described below as examples; other methods are not listed here. After obtaining the multiple sub-websites, feature extraction is performed on each sub-website to obtain its own URL feature set.

[0129] Method 1:

[0130] The system receives website matching requests for the first website. These requests can be received based on the actual scenario. For example, when accessing the main URL or a sub-URL of the first website through a client, the client generates and sends a website matching request for the first website to the server. Alternatively, the client may generate and send a website matching request for the first website to the server in response to a matching operation triggered by the main URL or a sub-URL of the first website. There are no specific restrictions on the specific requests.

[0131] From the access database, query multiple sub-websites belonging to the main URL of the first website. The access database is used to store the URLs accessed in historical time. The access database is, for example, a database used to record the access data of each website in historical time. Each website can be multiple preset websites, or websites detected to be accessed through one or more terminal devices, etc., and there are no specific restrictions.

[0132] By accessing a database and managing access data from various websites in a unified manner, the data formats of each access record are consistent, eliminating the need for format conversion and improving the efficiency of sub-URL queries. Responding to website matching requests allows for targeted website matching of the requested websites, enhancing the interactivity of the matching process. From multiple sub-URLs accessed historically, access characteristics specific to the primary website can be identified, such as easily accessed or frequently accessed sub-URLs. These URL feature sets from multiple sub-URLs can then be used as the basis for generating a website fingerprint for the primary website, achieving the goal of uniquely identifying the website.

[0133] Please refer to Figure 3AThe client can respond to a matching operation triggered for a target URL (such as http: / / xxxxx.com) and obtain a website matching request. The client sends a website matching request to the server, and the server receives the website matching request sent by the client. Based on the website matching request, the server obtains the main domain name (such as xxxxx.com) contained in the target URL.

[0134] The server queries the associated access database for multiple sub-URLs containing the main domain name to obtain multiple sub-URLs belonging to a single main URL. The access database records access data from multiple specified clients accessing the URLs.

[0135] Method 2:

[0136] When an access request for the main URL or a sub-URL of the first website is detected, multiple sub-URLs belonging to the main URL of the first website are retrieved from the access records of each application. These multiple sub-URLs can be obtained by retrieving the main domain name contained in the main URL or a sub-URL in the access request, and then retrieving multiple sub-URLs containing that main domain name from the respective application's access records.

[0137] When an access request for the main URL or a sub-URL of the first website is detected, for example, when an application accesses the main URL or a sub-URL of the first website, the application's associated program server receives the access request and sends the access request to the first website matching server, thereby the first website matching server detects the access request for the main URL or a sub-URL of the first website.

[0138] For example, each application's installation package may include a detection program. When an application accesses the main URL or a sub-URL of the first website, the detection program will determine that an access request for the main URL or a sub-URL of the first website has been detected.

[0139] Access records contain records of accesses to various URLs within the corresponding applications. These applications can be multiple pre-defined applications or multiple applications with URL access functionality published on network resources, such as applications with URL access functionality and download volumes exceeding a pre-defined download threshold.

[0140] Access records can be managed in batches by application, reducing the data volume of each record. This allows for parallel querying of sub-URLs across different records, improving query efficiency. Website matching is performed upon detecting an access request for a target URL, enabling an automated matching process that requires waiting for a response, thus enhancing the intelligence of the website matching process.

[0141] Please refer to Figure 3B When application A accesses a target URL (e.g., http: / / map.xxxxx.com), an access request for that URL is detected. The server extracts the main domain name (e.g., xxxxx.com) contained in the target URL from the access request. The server then obtains the access records of each application B and queries each access record for multiple sub-URLs containing that main domain name.

[0142] As one example, there are various methods for feature extraction from sub-URLs. Two of them will be introduced below as examples. Other feature extraction methods will not be listed here.

[0143] Feature extraction method 1:

[0144] For multiple sub-URLs, the following steps are performed: Extract the access path from each sub-URL. When the access path contains resource path information representing the webpage resource address, divide the resource path information based on a preset resource path separator to obtain at least one resource sub-information. Based on at least one resource sub-information, establish a URL feature set.

[0145] The access path is used to link to a webpage that presents resources, such as a webpage that presents a product list; or a webpage that presents detailed information about product A, etc., with no specific restrictions. The access path can include resource path information representing the webpage resource address, or query path information representing resource query parameters, etc. The resource path information representing the webpage resource address is used to link to a webpage that presents multiple resources, while the query path information representing resource query parameters is used to link to a webpage that presents one of those multiple resources.

[0146] The resource path separator can be set according to the URL writing strategy. For example, if the URL uses " / " as the separator in the path, then " / " can be preset as the resource path separator. Other possible resource path separators include "?", ":", and "&", etc., with no specific restrictions.

[0147] When determining the access path, which contains resource path information representing webpage resource addresses, since the resource path information may contain many characters, the information can be divided into segments, with each resource sub-information treated as a URL feature. This allows the website fingerprint generated based on the URL feature set to integrate comprehensive feature information from the sub-URLs, improving the uniqueness of the website fingerprint and thus enhancing the accuracy of website matching.

[0148] If the access path does not contain resource path information that represents the webpage resource address, other information in the access path can be determined, and a URL feature set can be built based on the other information.

[0149] Please refer to Figure 4A In (1), the resource path information can contain only one resource sub-information, which can link to a webpage. For example, in http: / / map.xxxxx.com, the resource path information is "map", and the divided resource sub-information is also "map". Then the corresponding URL feature set includes "map".

[0150] Please refer to Figure 4A In (2), the resource path information can contain multiple resource sub-information. Each resource sub-information represents a path that links to a webpage or a resource. For example, http: / / www.xxx.com / topic / 245836.html, the resource path information is "topic / 245836.html". Based on the resource path separator " / ", two resource sub-information can be divided into "topic" and "245836.html". Then the corresponding URL feature set includes "topic" and "245836.html".

[0151] Feature extraction method two:

[0152] For multiple sub-URLs, the following steps are performed: Each sub-URL is input into several trained feature extraction models, which output the corresponding access path features to obtain a URL feature set. Different feature extraction models have varying feature extraction capabilities. For example, the semantic recognition accuracy based on features extracted by feature extraction model A is higher than that based on features extracted by feature extraction model B; conversely, the character recognition accuracy based on features extracted by feature extraction model B is higher than that based on features extracted by feature extraction model A.

[0153] For example, the accuracy of webpage content type identification based on features extracted by feature extraction model C is higher than that based on features extracted by feature extraction model D; while the accuracy of keyword identification based on features extracted by feature extraction model D is higher than that based on features extracted by feature extraction model C.

[0154] As one embodiment, during feature extraction, the access path can be cleaned first, replacing characters with low relevance to uniquely representing the website with uniform preset characters. This reduces misleading factors during feature extraction and improves accuracy. The character cleaning process will be described using the first feature extraction method described above as an example.

[0155] The relevance of various characters to the first website can be determined based on the URL format, thereby setting the characters to be replaced; alternatively, a trained relevance recognition model can be used to determine the relevance between various characters and the first website, thereby determining the characters to be replaced based on the recognition results, etc., with no specific restrictions.

[0156] The preset character can have the same character format as the character to be replaced, such as both being in numeric format; or it can have a different character format than the character to be replaced, such as the preset character being in numeric format and the character to be replaced being an uncommon character or "_", etc. There are no specific restrictions. In this embodiment, the example of the preset character having the same character format as the character to be replaced is used for illustration.

[0157] Then, replace the path characters in the access path that match the character format of the preset characters with the preset characters to obtain the access cleaning path.

[0158] Please refer to Figure 4B For the sub-URL "http: / / www.xxx.com / topic / 245836.html", the access path is "topic / 245836.html". Since different numbers are usually used to distinguish different objects within the same category, such as distinguishing each product in the clothing category, numbers can be considered characters with low relevance to the website. Therefore, the preset character "1" can be used to replace all numbers to achieve character cleaning, resulting in the cleaned access path "topic / 111111.html".

[0159] After obtaining the access cleaning path, at least one path information can be extracted from the access cleaning path based on the preset URL writing format. This path information can represent a web page resource address or a resource query parameter. Thus, at least one path information can represent all web page resource addresses, all resource query parameters, or both.

[0160] Please refer to Figure 4C In (1), for the sub-URL "http: / / www.xxx.com / topic / 245836.html", the access path is "topic / 245836.html", and the access cleansing path is "topic / 111111.html". A path information is extracted from the access cleansing path, which is "topic / 111111.html". This path information represents the webpage resource address.

[0161] Please refer to Figure 4CIn (2), for the sub-URL "http: / / www.xxx.com / ?jump_from=1_05_37_01", the access path is "?jump_from=1_05_37_01", and the access cleaning path is "?jump_from=1_11_11_1". Based on the symbol "?", a path information is extracted from the access cleaning path, which is "jump_from=1_11_11_1". This path information represents the resource query parameters.

[0162] Please refer to Figure 4C In (3), for the sub-URL "http: / / www.xxx.com / topic / ?entryScene=1&jump_from=2", the access path is "topic / ?entryScene=1&jump_from=2", and the access cleaning path is "topic / ?entryScene=1&jump_from=1". Based on the symbol "?", two path information is extracted from the access cleaning path, namely "topic" and "entryScene=1&jump_from=1". Among them, the path information "topic" represents the web page resource address, and the path information "entryScene=1&jump_from=1" both represent resource query parameters.

[0163] Therefore, when determining that at least one path information includes resource path information representing the address of a webpage resource, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

[0164] Please continue to refer to this. Figure 4C Based on the resource path separator " / ", the path information "topic / 111111.html" is divided into two resource sub-information pieces: "topic" and "111111.html". The path information "jump_from=1_11_11_1" is also divided into one resource sub-information piece: "jump_from=1_11_11_1". The path information "topic" is further divided into one resource sub-information piece: "topic". Finally, based on the resource path separator "&", the path information "entryScene=1&jump_from=1" is divided into two resource sub-information pieces: "entryScene=1" and "jump_from=1".

[0165] As one embodiment, when the access path includes query path information representing resource query parameters, a URL feature set can be obtained based on the query path information; when the access path includes both resource path information representing webpage resource addresses and query path information representing resource query parameters, a URL feature set can be obtained based on both the resource path information and the query path information. Combining the foregoing description, the query path information and at least one resource sub-information can each be used as URL features to obtain a URL feature set.

[0166] URL feature sets can reflect the access characteristics of web page resources and query parameters in the corresponding sub-URLs from multiple perspectives, which can improve the uniqueness of the website fingerprint determined based on the URL feature set, thereby improving the accuracy of website matching.

[0167] Please refer to Table 1, which contains resource path information representing webpage resource addresses and query path information representing resource query parameters in the access paths of each of the multiple sub-URLs.

[0168] Table 1

[0169]

[0170] As one example, when the resource query parameter is in the form of key-value pairs, since the specific value has low relevance to the website, the specific value can be discarded, and only the key in the key-value pair can be used as the URL feature. For example, if the resource query parameter is "jump_from=1_05_37_01", where the key is "jump_from" and the value is "1_05_37_01", then "jump_from" can be used as a URL feature.

[0171] S202, fuse and encode the feature sets of each URL to obtain the website fingerprint of the first website.

[0172] After obtaining the URL feature sets corresponding to each sub-URL, the obtained URL feature sets can be fused and encoded to obtain the website fingerprint of the first website. Fusion encoding is used to fuse the URL features contained in the URL feature sets through encoding, and to fuse the URL feature sets themselves, finally generating the website fingerprint of the first website. The first website includes the main URL and the web pages corresponding to each of the multiple sub-URLs.

[0173] As one example, there are various methods of fusion coding. Two of them will be introduced below as examples. Other fusion coding methods will not be listed here.

[0174] Fusion coding method one:

[0175] For each obtained URL feature set, the following steps are performed: Based on the corresponding position of each URL feature in its respective sub-URL, the directory hierarchy of each URL feature is determined. Based on the obtained directory hierarchy, the URL features are concatenated to obtain the corresponding hierarchical URL features. The directory hierarchy can use quantified values ​​to represent the position of URL features in sub-URLs, thus reflecting the order in which the URL features appear within the sub-URLs. This order further increases the uniqueness of the website fingerprint, thereby improving the accuracy of website matching.

[0176] Each obtained hierarchical URL feature is encoded to obtain the corresponding encoded features. The obtained encoded features are then fused to obtain the website fingerprint of the first website.

[0177] There are several methods for determining directory hierarchy based on corresponding locations. Two examples are introduced below.

[0178] One method for determining directory hierarchy is to determine the directory hierarchy value by placing the URL feature closer to the access protocol location within the corresponding sub-URL, and conversely, placing the URL feature further away from the access protocol location within the corresponding sub-URL, resulting in a larger directory hierarchy value.

[0179] For example, a sub-URL like "http: / / www.xxx.com / topic / 245836.html" has a set of URL features consisting of two features: "topic" and "245836.html". Since the position of "topic" within the sub-URL is closer to the access protocol "http" than the position of "245836.html", the directory hierarchy value for "topic" is less than that for "245836.html". For instance, the directory hierarchy value for "topic" is 1, while that for "245836.html" is 2.

[0180] Please refer to Table 2, which shows a form of directory hierarchy corresponding to each obtained URL feature.

[0181] Table 2

[0182]

[0183] Another method for determining directory hierarchy is to consider the URL characteristics representing webpage resource addresses. The closer these characteristics are to the URL characteristics representing resource query parameters, the smaller the directory hierarchy value; conversely, the further away they are from the URL characteristics representing resource query parameters, the larger the directory hierarchy value. Similarly, the closer the URL characteristics representing resource query parameters are to the URL characteristics representing webpage resource addresses, the smaller the directory hierarchy value; and the further away they are from the URL characteristics representing webpage resource addresses, the larger the directory hierarchy value. Furthermore, the directory hierarchy value for the URL characteristics representing resource query parameters is always greater than the directory hierarchy value for the URL characteristics representing webpage resource addresses.

[0184] For example, a sub-URL like "http: / / www.xxx.com / search / index?entryScene=1&jump_from=2" has a URL feature set consisting of four features: "search", "index", "entryScene", and "jump_from". "search" and "index" are URL features representing the webpage resource address, while "entryScene" and "jump_from" are URL features representing the resource query parameters. Therefore, the directory hierarchy for "search" and "index" is 2 and 1 respectively; and the directory hierarchy for "entryScene" and "jump_from" is 3 and 4 respectively.

[0185] Directory concatenation is used to combine directory hierarchy and corresponding URL features into a single string, which serves as the corresponding hierarchical URL feature. This allows us to obtain the hierarchical URL features corresponding to each URL feature set. Each obtained hierarchical URL feature is then encoded to obtain its corresponding encoded feature. These encoded features are then fused to obtain the website fingerprint of the first website. Feature encoding can take various forms, such as binary encoding, hash encoding, hexadecimal encoding, etc., with no specific restrictions.

[0186] By using directory hierarchy, the positional characteristics of URL features within sub-URLs can be further enhanced in the URL feature set. This avoids character confusion between different paths, which can lead to inaccurate website fingerprints and improve the accuracy of website fingerprints, thereby improving website matching accuracy.

[0187] As one example, there are several methods for concatenating directories. Three of them will be introduced below as examples, and other methods will not be listed here.

[0188] Table of Contents Assembly Method 1:

[0189] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the corresponding hierarchical URL features.

[0190] By directly combining directory hierarchy with URL features, the access characteristics of sub-URLs can be accurately reflected from both content and location perspectives, improving the uniqueness of the generated website fingerprint and thus enhancing the accuracy of website matching.

[0191] Please refer to Figure 5A The sub-URL is "http: / / www.xxx.com / topic / 245836.html", and its corresponding URL feature set includes two URL features: "topic" and "111111.html". The directory level of "topic" is 1, and the directory level of "111111.html" is 2. Therefore, we can obtain two hierarchical URL features: "1topic" and "2111111.html".

[0192] Please refer to Table 3, which shows one form of the hierarchical URL features obtained from the directory hierarchy corresponding to each URL feature obtained from Table 2.

[0193] Table 3

[0194]

[0195] Method 2 for assembling a table of contents:

[0196] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the corresponding directory URL feature. Then, each pair of adjacent directory URL features within each directory hierarchy is concatenated to obtain the corresponding hierarchical URL feature.

[0197] The URL feature with the smallest or largest value in the directory hierarchy can also be directly used as the hierarchical URL feature. For example, the URL feature of the access protocol location in the nearest sub-URL can be directly used as the hierarchical URL feature.

[0198] By directly concatenating directory hierarchy with URL features and then horizontally concatenating them with adjacent URL features, the access characteristics of sub-URLs can be accurately reflected from three perspectives: content, location, and URL feature association. This can further improve the uniqueness of the generated website fingerprint, thereby improving the accuracy of website matching.

[0199] Please refer to Figure 5BThe sub-URL is "http: / / www.xxx.com / search / index?entryScene=1&jump_from=2", and its corresponding URL feature set includes four URL features: "search", "index", "entryScene", and "jump_from". The directory levels for "search", "index", "entryScene", and "jump_from" are 1, 2, 3, and 4, respectively. Therefore, we can obtain three hierarchical URL features: "1search_2index", "2index_3entryScene", and "3entryScene_4jump_from".

[0200] Please refer to Table 4, which shows one form of the hierarchical URL features obtained from the directory hierarchy corresponding to each URL feature obtained from Table 2.

[0201] Table 4

[0202]

[0203]

[0204] Method 3 for assembling a table of contents:

[0205] For each URL feature, concatenate every two adjacent URL features in the directory hierarchy to obtain the corresponding adjacent URL features. For each obtained adjacent URL feature, perform the following: concatenate the directory hierarchy corresponding to each of the two URL features contained in the adjacent URL feature with the adjacent URL feature to obtain the corresponding hierarchical URL feature.

[0206] Please refer to Figure 5C The sub-URL is "http: / / www.xxx.com / search / index?entryScene=1&jump_from=2", and its corresponding URL feature set includes four URL features: "search", "index", "entryScene", and "jump_from". The directory levels for "search", "index", "entryScene", and "jump_from" are 1, 2, 3, and 4, respectively. Therefore, we can obtain three hierarchical URL features: "12search_index", "23index_entryScene", and "34entryScene_jump_from".

[0207] As one example, when performing fusion encoding, the access counts of multiple sub-URLs can be combined for fusion encoding. From the perspective of the promotion of sub-URLs, this further reflects an access characteristic of the sub-URLs, which can further improve the uniqueness of the generated website fingerprint, thereby improving the accuracy of website matching.

[0208] Alternatively, the frequency of occurrence of each string within a historical period can be combined for fusion encoding. The frequency of string occurrence further reflects the access characteristics of the sub-URL, which can improve the uniqueness of the generated website fingerprint and thus increase the accuracy of website matching. When the identified URL features are not among the statistically analyzed strings, the frequency of the response features can be determined based on the average frequency of occurrence of each string. This avoids problems such as low encoding accuracy due to excessively large weight differences. The specific weight data referenced during fusion encoding is not limited. The following example, combining access frequency, illustrates the fusion encoding process.

[0209] If multiple sub-URLs are associated with access counts, then the frequency of each hierarchical URL feature can be calculated based on the access counts associated with each sub-URL. The hash value of each hierarchical URL feature is then determined to obtain the corresponding encoded feature. Based on the obtained feature frequencies, the encoded features are merged to obtain the website fingerprint of the first website.

[0210] One method to count the frequency of hierarchical URL features is to determine the sum of the access counts of sub-URLs that exhibit the corresponding hierarchical URL feature within each sub-URL. For example, hierarchical URL feature A corresponds to URL feature A. Among the sub-URLs, there are two sub-URLs exhibiting URL feature A: sub-URL B and sub-URL C. Sub-URL B has 15 access counts, and sub-URL C has 18 access counts. Therefore, the frequency of hierarchical URL feature A is 15 + 18 = 33.

[0211] Another method for calculating the frequency of hierarchical URL features is the ratio of the sum of access counts of sub-URLs corresponding to that hierarchical URL feature to the total number of access counts of all sub-URLs. For example, hierarchical URL feature A corresponds to URL feature A. Among the sub-URLs, there are two sub-URLs that contain URL feature A: sub-URL B and sub-URL C. Sub-URL B has 15 access counts, sub-URL C has 18 access counts, and the total number of access counts for all sub-URLs is 500. Therefore, the frequency of hierarchical URL feature A is 33 / 500.

[0212] There are various methods to determine the hash value, such as BKDRHash, APHash, DJBHash, etc., without any specific restrictions. A custom hash calculation method can also be used to determine the hash value. For example, define a hash calculation function to calculate the hash value of the input parameter; if the input parameter is empty, return 0; if the input parameter is not empty, define variable x as the result of shifting the ASCII code of the first character left by 7 bits, such as x = ord(input parameter [0]) << 7; define variable m as a large prime number, such as 1000003, for hash operations; define a mask, which is 2**128-1, to ensure that the result after each multiplication does not exceed 128 bits; enter the for loop, which iterates through each character c in the input parameter. In the for loop, multiply the current x by m, then XOR the ASCII code of the character c, and finally... Then perform a bitwise AND operation with mask to ensure that the result does not overflow. Use the obtained result to update the value of x, such as x = ((x*m)^ord(c))&mask; the for loop iterates through each character c in the input parameter and ends. After the loop ends, perform an XOR operation between x and the length of the input parameter string, such as x^ = len(input parameter); if x equals -1, update it to -2 to prevent potential problems with the two's complement representation of negative numbers; convert x to a binary string, remove the prefix '0b', and fill the left side with zeros to make its total length reach 64 bits, such as x = bin(x)('0b', '').zfill(64[-64:]); return the final string form of x as the hash value.

[0213] Please refer to Table 5, which shows one form of the number of visits associated with each of the multiple sub-URLs.

[0214] Table 5

[0215] Sub-website Number of visits http: / / www.xxx.com / topic / 245836.html 15 http: / / www.xxx.com / kj / 3 / 2023.html 10 http: / / www.xxx.com / ?entryScene=zhida_05_001&jump_from=1_13_18_00 5 http: / / www.xxx.com / topic / 240552.html 8 http: / / www.xxx.com / topic / 241637.html 14 http: / / www.xxx.com / ?jump_from=1_05_37_01 18 http: / / www.xxx.com / kj / zs / 3 / 2023.html 13 http: / / www.xxx.com / ?jump_from=1_13_18_00&entryScene=zhida_05_001 12 http: / / www.xxx.com / topic / 245775.html 50 …… ……

[0216] Please refer to Table 6, which shows a form of the feature frequency of each hierarchical URL feature obtained using the above directory splicing method one and directory splicing method two.

[0217] Table 6

[0218]

[0219]

[0220] As one embodiment, when determining the hash value of each hierarchical URL feature to obtain the corresponding encoded feature, and fusing the obtained encoded features based on the frequency of each feature to obtain the website fingerprint of the first website, the hierarchical URL features can first be sorted in descending order based on the frequency of each feature to obtain a hierarchical URL feature sequence. Multiple target URL features are then selected from the hierarchical URL feature sequence in a specified number from beginning to end. The hash value of each target URL feature is then determined to obtain the corresponding encoded feature. Finally, the website fingerprint of the first website is obtained by fusing the obtained encoded features based on the frequency of each feature. This approach reduces computational load and improves website matching efficiency.

[0221] Please refer to Table 7, which shows one form of each coding feature obtained based on Table 6.

[0222] Table 7

[0223]

[0224]

[0225] As one example, when fusing the obtained coding features based on the frequency of each feature, the frequency of each feature can be used as a weight, and the obtained coding features can be weighted and summed based on the frequency of each feature to obtain a comprehensive feature; alternatively, the coding features with the highest frequency and the coding features with the lowest frequency can be combined in a loop according to the magnitude of each feature frequency, and then the concatenated coding features can be summed to obtain a comprehensive feature, etc., without any specific limitations.

[0226] When performing weighted summation on the obtained encoded features, the 0 values ​​in the encoded features can be converted to -1 before weighted summation to avoid problems such as the result of the comprehensive features being too large.

[0227] Therefore, based on a preset binary mapping strategy, the comprehensive features can be mapped into a binary sequence to obtain the website fingerprint of the first website.

[0228] By fusing feature frequency and encoding features, the resulting comprehensive features can not only reflect the characteristics of the URL, but also the attention level of the sub-URL, further reflecting an access characteristic of the sub-URL. This can further improve the uniqueness of the generated website fingerprint, thereby improving the accuracy of website matching.

[0229] Please refer to Table 8 for a form of weighting of each coding feature.

[0230] Table 8

[0231]

[0232]

[0233] The weighted coding features are summed to obtain the comprehensive features as follows: -309, -361, -509, -317, -187, 101, -243, -25, 31, -21, etc., which will not be listed here.

[0234] Binary mapping strategies are used to map comprehensive features into binary sequences containing only 0s and 1s. There can be various binary mapping strategies, such as mapping values ​​greater than or equal to 0 to 1 and values ​​less than 0 to 0; mapping values ​​greater than or equal to a specified value to 1 and values ​​less than a specified value to 0; mapping single-digit or two-digit numbers to 0 and three-digit or longer numbers to 1, etc. The specific strategy is not limited. The specified value can be preset; it can also be determined based on the frequency of each feature, ensuring that the proportion of 0s and 1s in the resulting binary sequences is approximately equal, etc., and the specific strategy is not limited.

[0235] Please refer to Table 9, which shows a binary mapping process where values ​​less than 0 are mapped to 0, and values ​​greater than or equal to 0 are mapped to 1.

[0236] Table 9

[0237] Comprehensive features Encoding features -309 0 -361 0 -509 0 -317 0 -187 0 101 1 -243 0 -25 0 31 1 …… …… 86 1 184 1 -104 0 -90 0 -58 0

[0238] Fusion coding method two:

[0239] For each obtained URL feature set, the following steps are performed: Using the pre-defined quantization features of each reference character, feature mapping is performed on each URL feature in a given URL feature set to obtain the corresponding quantized URL features. The obtained quantized URL features are then weighted and summed to obtain the set-quantized features of the URL feature set. Quantized features can be in vector or matrix form, etc., without specific restrictions.

[0240] The obtained quantitative features of each set are weighted and summed to obtain the website quantitative features. The website quantitative features are then encoded to obtain the website fingerprint of the first website.

[0241] As one example, the website fingerprint of the first website can be of a specified length, such as 64 bits. The specific length can be set according to the usage scenario. For example, a 128-bit website fingerprint can be set for abnormal website identification, and a 64-bit website fingerprint can be set for recommendation scenarios.

[0242] S203, Based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of each of the pre-stored reference websites, generate website matching results.

[0243] After obtaining the fingerprint of the first website, website matching results can be generated based on the fingerprint similarity between the first website's fingerprint and the pre-stored reference website fingerprints of each reference website. The pre-stored reference website fingerprints of each reference website can be obtained in the same way as the first website's fingerprint, thereby improving matching accuracy. The reference website fingerprints of each reference website can be determined synchronously with the first website's fingerprint, or they can be predetermined; there are no specific restrictions.

[0244] Matching can be performed by comparing the website fingerprints of two websites to perform tasks in real-world scenarios. For example, by determining the fingerprint similarity between the website fingerprint of an unknown website and the website fingerprint of a known abnormal website, the task of determining whether an unknown website is an abnormal website can be performed. Or, by determining the fingerprint similarity between the website fingerprints of multiple websites, the task of clustering similar websites can be performed. There are no specific limitations.

[0245] Fingerprint similarity can be determined using Hamming distance, Euclidean distance, or similarity recognition models, without any specific restrictions.

[0246] The website matching results may include the fingerprint similarity between the first website and each reference website, or at least one similar website among the reference websites whose fingerprint similarity with the first website is greater than the reference similarity threshold, or other information related to the website matching scenario, without any specific restrictions.

[0247] As one embodiment, in an abnormal website identification scenario, the fingerprints and abnormal website types of multiple reference websites can be obtained from a pre-stored fingerprint blacklist. The Hamming distance between the fingerprint of the first website and the fingerprints of each of the other reference websites is determined to obtain the corresponding fingerprint similarity. If at least one reference website has a fingerprint similarity greater than a preset similarity threshold, the abnormal website type of the first website is determined based on the abnormal website type of that at least one reference website, thus obtaining a website matching result. Therefore, the abnormal website type of the first website can be identified through website matching. This allows for rapid identification of the abnormal website type without needing to crawl or identify the webpage content, improving the efficiency of anomaly detection. If no reference website has a fingerprint similarity greater than the preset similarity threshold, then the first website can be determined to be a non-abnormal website.

[0248] If at least one reference website has more than a specified number of reference websites with the same website exception type, then that website exception type can be designated as the first website exception type. The specified number can be determined based on the number of at least one reference website, etc., and is not specifically limited.

[0249] Please refer to Figure 6A The reference website fingerprint and website anomaly type are obtained from the fingerprint blacklist for each reference website. For example, abnormal fingerprint_1 and its corresponding anomaly type_1, abnormal fingerprint_2 and its corresponding anomaly type_2, ..., and abnormal fingerprint_N and its corresponding anomaly type_N.

[0250] Taking the abnormal fingerprint_1 and its corresponding abnormal type_1 as an example, the Hamming distance between the website fingerprint of the first website and the abnormal fingerprint_1 is determined. The smaller the Hamming distance, the higher the fingerprint similarity; the larger the Hamming distance, the lower the fingerprint similarity. An XOR operation is performed on the website fingerprint of the first website and the abnormal fingerprint_1 using the Hamming distance. If the Hamming distance is 1, then it can be determined that their fingerprint similarity is greater than a preset similarity threshold.

[0251] This allows us to identify at least one reference website among the reference websites whose fingerprint similarity exceeds a preset similarity threshold. For example, this could be the reference website corresponding to each of the abnormal fingerprints_1 and_2. If the anomaly type of the reference websites corresponding to both abnormal fingerprints_1 and_2 is abnormal_type_1, then the website anomaly type of the first website is abnormal_type_1. If the anomaly types of the reference websites corresponding to abnormal fingerprints_1 and_2 are different, other methods can be used to further determine the website anomaly type of the first website.

[0252] As one embodiment, in a website classification scenario, the website fingerprint of a first website can be determined, along with the fingerprint similarity between the first website and the reference website fingerprints of each reference website obtained from a pre-stored set of fingerprints to be matched, and the fingerprint similarity between any two reference website fingerprints. Based on the obtained fingerprint similarities, clustering is performed on the first website and each reference website to obtain at least one website cluster. Each website cluster contains at least one website to be classified, which can be either the first website or a reference website. The fingerprint similarity of websites to be classified within the same website cluster is greater than the fingerprint similarity between websites to be classified in different website clusters. Based on at least one website cluster, the website categories of the first website and each reference website are identified to obtain website matching results. By matching websites, at least one reference website similar to the first website can be identified, thus enabling the combination of multiple websites for website classification. This avoids the situation where there is insufficient classification reference data for classifying a single website, resulting in inaccurate classification and improving the accuracy of website classification.

[0253] Please refer to Figure 6B The website fingerprint of the matching website is denoted as website fingerprint 1, and the reference website fingerprints of each reference website are denoted as website fingerprint 2_1, website fingerprint 2_2, ... and website fingerprint 2_N, respectively.

[0254] The fingerprint similarity between each pair of website fingerprints, including website fingerprint 1, website fingerprint 2_1, website fingerprint 2_2, and so on, is determined. Based on the obtained fingerprint similarity, the first website and each reference website are clustered to obtain three website clusters: website cluster A, website cluster B, and website cluster C. Website cluster A contains website fingerprint 1, website fingerprint 2_2, and website fingerprint 2_M; website cluster B contains website fingerprint 2_1, website fingerprint 2_X, and website fingerprint 2_Y; and website cluster C contains website fingerprint 2_Z and other website fingerprints.

[0255] Website category identification is performed based on website fingerprint 1, website fingerprint 2_2, and website fingerprint 2_M to obtain the search type; website category identification is performed based on website fingerprint 2_1, website fingerprint 2_X, and website fingerprint 2_Y to obtain the audio-visual type; and website category identification is performed based on other website fingerprints such as website fingerprint 2_Z to obtain the news type.

[0256] The following example illustrates the website matching method provided in this application embodiment, using a website deduplication scenario as an example. Please refer to [link / reference]. Figure 7A and Figure 7B .

[0257] S701 receives a website storage command triggered for a target URL.

[0258] The website storage instruction is used to instruct the website information of the target website corresponding to the target URL to be stored in the website database, such as a URL that contains the main domain name "xxx.com".

[0259] S702, query the URL access records from the specified application to obtain multiple sub-URLs that belong to the same main URL as the target URL.

[0260] The sub-URL contains the main domain "xxx.com" and the historical access path. The historical access path contains at least one of the following: resource path information and resource query parameters. For example, taking a URL as an example, the historical access path may include at least one of PATH and QUERY. Resource query parameters in key-value pair form contain two parts: a key and a value. For example, QUERY contains a key and a value.

[0261] For example, the PATH for http: / / www.xxx.com / topic / 245836.html is / topic / 245836.html, and this URL does not have a QUERY; another example is http: / / www.xxx.com / ?jump_from=1_05_37_01, which does not have a PATH, and the QUERY is jump_from=1_05_37_01, where jump_from is the key and 1_05_37_01 is the value.

[0262] For multiple sub-URLs, execute S703 to S706 respectively:

[0263] S703 replaces the numbers in each path character contained in the historical access path extracted from the sub-URL with preset characters to obtain the access cleaning path.

[0264] For example, replacing every number that appears in the historical access path with 1, such as replacing all six numbers in 245836.html with 1, results in 111111.html.

[0265] S704, when it is determined that the access cleaning path contains resource path information that represents the web page resource address, the resource path information is divided based on the preset resource path separator to obtain a URL feature set, which contains at least one URL feature.

[0266] For example, when the access cleaning path contains PATH, the information of PATH can be divided using " / ", obtaining at least one URL feature. Taking the sub-URL http: / / www.xxx.com / topic / 245836.html as an example, the access cleaning path is topic / 111111.html, where PATH is topic / 111111.html. Using " / " to divide PATH, the URL features topic and 111111.html are obtained.

[0267] S705, based on the corresponding position of each URL feature in the corresponding sub-URL in the URL feature set, determine the directory hierarchy of each URL feature.

[0268] Taking the example that a smaller directory hierarchy represents a more detailed path within a sub-URL, and a larger directory hierarchy represents a broader path, a query can use only the key; the value can be selected for application or discarded based on actual needs. If a query in the same URL contains multiple keys, these keys can be assigned the same directory hierarchy, regardless of their directory level. For example, in the sub-URL http: / / www.xxx.com / ?entryScene=1&jump_from=2, the query contains two keys, entryScene and jump_from. Therefore, entryScene and jump_from can be assigned the same directory hierarchy.

[0269] S706, Based on the obtained directory hierarchy, the URL features contained in the URL feature set are concatenated to obtain the corresponding hierarchical URL features.

[0270] S7061, for each URL feature, perform the following respectively: when there is a URL feature corresponding to the parent directory level of the directory hierarchy that is smaller than the current URL feature, concatenate the parent directory level with its corresponding URL feature; and after concatenating the current URL feature with its corresponding directory level, continue to concatenate the two concatenation results in ascending order of directory hierarchy to obtain hierarchical URL features.

[0271] If there is no corresponding URL feature for a parent directory hierarchy with a lower value than the current URL feature, the current URL feature is concatenated with its corresponding directory hierarchy to obtain the hierarchical URL feature.

[0272] S7062, for each URL feature, perform the following: concatenate the URL feature with its corresponding directory hierarchy to obtain hierarchical URL features.

[0273] S707, based on the access counts associated with each of the multiple sub-URLs, statistically analyzes the feature frequencies of each hierarchical URL.

[0274] S708. Based on the obtained frequency of each feature, sort the hierarchical URL features and select the top N target URL features. For example, select the top 100 hierarchical URL features with the highest feature frequency as the target URL features.

[0275] For each target URL feature obtained, execute steps S709 to S710 respectively:

[0276] S709 uses a custom hash function to map the string in the target URL feature into a fixed-length binary sequence to obtain the corresponding encoded features.

[0277] S710 converts 0 in the coding features to the corresponding feature frequencies and 1 in the coding features to the inverse of the feature frequencies to obtain the corresponding weighted coding features.

[0278] S711 sums up the obtained weighted coding features to obtain the comprehensive features.

[0279] S712, based on a preset binary mapping strategy, maps the comprehensive features into a binary sequence to obtain the website fingerprint of the target website. For example, the binary mapping strategy maps values ​​greater than 0 to 1 and other values ​​to 0.

[0280] S713, determine the fingerprint similarity between the website fingerprint contained in each website information stored in the website database and the website fingerprint of the target website.

[0281] S714 If there is already stored website information where the fingerprint similarity between the included website fingerprint and the target website fingerprint is greater than a preset similarity threshold, a duplicate storage prompt is returned. The duplicate storage prompt indicates that the target website information has already been stored in the website database and does not need to be stored again.

[0282] S715, if there is no stored website information for which the fingerprint similarity between the included website fingerprint and the target website fingerprint is greater than a preset similarity threshold, the target website information is stored in the website database.

[0283] In this embodiment, the high dependence of DOM structure fingerprints or resource request record fingerprints on webpage content crawling can be effectively addressed; it can also effectively counter the countermeasures of black market activities that cause DOM structure fingerprints and resource request record fingerprints to become invalid, providing core data for subsequent detection of black and gray market website spread, and improving the detection range and detection quality; by determining fingerprint similarity, it can also provide basic data guarantee for various recommendation algorithms in website recommendation tasks.

[0284] Based on the same inventive concept, embodiments of this application provide a website matching device capable of realizing the functions corresponding to the aforementioned website matching method. Please refer to... Figure 8 The device includes a processing module 801 and an acquisition module 802, wherein:

[0285] Processing module 801: used to extract features from multiple sub-websites belonging to a main website, and obtain the website feature set corresponding to each of the multiple sub-websites;

[0286] The processing module 801 is also used to: perform fusion encoding on the feature sets of each URL to obtain the website fingerprint of the first website, the first website including the main URL and the web pages corresponding to each of the multiple sub-URLs;

[0287] The processing module 801 is also used to generate a website matching result based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of the reference websites of the pre-stored multiple reference websites.

[0288] In one possible embodiment, the apparatus further includes an acquisition module 802, which is configured to:

[0289] Receive website matching requests for the first website;

[0290] From the access database, query multiple sub-websites belonging to a main website. The access database is used to store the websites accessed at historical times.

[0291] Processing module 801 is specifically used for:

[0292] Feature extraction is performed on multiple sub-URLs to obtain the URL feature sets corresponding to each sub-URL.

[0293] In one possible embodiment, the processing module 801 is specifically used for:

[0294] For multiple sub-URLs, perform the following operations respectively:

[0295] Extract the access path from a sub-URL;

[0296] When the access path contains resource path information that represents the address of a web page resource, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

[0297] Based on at least one resource sub-information, establish a set of URL features corresponding to the sub-URL.

[0298] In one possible embodiment, the processing module 801 is specifically used for:

[0299] Replace the path characters in the access path that match the character format of the preset characters with the preset characters to obtain the access cleaning path;

[0300] Based on a preset URL format, at least one path information is extracted from the access cleaning path. The path information represents: webpage resource address or resource query parameters.

[0301] When at least one path information contains resource path information representing the address of a webpage resource, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

[0302] In one possible embodiment, the processing module 801 is specifically used for:

[0303] When the access path contains query path information that represents resource query parameters, the query path information and at least one resource sub-information are each used as URL features to obtain a set of URL features corresponding to a sub-URL.

[0304] In one possible embodiment, the processing module 801 is specifically used for:

[0305] For each set of URL features, perform the following operations:

[0306] Based on the corresponding position of each URL feature in the corresponding sub-URL in a set of URL features, the directory hierarchy of each URL feature is determined.

[0307] Based on the directory hierarchy, the URL features of each URL are concatenated to obtain the URL features of each hierarchy.

[0308] Each hierarchical URL feature is encoded separately to obtain each encoded feature, and the encoded features are then merged to obtain the website fingerprint of the first website.

[0309] In one possible embodiment, the processing module 801 is specifically used for:

[0310] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features for each hierarchy.

[0311] In one possible embodiment, the processing module 801 is specifically used for:

[0312] Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features of each directory.

[0313] By concatenating every two adjacent directory URL features in each directory hierarchy, we obtain the URL features for each hierarchy.

[0314] In one possible embodiment, multiple sub-URLs are each associated with a number of visits over a historical period; then the processing module 801 is specifically used for:

[0315] Based on the number of visits associated with each of the multiple sub-URLs, the frequency of features of each hierarchical URL is statistically analyzed.

[0316] Determine the hash value of each hierarchical URL feature to obtain each encoded feature;

[0317] Based on the frequency of each feature, the website fingerprint of the first website is obtained by fusing the various coding features.

[0318] In one possible embodiment, the processing module 801 is specifically used for:

[0319] Based on the frequency of each feature, the weighted sum of each encoded feature is performed to obtain the comprehensive feature;

[0320] Based on a preset binary mapping strategy, the comprehensive features are mapped into a binary sequence to obtain the website fingerprint of the first website.

[0321] In one possible embodiment, the processing module 801 is specifically used for:

[0322] Retrieve the reference website fingerprints and website anomaly types for each of the multiple reference websites from the pre-stored fingerprint blacklist;

[0323] Determine the website fingerprint of the first website and the Hamming distance between it and the respective reference website fingerprints of multiple reference websites to obtain the fingerprint similarity of each of the multiple reference websites.

[0324] When there is at least one reference website among multiple reference websites whose fingerprint similarity is greater than a preset similarity threshold, the website anomaly type of the first website is determined based on the website anomaly type of each of the at least one reference website, and the website matching result is obtained.

[0325] In one possible embodiment, the processing module 801 is specifically used for:

[0326] The fingerprint of the first website is determined, as well as the fingerprint similarity between the first website and the reference website fingerprints of each of the multiple reference websites obtained from the pre-stored set of fingerprints to be matched, and the fingerprint similarity between any two reference website fingerprints.

[0327] Based on the fingerprint similarity, the first website and multiple reference websites are clustered to obtain at least one website cluster. Each website cluster contains at least one website to be classified, which is either the first website or a reference website. The fingerprint similarity of the websites to be classified in the same website cluster is greater than the fingerprint similarity of the websites to be classified in different website clusters.

[0328] Based on at least one website cluster, the website categories of the first website and multiple reference websites are identified respectively to obtain website matching results.

[0329] Please refer to Figure 9 This is a computer device 900 provided in the embodiments of this application. The computer device 900 can, for example, be... Figure 1C The client 102 or server 101 in the system. The current and historical versions of the data storage program and the corresponding application software can be installed on a computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, which includes a display panel 941 for displaying a user-interactive interface, etc.

[0330] In one possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0331] The processor 980 is used to read a computer program and then execute the methods defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this application.

[0332] The memory 920 generally includes main memory and secondary storage. Main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. Secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 920 is used to store computer programs and other data. The computer programs include applications corresponding to each client, and other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, the computer program is stored in the memory 920, and the processor 980 executes the computer program in the memory 920 to implement any of the methods described in the preceding figures.

[0333] The aforementioned display unit 940 is used to receive input digital information, character information, or contact touch operations / non-contact gestures, and to generate signal inputs related to user settings and function control of the computer device 900. Specifically, in this embodiment, the display unit 940 may include a display panel 941. The display panel 941, for example, is a touch screen, which can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0334] In one possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the player's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980. It can also receive and execute commands from the processor 980.

[0335] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may also include an input unit 930. The input unit 930 may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0336] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, and a pressure sensor. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, for example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0337] As one embodiment, the number of processors 980 can be one or more, and the processors 980 and the memory 920 can be coupled together or relatively independent.

[0338] As one example, Figure 9 The processor 980 in the middle can be used to implement, for example Figure 8 The functions of the processing module 801 and the acquisition module 802.

[0339] As one example, Figure 9 The processor 980 in the text can be used to implement the functions of the server or terminal devices discussed above.

[0340] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the computer program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0341] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of software products, for example, through computer program products. These computer program products are stored in a storage medium and include computer programs used to cause a computer device to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0342] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A website matching method, characterized in that, include: Feature extraction is performed on multiple sub-websites belonging to a main website to obtain the website feature set corresponding to each of the multiple sub-websites; The feature sets of each URL are fused and encoded to obtain the website fingerprint of the first website, which includes the main URL and the web pages corresponding to each of the multiple sub-URLs; Based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of the reference websites of multiple pre-stored reference websites, a website matching result is generated.

2. The method according to claim 1, characterized in that, The step of extracting features from multiple sub-websites belonging to a main website to obtain a website feature set corresponding to each of the multiple sub-websites includes: Receive a website matching request for the first website; From the access database, query the multiple sub-websites belonging to the main website, the access database being used to store each website accessed in historical time; Feature extraction is performed on each of the multiple sub-URLs to obtain the URL feature set corresponding to each of the multiple sub-URLs.

3. The method according to claim 1, characterized in that, The step of extracting features from multiple sub-websites belonging to a main website to obtain a website feature set corresponding to each of the multiple sub-websites includes: For each of the multiple sub-URLs, perform the following operations: Extract the access path from a sub-URL; When it is determined that the access path contains resource path information representing the address of a web page resource, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information. Based on the at least one resource sub-information, establish a set of URL features corresponding to the sub-URL.

4. The method according to claim 3, characterized in that, When determining that the access path contains resource path information representing webpage resource addresses, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information, including: Replace the path characters in the access path that match the character format of the preset character with the preset character to obtain the access cleaning path; Based on a preset URL format, at least one path information is extracted from the access cleaning path, wherein the path information represents: the webpage resource address or resource query parameters; When the at least one path information includes resource path information representing the webpage resource address, the resource path information is divided based on a preset resource path separator to obtain at least one resource sub-information.

5. The method according to claim 3, characterized in that, The step of establishing a URL feature set corresponding to a sub-URL based on the at least one resource sub-information includes: When the access path includes query path information representing resource query parameters, the query path information and the at least one resource sub-information are each used as URL features to obtain the URL feature set corresponding to the sub-URL.

6. The method according to any one of claims 1 to 5, characterized in that, The step of fusing and encoding the feature sets of each URL to obtain the website fingerprint of the first website includes: For each set of URL features, perform the following operations: Based on the corresponding position of each URL feature in the corresponding sub-URL in a set of URL features, the directory hierarchy of each URL feature is determined; Based on each directory hierarchy, the URL features are concatenated to obtain the URL features of each hierarchy. Each hierarchical URL feature is encoded separately to obtain each encoded feature, and the encoded features are then fused to obtain the website fingerprint of the first website.

7. The method according to claim 6, characterized in that, The process involves concatenating the URL features based on each directory hierarchy to obtain the URL features for each hierarchy, including: Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features of each hierarchy.

8. The method according to claim 6, characterized in that, The process involves concatenating the URL features based on each directory hierarchy to obtain the URL features for each hierarchy, including: Each URL feature is concatenated with its corresponding directory hierarchy to obtain the URL features of each directory. By concatenating every two adjacent directory URL features in each directory hierarchy, we obtain the URL features for each hierarchy.

9. The method according to claim 6, characterized in that, Each of the multiple sub-URLs is associated with a number of visits over a historical period; The step of encoding each hierarchical URL feature to obtain each encoded feature, and then fusing the encoded features to obtain the website fingerprint of the first website, includes: Based on the number of visits associated with each of the multiple sub-URLs, the frequency of each feature of each hierarchical URL is statistically analyzed; The hash value of each hierarchical URL feature is determined to obtain each encoding feature; Based on the frequency of each feature, the website fingerprint of the first website is obtained by fusing the encoded features.

10. The method according to claim 9, characterized in that, The step of fusing the encoded features based on the frequency of each feature to obtain the website fingerprint of the first website includes: Based on the frequency of each feature, the weighted sum of each encoded feature is performed to obtain the comprehensive feature; Based on a preset binary mapping strategy, the comprehensive features are mapped into a binary sequence to obtain the website fingerprint of the first website.

11. The method according to any one of claims 1 to 5, characterized in that, The process of generating website matching results based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of the reference websites of multiple pre-stored reference websites includes: Retrieve the reference website fingerprints and website anomaly types for each of the multiple reference websites from the pre-stored fingerprint blacklist; The Hamming distance between the website fingerprint of the first website and the reference website fingerprint of each of the plurality of reference websites is determined, and the fingerprint similarity of each of the plurality of reference websites is obtained. When at least one of the multiple reference websites has a fingerprint similarity greater than a preset similarity threshold, the website anomaly type of the first website is determined based on the website anomaly type of each of the at least one reference website, and a website matching result is obtained.

12. The method according to any one of claims 1 to 5, characterized in that, The process of generating website matching results based on the fingerprint similarity between the website fingerprint of the first website and the fingerprints of the reference websites of multiple pre-stored reference websites includes: The fingerprint of the first website is determined, as well as the fingerprint similarity between the first website and the reference website fingerprints of each of the multiple reference websites obtained from the pre-stored set of fingerprints to be matched, and the fingerprint similarity between any two of the reference website fingerprints is determined. Based on the fingerprint similarity, the first website and the plurality of reference websites are clustered to obtain at least one website cluster. The website cluster contains at least one website to be classified, which is either the first website or the reference website. The fingerprint similarity of the websites to be classified in the same website cluster is greater than the fingerprint similarity of the websites to be classified in different website clusters. Based on the at least one website cluster, the website categories of the first website and the plurality of reference websites are identified respectively to obtain website matching results.

13. A website matching device, characterized in that, include: Processing module: used to extract features from multiple sub-websites belonging to a main website, and obtain the website feature set corresponding to each of the multiple sub-websites; The processing module is further configured to: perform fusion encoding on each URL feature set to obtain the website fingerprint of the first website, wherein the first website includes the main URL and the web pages corresponding to each of the multiple sub-URLs; The processing module is further configured to: generate a website matching result based on the fingerprint similarity between the website fingerprint of the first website and the reference website fingerprints of each of the pre-stored multiple reference websites.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 12.

15. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 12 according to the obtained program instructions.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 12.

Images