Website type determination method, website type prediction model training method and device

Abstract

The embodiment of the application discloses a website type determination method, a website type prediction model training method and device, and relates to the fields of artificial intelligence, cloud computing and the like. The method comprises the following steps: obtaining a to-be-detected website; inputting the to-be-detected website into a website type prediction model to obtain a website type prediction result of the to-be-detected website, and determining the website type of the to-be-detected website based on the website type prediction result; wherein the website type prediction model determines the website type prediction result based on the following manner: determining an associated website having an association relationship with the to-be-detected website, and determining initial features of the to-be-detected website and the associated website; determining target features of the to-be-detected website based on the initial features of each website and the association relationship between the websites; and determining the website type prediction result of the to-be-detected website based on the target features of the to-be-detected website. By adopting the embodiment of the application, the accuracy of determining the website type can be improved, and the applicability is high.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method for determining website type, a method for training a website type prediction model, and an apparatus. Background Technology

[0002] With the continuous development of internet technology, people frequently access different websites to obtain information, work, or entertainment. Therefore, it is of paramount importance to effectively and accurately determine the type of website corresponding to a URL, such as how to identify malicious URLs containing sensitive information.

[0003] Existing methods for website type determination primarily involve extracting text and image information from a website, classifying this information, or, when malicious websites often use similar templates, using clustering techniques based on a certain number of website templates to determine the website type. However, current technologies often only focus on detecting and judging the website itself corresponding to the URL, resulting in poor accuracy and low applicability in determining website type. Summary of the Invention

[0004] This application provides a method for determining website types, a method for training a website type prediction model, and related apparatus, which can improve the accuracy of determining website types and has high applicability.

[0005] On one hand, embodiments of this application provide a method for determining website type, the method comprising:

[0006] Input the above-mentioned URL to be detected into the website type prediction model to obtain the website type prediction result of the above-mentioned URL to be detected, and determine the website type of the above-mentioned URL to be detected based on the above-mentioned website type prediction result.

[0007] The website type prediction model determines the prediction results based on the following methods:

[0008] Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs;

[0009] Based on the initial features of each URL and the relationships between the URLs, the target features of the URLs to be detected are determined.

[0010] Based on the target characteristics of the URLs to be detected, the website type prediction results for the URLs to be detected are determined.

[0011] On the other hand, embodiments of this application provide a method for training a website type prediction model, the method comprising:

[0012] Obtain a training sample set. Each sample URL in the training sample set is labeled with a sample tag, which represents the real website type of the sample URL.

[0013] Input each of the above sample URLs into the initial model to obtain the website type prediction results for each of the above sample URLs. Based on the above website type prediction results, determine the predicted website type for the corresponding sample URLs.

[0014] Based on the sample tags and predicted website types of each of the above sample URLs, the training loss value is determined. The initial model is iteratively trained according to the training loss value and the training sample set until the training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model.

[0015] The initial model above determines the website type prediction result for each of the above sample URLs based on the following method:

[0016] Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the aforementioned associated URLs;

[0017] Based on the initial characteristics of the sample URL and each of the aforementioned associated sample URLs, as well as the association between the sample URL and each of the aforementioned associated sample URLs, the sample target characteristics of the sample URL are determined.

[0018] Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

[0019] On the other hand, embodiments of this application provide a website type determination device, the device comprising:

[0020] The URL acquisition module is used to obtain the URL to be detected.

[0021] The website type determination module is used to input the above-mentioned URL to be detected into the website type prediction model, obtain the website type prediction result of the above-mentioned URL to be detected, and determine the website type of the above-mentioned URL to be detected based on the above-mentioned website type prediction result.

[0022] Specifically, the website type prediction model mentioned above is used for:

[0023] Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs;

[0024] Based on the initial features of each URL and the relationships between the URLs, the target features of the URLs to be detected are determined.

[0025] Based on the target characteristics of the URLs to be detected, the website type prediction results for the URLs to be detected are determined.

[0026] On the other hand, embodiments of this application provide a training apparatus for a website type prediction model, the apparatus comprising:

[0027] The sample acquisition module is used to acquire a training sample set. Each sample URL in the training sample set is labeled with a sample tag, which represents the real website type of the sample URL.

[0028] The training module is used to input each of the above sample URLs into the initial model, obtain the website type prediction results for each of the above sample URLs, and determine the predicted website type of the corresponding sample URL based on the above website type prediction results;

[0029] The training evaluation module is used to determine the training loss value based on the sample tags and predicted website types of each of the above sample URLs, and to iteratively train the above initial model according to the above training loss value and the above training sample set until the above training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model.

[0030] Specifically, when determining the website type prediction result for each of the above sample URLs, the initial model is used for:

[0031] Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the aforementioned associated URLs;

[0032] Based on the initial characteristics of the sample URL and each of the aforementioned associated sample URLs, as well as the association between the sample URL and each of the aforementioned associated sample URLs, the sample target characteristics of the sample URL are determined.

[0033] Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

[0034] On the other hand, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected;

[0035] The aforementioned memory is used to store computer programs;

[0036] The processor described above is configured to execute the website type determination method or the website type prediction model training method provided in the embodiments of this application when the computer program described above is invoked.

[0037] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the website type determination method or the website type prediction model training method provided in embodiments of this application.

[0038] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the website type determination method or the website type prediction model training method provided in embodiments of this application.

[0039] In this embodiment, the target features of the URL to be detected are determined by the relationship between the URL to be detected and associated URLs, as well as the initial features of each URL. This approach considers not only the features of the URL to be detected itself, but also the feature correlation between URLs. As a result, the target features of the URL to be detected can characterize the common features of the URL to be detected and associated URLs. This improves the accuracy of determining the website type based on the target features and allows for the determination of the website type within a wide range of URLs, making it highly applicable. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the website type determination method provided in an embodiment of this application;

[0042] Figure 2 This is a flowchart illustrating the process of determining the prediction result of website type provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the URL relationship diagram provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of a scenario for determining the initial features of a URL according to an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of a scenario for determining URL features provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of a scenario for determining attribute features provided in an embodiment of this application;

[0047] Figure 7a This is a schematic diagram of a scenario for determining webpage features provided in an embodiment of this application;

[0048] Figure 7b This is another scenario illustration of determining webpage features provided in the embodiments of this application;

[0049] Figure 7c This is another schematic diagram illustrating a scenario for determining webpage features provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of the URL relationship diagram provided in an embodiment of this application;

[0051] Figure 9 This is a flowchart illustrating the method for determining the first feature provided in an embodiment of this application;

[0052] Figure 10 This is a schematic diagram of a scenario for determining target features provided in an embodiment of this application;

[0053] Figure 11 This is a schematic diagram of a scenario for determining the prediction result of website type provided in an embodiment of this application;

[0054] Figure 12 This is a flowchart illustrating a training method for a website type prediction model provided in an embodiment of this application;

[0055] Figure 13 This is a schematic diagram of the website type determination device provided in the embodiments of this application;

[0056] Figure 14 This is a schematic diagram of the structure of the training device for the website type prediction model provided in the embodiments of this application;

[0057] Figure 15 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] The website type determination method provided in this application embodiment can determine the website type of any website corresponding to a website to be detected, such as an educational website, a sports website, or a malicious website containing sensitive information, and then determine whether the website to be detected is an educational website, a sports website, or a malicious website based on the website type.

[0060] The specific granularity and dimensions of the above-mentioned website type classification can be determined based on the actual application scenario requirements. For example, website types can be classified as corporate websites, public welfare websites, and gaming websites. Alternatively, website types can be classified as malicious websites containing sensitive information and normal websites that do not contain sensitive information. No restrictions are imposed here.

[0061] The website type determination method and website type prediction model training method provided in this application embodiment are applicable to fields such as computer science and artificial intelligence (AI).

[0062] Artificial intelligence (AI) encompasses the theories, methods, technologies, and application systems that utilize digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. The embodiments of this application can train a website type prediction model based on machine learning, natural language processing, and other AI technologies, and then use this trained model to predict the website type corresponding to the URL to be detected.

[0063] The website type determination method provided in this application can be executed based on a server or a terminal. The server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these.

[0064] See Figure 1 , Figure 1 This is a flowchart illustrating the website type determination method provided in this application embodiment. The website type determination method provided in this application embodiment may include the following steps:

[0065] Step S11: Obtain the URL to be tested.

[0066] In some feasible implementations, the URL to be detected can be a URL that the user visits in real time. That is, when the user visits a certain URL, the URL visited by the user is obtained and used as the URL to be detected in order to determine the website type of the URL to be detected, and then the user is reminded of the website type of the URL he visited in real time.

[0067] Optionally, the URL to be detected can be a URL outside the URL whitelist in the network management system. For each URL outside the whitelist, it can be used as the URL to be detected and the website type of the URL can be determined based on the website type determination method provided in the embodiments of this application, so as to determine whether to put the URL into the URL whitelist according to the determined website type.

[0068] It should be noted that the above method of obtaining the URL to be detected is only an example. Any URL on the Internet can be used as the URL to be detected and its website type can be determined based on the website type determination method provided in the embodiments of this application. No restrictions are imposed here.

[0069] Step S12: Input the URL to be detected into the website type prediction model to obtain the website type prediction result of the URL to be detected, and determine the website type of the URL to be detected based on the website type prediction result.

[0070] In some feasible implementations, after obtaining the URL to be detected, the URL can be input into the website type prediction model, and then the website type prediction result of the URL to be detected can be obtained based on the output of the website type prediction model.

[0071] The website type prediction model mentioned above may include multiple task modules for performing different data processing tasks in the process of determining the website type prediction results.

[0072] For details on how the website type prediction model determines the website type prediction result for the URL to be detected, please refer to [link to website type prediction model]. Figure 2 . Figure 2 This is a flowchart illustrating the process of determining the website type prediction result provided in this application embodiment. The specific steps for determining the website type prediction result provided in this application embodiment may include the following:

[0073] Step S21: Determine the associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs.

[0074] In some feasible implementations, associated URLs that are related to the URL to be detected include, but are not limited to, URLs with one-way or two-way connections to the URL to be detected. A one-way connection means that one can jump from the URL to be detected to the associated URL, but cannot jump from the associated URL to the URL to be detected; or, one can jump from the associated URL to the URL to be detected, but cannot jump from the URL to the associated URL. A two-way connection means that one can jump from the URL to the associated URL, and can also jump from the associated URL to the URL to be detected.

[0075] Optionally, associated URLs also include URLs that are indirectly linked to the URL to be detected through at least one URL. For example, if URL A has a one-way or two-way connection with the URL to be detected, and URL B has a one-way or two-way connection with the URL to be detected, then URL B can be identified as an associated URL to the URL to be detected.

[0076] For example, if URL A has a one-way or two-way connection with the URL to be detected, URL B has a one-way or two-way connection with URL A, and URL C has a one-way or two-way connection with URL B, then URL C can also be identified as an associated URL that is associated with the URL to be detected.

[0077] The specific number of URLs between the URL to be detected and any associated URL can be determined based on the actual application scenario requirements, and is not limited here.

[0078] In some feasible implementations, when determining associated URLs that are related to the URL to be detected, URL sampling can also be used.

[0079] Specifically, the sampling depth and corresponding sampling quantity are determined, and the URL to be detected is selected as the initial sampling URL. Then, based on the aforementioned sampling depth and corresponding sampling quantity, URL sampling is performed to obtain associated URLs that are related to the URL to be detected. The specific sampling depth and corresponding sampling quantity can be determined based on the actual application scenario requirements and are not limited here.

[0080] When sampling URLs based on the aforementioned sampling depth and corresponding sampling quantity, the one-way and two-way connection relationships between URLs on the World Wide Web or in the URL database can be determined first, and then URL sampling can be performed based on the connection relationships between URLs on the World Wide Web or in the URL database.

[0081] The World Wide Web is a typical example of heterogeneous graph data. It contains not only URL information, but also a lot of information such as Internet Protocol (IP) addresses and registration details. In addition, the URLs also contain information about the one-way and two-way connections between them.

[0082] Based on this, the various URLs appearing on the World Wide Web can be sorted out and a URL relationship graph can be constructed, such as a heterogeneous graph. In this graph, each node corresponds to a URL, and the edges between nodes represent the connections between corresponding URLs. As shown in Table 1, based on the URLs in the URL database listed in Table 1 and the connections between them, a graph can be constructed as follows: Figure 3 The URL relationship diagram shown.

[0083] Table 1

[0084]

[0085] In this context, Uniform Resource Locator (URL), SITE, DOMAIN, IP, and EMAIL represent different URL types and are also used to indicate different ranges of references. URL represents a complete website address, such as www.AAAAA.com / index.html; SITE indicates a URL pointing to a website site; DOMAIN is a domain name address, such as www.AAAAA.com; IP represents an IP address; and EMAIL represents an email address. A website site is a storage area that stores all the files contained within a website. Simply put, a website site is a folder containing all the content of a website.

[0086] Based on the sampling method, it can accurately identify the associated URLs that are strongly related to the URL to be detected, while also saving time in determining the associated URLs of the URL to be detected, and has high applicability.

[0087] See Figure 3 , Figure 3 This is a schematic diagram of the URL relationship diagram provided in an embodiment of this application. For example... Figure 3 As shown, if URL1 and URL2 have a one-way connection and can navigate from URL1 to URL2, then URL1 and URL2 can be connected with a dashed line in the URL relationship diagram. If URL6 unilaterally references URL5, meaning URL6 and URL5 have a one-way connection, then URL6 and URL5 can be connected with a dashed line in the URL relationship diagram. For URLs with a two-way connection, a solid line can be used for connection.

[0088] Furthermore, based on the sampling depth and the corresponding number of samples, the obtained sampling URL corresponding to the sampling depth can be determined, and the sampling URL can be identified as an associated URL that is related to the URL to be detected.

[0089] If the URL to be detected is Figure 3 If URL1 is used as the initial URL, and the sampling depth is 1, the corresponding number of samples is 2; if the sampling depth is 2, the number of samples is also 2. The sampled URLs obtained by using URL1 as the initial URL for sampling are shown in Table 2.

[0090] Table 2

[0091]

[0092] It should be noted that when sampling URLs, sampling can be performed based on the random breadth-first sampling algorithm or other sampling algorithms. The specific method can be determined based on the actual application scenario requirements, and there are no restrictions here.

[0093] Furthermore, when identifying associated URLs that are related to the URL to be detected, if there are many associated URLs, only a certain number of associated URLs need to be identified. The specific number can be determined based on the actual application scenario requirements, and there are no restrictions here.

[0094] Optionally, when determining associated URLs related to the URL to be detected, the URL relationships between URLs on the World Wide Web or in a URL database can be used. For example, the URL relationship between two URLs can be used to determine whether the two URLs have a redirect relationship, thereby identifying associated URLs of the URL to be detected. Alternatively, the similarity between URLs on the World Wide Web or in a URL database can be determined, and the association between URLs can be determined based on the similarity of each URL, thereby identifying associated URLs related to the URL to be detected.

[0095] It should be noted that the above implementation method for determining the associated URLs that are related to the URL to be detected is only an example. The specific implementation method can be determined based on the actual application scenario requirements, and no restrictions are imposed here.

[0096] In some feasible implementations, after determining the associated URLs of the URL to be detected, the URL characteristics of each URL in the URL to be detected and the associated URLs can be determined. Specifically, for each URL, at least one of the URL characteristics, attribute characteristics, or webpage characteristics corresponding to that URL can be determined, and then the initial characteristics of that URL can be determined based on at least one of the determined URL characteristics, attribute characteristics, or webpage characteristics.

[0097] As an example, for a URL to be detected, the URL features, attribute features, and webpage features of the URL to be detected can be determined, and then the initial features of the URL to be detected can be determined based on the determined URL features, attribute features, and webpage features.

[0098] Specifically, for any URL among the URL to be detected and associated URLs, if only one of the URL features, attribute features, or URL features is determined, the determined feature can be used as the initial feature of the URL. If at least two of the URL features, attribute features, or URL features are determined, the two determined features can be further fused to obtain the URL features of the URL.

[0099] See Figure 4 , Figure 4 This is a schematic diagram of a scenario for determining the initial features of a URL according to an embodiment of this application. For example... Figure 4As shown, the URL to be detected is WWW.AAABBB.COM. Based on this URL, the corresponding URL features, attribute features, and webpage features can be determined. Then, the URL features, attribute features, and webpage features are fused to obtain the initial features of the URL to be detected.

[0100] Feature fusion of multiple features includes, but is not limited to, vertical feature concatenation and horizontal feature concatenation. It can also be implemented based on a feature fusion model. The specific method can be determined based on actual application requirements and is not limited here.

[0101] Specifically, for each URL in the URL to be detected and associated URLs, when determining the URL features of the URL, the word vectors of each word in the URL can be determined, and the URL features of the URL can be determined based on the word vectors of each word in the URL.

[0102] As an example, for each URL in the target URL and associated URLs, the word vectors of each word in the URL are determined. These word vectors are then input into a neural network model, which converts them into URL features represented by a single vector. Alternatively, for each URL in the target URL and associated URLs, the URL can be directly input into the neural network model. The model determines the word vectors of each word in the URL and converts them into URL features represented by a single vector.

[0103] See Figure 5 , Figure 5 This is a schematic diagram of a scenario for determining URL features provided in an embodiment of this application. For example... Figure 5 As shown, the URL to be detected is WWW.AAABBB.COM. When determining the URL features of the URL to be detected, the URL can be input into a neural network model. Based on the neural network model, the word vectors of each word in the URL to be detected can be determined, such as determining the word vector corresponding to each character in the URL to be detected. Furthermore, the word vectors of each word in the URL to be detected are further processed to obtain the URL features of the URL to be detected.

[0104] The aforementioned neural network can be a recurrent neural network (RNN), or other neural networks, or a URL feature conversion model built based on multiple neural networks. The specific choice can be determined based on the actual application scenario requirements, and no restrictions are imposed here.

[0105] Specifically, for each URL in the URL to be detected and the associated URLs, when determining the attribute characteristics of the URL, the attribute information of the URL can be determined, and then the attribute characteristics corresponding to the URL can be determined based on the attribute information of the URL.

[0106] For any given URL, the attribute information includes, but is not limited to, the number of characters in the URL, whether it contains preset characters, whether the corresponding domain name is an IP domain name, whether the corresponding site contains random strings, the domain name registration time, the first appearance time of the domain name, the number of Chinese and / or English characters in the website corresponding to the URL, the URL's access popularity, whether there are sudden changes in popularity within a preset time period, whether the Internet Content Provider (ICP) filing information is eligible for filing, and whether it contains port information, etc., which are related to the URL or the website corresponding to the URL. The specific information can be determined based on actual application needs and is not restricted here.

[0107] Specifically, for each URL in the list of URLs to be detected and associated URLs, the attribute vectors corresponding to each attribute information of that URL can be determined. Then, based on the attribute vectors corresponding to each attribute information, the attribute features of that URL, represented by a single vector, can be determined. Alternatively, for each URL in the list of URLs to be detected and associated URLs, the attribute feature values ​​corresponding to each attribute information of that URL can be determined. Based on the attribute feature values ​​corresponding to each attribute information, the attribute features of that URL can be constructed. In other words, for any URL, the various attribute information of that URL can be statistically analyzed, and then the statistically analyzed attribute information can be converted into attribute features represented by vectors.

[0108] See Figure 6 , Figure 6 This is a schematic diagram of a scenario for determining attribute features provided in an embodiment of this application. For example... Figure 6 As shown, the URL to be detected is WWW.AAABBB.COM, and the number of characters in the URL to be detected, whether the URL to be detected contains preset characters, whether the URL to be detected is an IP domain name, and whether the site corresponding to the URL to be detected contains random strings can be determined. Then, the determined attribute information is converted into the attribute features corresponding to the URL to be detected.

[0109] In this case, determining the attribute characteristics of a URL based on its attribute information can also be achieved using a neural network model constructed from one or more neural networks. The specific method can be determined based on the actual application scenario requirements, and there are no restrictions here.

[0110] Specifically, for each URL in the URL to be detected and the associated URLs, when determining the webpage features of the URL, the webpage content corresponding to the URL can be determined, and at least one of the text features or image features of the webpage can be determined based on the webpage content corresponding to the URL. Then, based on the determined text features or image features, the webpage features corresponding to the URL can be determined.

[0111] Specifically, for each URL in the URL to be detected and the associated URLs, when determining the text features corresponding to the URL, the text information in the webpage corresponding to the URL can be determined, and the word vectors of each word in the text information can be determined. Based on the word vectors of each word in the text information, the text features of the webpage corresponding to the URL can be determined.

[0112] As an example, for each URL in the URL to be detected and the associated URLs, the word vectors of each word in the text information of the webpage corresponding to that URL are determined, and the text features of the webpage corresponding to that URL are further determined based on the word vectors of each word in the text information.

[0113] See Figure 7a , Figure 7a This is a schematic diagram of a scenario for determining webpage features provided in an embodiment of this application. For example... Figure 7a As shown, for the URL to be detected, WWW.AAABBB.COM, the text information in the webpage corresponding to the URL can be determined, and then the word vectors of each word in the text information of the webpage can be determined. For example, the word vectors of each word in the text information "Today and tomorrow, south of the Huai River..." in the webpage corresponding to the URL to be detected can be determined, and the text features corresponding to the text information can be determined based on the word vectors of each word. The determined text features are then used as the webpage features of the URL to be detected.

[0114] Specifically, for each URL in the URL to be detected and the associated URLs, when determining the image features corresponding to the URL, the image information in the webpage corresponding to the URL can be determined, and then the image information can be used to extract features to obtain the image features corresponding to the URL.

[0115] See Figure 7b , Figure 7b This is a schematic diagram illustrating another scenario for determining webpage features provided in an embodiment of this application. For example... Figure 7b As shown, for the URL to be detected, WWW.AAABBB.COM, the image information in the webpage corresponding to the URL can be determined, and then the image features of the image information in the webpage can be determined, and the determined image features can be used as the webpage features of the URL to be detected.

[0116] For each URL in the list of URLs to be detected and associated URLs, when determining the webpage features of that URL, a webpage image including the text information and / or image information of that webpage can be generated based on the webpage content of that URL. This can be done by determining a webpage snapshot of the webpage corresponding to that URL, taking a screenshot of the webpage corresponding to that URL, etc. Then, feature extraction is performed on the webpage image to obtain the image features of the webpage image, and the determined image features are identified as the webpage features of that URL.

[0117] In determining the image features of image information, specific methods such as graph neural networks or related models can be used, depending on the actual application scenario requirements, and no restrictions are imposed here.

[0118] Specifically, for the URL to be detected, the text and image information in the webpage corresponding to the URL can be determined, and the text features corresponding to the text information and the image features corresponding to the image information can be further determined. For example... Figure 7c As shown, Figure 7c This is another schematic diagram illustrating a scenario for determining webpage features provided in this application embodiment. After determining the text features corresponding to the text information and the image features corresponding to the image information, feature fusion can be performed on the text features and image features, and the fused features can be determined as the webpage features corresponding to the URL to be detected. Alternatively, the text features and image features can be horizontally or vertically concatenated to obtain the webpage features of the URL to be detected.

[0119] Step S22: Based on the initial features of each URL and the relationships between URLs, determine the target features of the URL to be detected.

[0120] In some feasible implementations, based on the relationship between the URLs to be detected and the associated URLs, a URL isomorphism graph or a URL heteromorphism graph is constructed. The constructed URL isomorphism graph or URL heteromorphism graph is determined as a URL relationship graph, and the target features of the URL to be detected are determined based on the URL relationship graph.

[0121] As an example, if the URL to be detected and each associated URL are of the same URL type, and the association relationships between each URL are the same, such as the URL to be detected and each associated URL being URLs, and the association relationship between any two related URLs being bidirectional, then based on the association relationships between the URL to be detected and each of the associated URLs, a URL isomorphism graph can be constructed and used as a URL relationship graph.

[0122] As an example, if the URL to be detected and its associated URLs contain multiple URL types, and the relationships between the URLs contain multiple relationships, such as URLs and email addresses, and one-way and two-way connections, then a URL heterogeneity graph can be constructed based on the relationships between the URL to be detected and its associated URLs, and this graph can be used as a URL relationship graph.

[0123] The URL relationship diagram includes a first URL and a second URL. The first URL is the URL that is associated with only one URL, excluding the URL to be detected. The second URL is the other URLs that are associated with the first URL.

[0124] In the URL relationship graph described above, the line connecting any two URLs can be used to represent the association between the two URLs. Each URL in the URL to be detected and the associated URLs corresponds to a node in the URL heterogeneity graph or the URL isogeneity graph, and the edges between nodes represent the association between the URLs, such as a unidirectional connection or a bidirectional connection.

[0125] For example, if the associated URLs obtained based on URL sampling are as shown in Table 2 above, then the association between the associated URLs and the URLs to be detected can be determined. Figure 8 The URL relationship diagram shown. Figure 8 This is a schematic diagram of the URL relationship diagram provided in the embodiments of this application. Figure 8 In this context, URL1 is the URL to be detected, URL3 is only associated with URL2, SITE2 is only associated with URL2, and DOMAIN1 is only associated with SITE1. Therefore, URL3, SITE2, and DOMAIN1 are determined as the first URL, and URL1, URL2, and SITE1 can be determined as the second URL.

[0126] Furthermore, for each second URL in the URL relationship graph, based on the initial characteristics of the second URL and the first characteristics of the first neighbor URLs of the second URL, the first characteristics of the second URL are determined. The first neighbor URLs of the second URL are the other neighbor URLs of the second URL excluding the parent URL of the second URL. If the first neighbor URLs of the second URL include the first URL, then the first characteristics of the first URL are the initial characteristics of the first URL.

[0127] Specifically, for each second URL, if the URL to be detected has a direct one-way or two-way connection with it, then the URL to be detected is the parent URL of that second URL. If the second URL establishes a connection with the URL to be detected through another URL, then that URL is the parent URL of that second URL.

[0128] Based on the above method, the first feature of each second URL in the URL relationship graph can be determined, and then the target feature of the URL to be detected can be determined from the first feature of each second URL. That is, the URL to be detected is determined from each second URL, and the first feature of the URL to be detected is determined as the target feature of the URL to be detected.

[0129] Optionally, after obtaining the first feature of the URL to be detected, the first feature can be subjected to a nonlinear transformation, and the transformed feature can be determined as the target feature of the URL to be detected.

[0130] Optionally, to further prevent feature fitting, the target features of the URL to be detected can be regularized, such as determining the norm of the target features, and the final target features used to predict the website type are obtained based on the norm of the target features. The specific method is as follows: To achieve, among which, For target features, express norm, The ultimate target feature.

[0131] See also Figure 8 For URL2, its first characteristic can be determined based on its initial characteristics and the first characteristic of its first neighboring URL (URL3) among its neighboring URLs (URL3 and URL1), excluding URL2's parent URL (URL1). Since URL3 is the first URL, for URL2, its first characteristic can be determined based on both its initial characteristics and those of URL3. Similarly, the first characteristic of SITE1 can be determined using the same method.

[0132] For URL1, the first feature of URL1 can be determined based on the initial feature of URL1 and the first feature of the first neighboring URLs (URL2 and SITE1), and then the first feature of URL1 can be determined as the target feature of URL1.

[0133] For any second URL in the URL relationship graph, the method for determining the first feature of the second URL based on its initial features and the first features of its first neighboring URLs can be found in [reference needed]. Figure 9 . Figure 9 This is a flowchart illustrating the method for determining the first feature provided in an embodiment of this application, as shown below. Figure 9 As shown, the method for determining the first feature provided in this application embodiment may specifically include the following steps:

[0134] Step S221: Based on the first feature of the first neighboring URL of the second URL, determine the neighbor aggregation feature corresponding to the second URL.

[0135] In some feasible implementations, for any second URL in the URL relationship graph, the URL type of the first neighboring URLs of the second URL can be determined, and the first features corresponding to the same URL type can be aggregated to obtain the first aggregated feature.

[0136] See also Figure 8For the second URL (URL2), its first neighboring URLs include two URLs of type URL3 and two SITEs of type SITE2. Therefore, we can aggregate the first features of the two URLs of type URL3 and the first features of the two SITEs of type SITE2. Since URL3 and SITE2 are the first nodes in the URL relationship graph, this essentially involves aggregating the initial features of the two URLs of type URL3 and the initial features of the two SITEs of type SITE2, resulting in the first aggregated features corresponding to the URL type and the first aggregated features corresponding to the SITE type.

[0137] Furthermore, the weight features corresponding to each URL type are determined. For each URL type, a second aggregate feature corresponding to that URL type can be determined based on the first aggregate feature and weight feature corresponding to that URL type. Essentially, determining the second aggregate feature corresponding to each URL type based on the first aggregate feature and weight feature means uniformly mapping the first aggregate features corresponding to different URL types to a second aggregate feature of the same dimension. Different URL types correspond to different weight features, and the weight features for the same URL type may also differ when determining the second aggregate feature for different second nodes. The specific weight feature corresponding to each URL type can be determined based on the actual application scenario requirements, and is not limited here.

[0138] After determining the second aggregation features corresponding to each URL type, the neighbor aggregation features corresponding to that second URL can be determined based on these second aggregation features. Specifically, the second aggregation features can be concatenated to obtain concatenated features, which are then aggregated to obtain the neighbor aggregation features corresponding to that second URL. For any given second URL, the neighbor aggregation features represent the features of all its first neighbor URLs.

[0139] Step S222: Based on the initial features of the second URL and the neighbor aggregation features, determine the first feature of the second URL.

[0140] In some feasible implementations, for any second URL in the URL relationship graph, after determining the neighbor aggregation characteristics of the second URL, the initial characteristics and neighbor aggregation characteristics of the second URL can be concatenated to obtain the first characteristic of the second URL. Based on this method, the first characteristics of each second URL in the URL relationship graph can be determined.

[0141] It should be noted that the specific methods for aggregating the spliced ​​features obtained based on the second aggregation feature, as well as aggregating each first feature (or aggregating each initial feature), include, but are not limited to, mean aggregation, maximum value aggregation, or minimum value aggregation. The specific method can be determined based on the actual application scenario requirements and is not limited here.

[0142] It should be noted that the specific methods for splicing the second aggregated features and splicing the initial features of the second URL with the corresponding neighbor aggregated features are not limited to vertical splicing, horizontal splicing, or feature fusion. The specific methods can be determined based on the actual application scenario requirements and are not limited here.

[0143] The following is combined with Figure 8 The URL1 is a URL relationship graph of the website to be detected, obtained by sampling at a depth of 2 layers (sampling depth 1 and sampling depth 2), with the number of samples being 1 and 2 respectively. Figure 10 The feature processing procedure shown determines the target features of the URL to be detected. Figure 10 This is a schematic diagram of a scenario for determining target features provided in an embodiment of this application, such as... Figure 10 As shown:

[0144] Step (1) performs maximum value aggregation on the initial features of the URLs corresponding to the same URL type in the first neighboring URLs of URL2 (two URL3s and two SITE2s). For example, the initial features of the two URL3s are aggregated. The first aggregated feature is obtained after performing maximum aggregation. Initial features of the two SITE2s The first aggregated feature is obtained after performing maximum aggregation. The initial characteristics of the first neighbor URL of SITE1 (two DOMAIN1s) The first aggregated feature is obtained after performing maximum aggregation. .in, This represents the aggregate function for the maximum value.

[0145] Step (2) multiplies the first aggregated feature obtained in step (1) by the weight feature corresponding to the same URL type to obtain the corresponding second aggregated feature. For example... , , This maps the first aggregated features of different dimensions to feature vectors of the same dimension, i.e., the second aggregated features. , , The dimensions are consistent.

[0146] in, , as well as This represents the weight features corresponding to different URL types.

[0147] Step (3) involves vertically concatenating the second aggregation features corresponding to URL2 and then performing maximum value aggregation to obtain the neighbor aggregation features used to represent the neighbor information of each first neighbor URL of URL2. That is... ( , ))= ,in, ( ) indicates vertical concatenation. Similarly, after vertically concatenating the second aggregation features corresponding to SITE1 and then performing maximum value aggregation, the neighbor aggregation features used to represent the neighbor information of each first neighbor URL of URL1 are obtained, i.e. ( ))= .

[0148] Step (4) sets the initial characteristics of URL2. and corresponding neighbor aggregation features By performing horizontal concatenation, we obtain the first feature of URL2. ,Right now Similarly, the initial features of SITE... and corresponding neighbor aggregation features By performing horizontal stitching, the first feature of SITE1 is obtained. ,Right now .in, This represents the function for horizontal splicing.

[0149] Step (5): For URL1, perform maximum value aggregation on the first feature of the first neighbor URL2 of URL1 to obtain the first aggregated feature corresponding to the URL type. The first feature of the first neighboring URL SITE1 of URL1 is aggregated by maximizing the maximum value to obtain the first aggregated feature corresponding to the SITE type. .

[0150] Step (6) multiplies each of the first aggregated features obtained in step (5) by the weight features corresponding to the same URL type to obtain the corresponding second aggregated features. For example... , ,in, and Similarly, this represents the weight features corresponding to different URL types.

[0151] Step (7) involves vertically concatenating the second aggregation features obtained in step (6) and then performing maximum value aggregation to obtain the neighbor aggregation features used to represent the neighbor information of each first neighbor URL of URL1. That is... ( , ))= .

[0152] Step (8) sets the initial characteristics of URL1. The first feature of URL1 is obtained by horizontally concatenating the features aggregated with those of its neighbors. That is, the first characteristic of URL1. For its target features .

[0153] Step S23: Determine the website type prediction result of the website to be detected based on the target features of the website to be detected.

[0154] In some feasible implementations, after determining the target features of the URL to be detected, the probability of the URL to be detected belonging to each preset type can be determined based on fully connected layers and softmax functions, and the probability of the URL to be detected belonging to each type can be determined as the URL type prediction result of the URL to be detected.

[0155] Optionally, based on the target features of the URL to be detected, a predicted type identifier of the website type corresponding to the URL to be detected can be obtained, and this identifier can be used as the website type prediction result of the URL to be detected.

[0156] The following is combined with Figure 11 The specific implementation method for determining the website type prediction result of the URL to be detected in the embodiments of this application is further described below. See also Figure 11 , Figure 11 This is a schematic diagram of a scenario for determining the prediction result of website type provided in an embodiment of this application. Figure 11 In this example, the URL to be detected (URL1) is WWW.AAABBB.COM. For this URL, the word vectors of each word in the URL can be determined, and the URL features of the URL can be determined based on these word vectors. Furthermore, the attribute information of the URL can be determined, and the corresponding attribute features can be determined based on this attribute information, such as constructing attribute features of the URL based on the attribute vectors of each attribute. If the webpage of the URL contains text information, the word vectors of each word in the text can be determined, and the webpage features of the URL can be determined based on these word vectors. Further, based on the URL features, attribute features, and webpage features of the URL, the initial features of the URL can be determined, such as by concatenating these features.

[0157] Furthermore, a heterogeneous graph of the World Wide Web can be constructed based on the various URL types and their interrelationships within the web. This heterogeneous graph can be viewed as a graph showing the URL relationships among the URLs on the web, such as... Figure 3 The URLs shown are represented by nodes, each node representing a URL, and the edges between nodes represent the relationships between the corresponding two URLs, such as one-way links, two-way links, etc.

[0158] Based on the heterogeneous graph of the World Wide Web, URL sampling can be performed based on sampling depth and the corresponding number of samples. The sampled URLs are then identified as associated URLs with a relationship to the URL to be detected. For example, a sampling depth of 1 corresponds to a sampling quantity of 2, and a sampling depth of 2 corresponds to a sampling quantity of 2. Based on this, URL sampling based on the heterogeneous graph of the World Wide Web yields the sampled URLs shown in Table 2 above. Based on the association between the URL to be detected and each sampled URL shown in Table 2, a URL heterogeneous graph representing the URL relationship between the URL to be detected and each associated URL can be constructed, such as... Figure 8 The URL relationship diagram shown.

[0159] Based on the URL heterogeneity graph representing the URL relationships between the URL to be detected and its associated URLs, the initial features of the two URLs 3 in the first neighboring URLs of URL2 are aggregated by maximizing their values ​​to obtain the first aggregated feature. Similarly, the initial features of the two SITE2 URLs are aggregated by maximizing their values ​​to obtain the first aggregated feature. The first aggregated features corresponding to the two URLs 3 are multiplied by the weight features corresponding to the URL type to obtain the second aggregated feature. The first aggregated features corresponding to the two SITE2 URLs are multiplied by the weight features corresponding to the SITE URL type to obtain the second aggregated feature. For URL2, the second aggregated features corresponding to URL2 are vertically concatenated and then subjected to maximum value aggregation to obtain the neighbor aggregated feature representing the neighbor information of each first neighboring URL of URL2. This neighbor aggregated feature is then horizontally concatenated with the initial features of URL2 to obtain the first feature of URL2. Similarly, the first feature of SITE1 can be obtained using the same method.

[0160] Furthermore, the first feature of URL1's first neighbor URL2 is aggregated by maximum value to obtain the first aggregated feature corresponding to the URL type. Similarly, the first feature of URL1's first neighbor SITE1 is aggregated by maximum value to obtain the first aggregated feature corresponding to the SITE type. The first aggregated features of each of URL1's first neighbor URLs are multiplied by the weight feature of the corresponding URL type to obtain the second aggregated features corresponding to each of URL1's first neighbor URLs. The second aggregated features corresponding to URL1 are vertically concatenated and then subjected to maximum value aggregation to obtain the neighbor aggregated features representing the neighbor information of each of URL1's first neighbor URLs. Finally, the neighbor aggregated features corresponding to URL1 are horizontally concatenated with their initial features to obtain the first feature of URL1, which is also the target feature of the URL to be detected.

[0161] The method for determining the initial features of each URL in the URL heterogeneity graph, which represents the URL to be detected and the URL relationships between each associated URL, is the same as the method for determining the initial features of the URL to be detected.

[0162] in, Figure 11The specific process for determining the target characteristics of URL1 is only a simplified example; for details, please refer to [link to documentation]. Figure 10 The implementation method for determining the target features of the URL to be detected.

[0163] After determining the target features of the URL to be detected (URL1), the website type prediction result can be determined based on the target features of the URL to be detected, and then the website type of the URL to be detected can be determined based on the website type prediction result. For example, based on the probability that the website of the URL to be detected belongs to each preset type, the preset type corresponding to the highest probability is determined as the website type of the URL to be detected. Alternatively, based on the correspondence between type identifiers and each preset type, the preset type corresponding to the predicted type identifier is determined as the website type of the URL to be detected.

[0164] In this embodiment, the target features of the URL to be detected are determined by the relationship between the URL to be detected and associated URLs, as well as the initial features of each URL. This approach considers not only the features of the URL to be detected itself, but also the feature correlation between URLs. As a result, the target features of the URL to be detected can characterize the common features of the URL to be detected and associated URLs. This improves the accuracy of determining the website type based on the target features and allows for the determination of the website type within a wide range of URLs, making it highly applicable.

[0165] Furthermore, in the process of determining the first feature of each URL, feature aggregation can reduce the amount of feature processing and improve the feature processing effect. Moreover, by using corresponding weighted features, the second aggregated features have the same feature dimension, which reduces the difficulty of feature processing and improves the accuracy of the first feature.

[0166] In addition to providing a method for determining website type, this application also provides a method for training a website type prediction model. The training method for the website type prediction model provided in this application can also be executed based on a server or a terminal. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these.

[0167] For details on the training method of the website type prediction model provided in this application embodiment, please refer to [link / reference]. Figure 12 , Figure 12 This is a flowchart illustrating a training method for the website type prediction model provided in an embodiment of this application. For example... Figure 12 As shown, this training method may specifically include the following steps:

[0168] Step S41: Obtain the training sample set.

[0169] In some feasible implementations, the training sample set includes multiple sample URLs, and each sample URL is labeled with a sample tag. For each sample URL, the sample tag ensures that the URL corresponds to the actual website type.

[0170] The sample URLs in the aforementioned training sample set can be obtained through methods such as acquiring user historical access records, URL sampling, and big data, or they can be obtained from databases, cloud storage, or blockchains used for storing URLs. The specific method can be determined based on the actual application scenario requirements and is not limited here. In short, a database can be viewed as an electronic filing cabinet—a place to store electronic files, and in this application, it can be used to store the sample training set.

[0171] Among them, based on big data technologies such as data mining, URLs can be mined to form the training sample set in this application.

[0172] Blockchain, in particular, is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. In this application, each data block in the blockchain can store the aforementioned training sample set.

[0173] Cloud storage is a new concept that extends and develops from the concept of cloud computing. It refers to the use of cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (also known as storage nodes) in the network to work together and store a large number of URLs.

[0174] Step S42: Input each sample URL into the initial model to obtain the website type prediction result for each sample URL, and determine the predicted website type of the corresponding sample URL based on the website type prediction result.

[0175] In some feasible implementations, the initial model described above determines the website type prediction result for each sample URL based on the following method:

[0176] Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and its associated URLs;

[0177] Based on the initial characteristics of the sample URL and its associated URLs, as well as the association between the sample URL and its associated URLs, the target characteristics of the sample URL are determined.

[0178] Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

[0179] Furthermore, the website type of the sample URL can be determined based on the website type prediction result. The specific determination method is the same as the method for determining the website type of the URL to be detected based on the website type prediction result of the URL to be detected, and will not be repeated here.

[0180] For details on how the initial model determines the website type prediction result for each sample URL, please refer to [link to relevant documentation]. Figure 2 The implementation method for determining the website type prediction result of the URL to be detected as shown in steps S21 to S23 will not be described in detail here.

[0181] As an example, for each sample URL, the initial features of the sample URL and its associated URLs are determined, including:

[0182] Determine at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to each of the sample URLs and the associated sample URLs. Based on at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to the URL, determine the initial sample features of the URL.

[0183] For details on how to determine the initial features of each URL in the sample URL and associated URLs, please refer to [link to relevant documentation]. Figure 2 The implementation method for determining the initial features of the URL to be detected and the associated URL in step S21 will not be described in detail here.

[0184] As an example, for each sample URL, based on the initial features of the sample URL and its associated URLs, and the association between the sample URL and its associated URLs, the target features of the sample URL are determined, including:

[0185] Based on the relationship between the sample URL and each associated sample URL, a sample URL relationship diagram is determined. The sample URL relationship diagram includes a first sample URL and a second sample URL. The first sample URL is the URL that is associated with only one URL among the sample URL and each associated sample URL, excluding the sample URL itself. The second sample URL is the URL that is associated with the sample URL and each associated sample URL, excluding the first sample URL.

[0186] For each second sample URL, based on the initial sample features of the second sample URL and the first sample features of the second neighbor URLs of the second sample URL, the first sample features of the second sample URL are determined. The second neighbor URLs of the second sample URL are the other neighbor URLs of the second sample URL excluding the parent URL of the second sample URL. If the second neighbor URLs of the second sample URL include the first sample URL, then the first sample features of the first sample URL are the initial sample features of the first sample URL.

[0187] The target features of each sample URL are determined from the first sample features of each second sample URL.

[0188] For details on how to determine the target features of the sample URL, please refer to [link to relevant documentation]. Figure 2 The implementation method for determining the target features of the URL to be detected in step S22 will not be described in detail here.

[0189] Step S43: Based on the sample labels and predicted website types of each sample URL, determine the training loss value. Iterate the initial model according to the training loss value and the training sample set until the training loss value meets the training termination condition. Then, determine the model at the end of training as the website type prediction model.

[0190] In some feasible implementations, a training loss value can be determined based on the sample labels and predicted website types of each sample URL. This training loss value characterizes the difference between the true website type and the predicted website type of the sample URL. The training loss value can be determined using the cross-entropy loss function or other loss functions, depending on the specific application scenario requirements; no restrictions are imposed here.

[0191] In some feasible implementations, based on the training loss value and the URLs of each sample in the training sample set, the initial model can be iteratively trained using the above method, and the relevant parameters in the initial model can be adjusted through a backpropagation network during each training process. When the training loss value meets the training termination condition, the model at the end of training can be determined as the final website type prediction model. The training termination condition can be that the training loss value reaches a convergent state, or that the training loss value is lower than a preset threshold, etc., and can be determined based on the actual application scenario requirements, without limitation here.

[0192] In some feasible implementations, the initial model described above may include multiple sub-models, each of which performs a different data processing task. For example, the initial model may include a feature construction sub-model, a target feature construction sub-model, and a prediction sub-model. The feature construction sub-model is used to determine the associated sample URLs that are related to each sample URL, and to determine the initial sample features of the sample URL and the associated sample URLs. The target feature construction sub-model is used to determine the target sample features of the sample URL based on the initial sample features of each sample URL and the associated sample URLs, as well as the relationship between the sample URL and the associated sample URLs. The prediction sub-model is used to determine the predicted website type of the sample URL based on the target sample features of each sample URL.

[0193] Furthermore, during the training process of the model based on the training loss value and the URLs of each sample in the training sample set, the relevant parameters of each of the above sub-models can be adjusted, and the training of each sub-model is completed when the training loss value meets the training termination condition, thus obtaining the final website type prediction model.

[0194] The feature processing involved in the website type determination method and the website type prediction model training method provided in this application, such as maximum value aggregation, feature concatenation, and fusion, can be implemented based on cloud computing. Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0195] As shown in Table 3, the website type prediction model trained based on the training method of the website type prediction model provided in the embodiments of this application has higher precision and recall compared with the prior art, and the combined performance (f1-score) of precision and recall is far superior to the prior art.

[0196] Table 3

[0197]

[0198] See Figure 13 , Figure 13 This is a schematic diagram of the website type determination device provided in an embodiment of this application. The website type determination device provided in an embodiment of this application includes:

[0199] The URL acquisition module 51 is used to acquire the URL to be detected;

[0200] The website type determination module 52 is used to input the above-mentioned URL to be detected into the website type prediction model, obtain the website type prediction result of the above-mentioned URL to be detected, and determine the website type of the above-mentioned URL to be detected based on the above-mentioned website type prediction result.

[0201] Specifically, the website type prediction model mentioned above is used for:

[0202] Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs;

[0203] Based on the initial features of each URL and the relationships between the URLs, the target features of the URLs to be detected are determined.

[0204] Based on the target characteristics of the URLs to be detected, the website type prediction results for the URLs to be detected are determined.

[0205] In some feasible implementations, the website type determination module 52 described above is used for:

[0206] A URL relationship graph is determined based on the association relationships between the aforementioned URLs. The URL relationship graph includes a first URL and a second URL. The first URL is the URL among the aforementioned URLs that has an association relationship with only one URL, excluding the URL to be detected. The second URL is the URL among the aforementioned URLs that is excluding the first URL.

[0207] For each of the above-mentioned second URLs, based on the initial characteristics of the second URL and the first characteristics of the first neighbor URLs of the second URL, the first characteristics of the second URL are determined. The first neighbor URLs of the second URL are the other neighbor URLs of the second URL excluding the parent URL of the second URL. If the first neighbor URLs of the second URL include the first URL, then the first characteristics of the first URL are the initial characteristics of the first URL.

[0208] The target features of the URL to be detected are determined from the first features of each of the aforementioned second URLs.

[0209] In some feasible implementations, for each of the aforementioned second URLs, the website type determination module 52 is used to:

[0210] Based on the first feature of the first neighboring URL of the second URL, determine the neighbor aggregation feature corresponding to the second URL;

[0211] Based on the initial features of the second URL and the aforementioned neighbor aggregation features, the first feature of the second URL is determined.

[0212] In some feasible implementations, for each of the aforementioned second URLs, the website type determination module 52 is used to:

[0213] Determine the URL type of the first neighbor URL of the second URL, and aggregate the first features of the first neighbor URLs corresponding to the same URL type to obtain the first aggregated features corresponding to each URL type;

[0214] Determine the weight features corresponding to each of the above URL types. For each of the above URL types, determine the second aggregate feature corresponding to that URL type based on the first aggregate feature and weight feature corresponding to that URL type.

[0215] Based on the second aggregation features of each location, the neighbor aggregation features corresponding to the second URL are determined.

[0216] In some feasible implementations, for each of the aforementioned second URLs, the website type determination module 52 is used to:

[0217] The initial features of the second URL are concatenated with the aforementioned neighbor aggregation features to obtain the first feature of the second URL.

[0218] In some feasible implementations, the website type determination module 52 described above is used for:

[0219] For each URL, determine at least one of the URL features, attribute features, or webpage features corresponding to the URL, and determine the initial features of the URL based on at least one of the URL features, attribute features, or webpage features corresponding to the URL.

[0220] In some feasible implementations, the website type determination module 52 described above is used for:

[0221] Determine the attribute information for each URL, and based on the attribute information, determine the attribute features corresponding to that URL;

[0222] The aforementioned attribute information includes at least one of the following: the number of characters in the URL, whether it contains preset characters, whether the corresponding domain name is an Internet Protocol IP domain name, or whether the corresponding site contains a random string.

[0223] In some feasible implementations, the website type determination module 52 described above is used for:

[0224] Determine at least one of the text features or image features of the webpage corresponding to each URL, and determine the webpage features corresponding to the URL based on the aforementioned text features or image features.

[0225] The aforementioned text features are determined based on word vectors of the text information in the aforementioned webpage, and the aforementioned image features are determined based on image information in the aforementioned webpage.

[0226] In some feasible implementations, the website type determination module 52 described above is used for:

[0227] Determine the sampling depth and the corresponding number of samples;

[0228] The URL to be detected is determined as the initial sampling URL. Based on the sampling depth and the corresponding sampling quantity, as well as the initial sampling URL, URL sampling is performed to obtain the associated URLs that are related to the URL to be detected.

[0229] In practice, the determination of the aforementioned website type can be performed through its built-in functional modules as described above. Figure 1 , Figure 2 and / or Figure 9 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0230] See Figure 14 , Figure 14 This is a schematic diagram of the structure of the training device for the website type prediction model provided in this application embodiment. The training device for the website type prediction model provided in this application embodiment includes:

[0231] The sample acquisition module 61 is used to acquire a training sample set. Each sample URL in the training sample set is labeled with a sample tag, which represents the real website type of the sample URL.

[0232] Training module 62 is used to input each of the above sample URLs into the initial model, obtain the website type prediction results of each of the above sample URLs, and determine the predicted website type of the corresponding sample URL based on the above website type prediction results;

[0233] The training evaluation module 63 is used to determine the training loss value based on the sample labels and predicted website types of each of the above sample URLs, and to iteratively train the above initial model according to the above training loss value and the above training sample set until the above training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model.

[0234] Specifically, when determining the website type prediction result for each of the above sample URLs, the initial model is used for:

[0235] Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the aforementioned associated URLs;

[0236] Based on the initial characteristics of the sample URL and each of the aforementioned associated sample URLs, as well as the association between the sample URL and each of the aforementioned associated sample URLs, the sample target characteristics of the sample URL are determined.

[0237] Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

[0238] In some feasible implementations, for each of the above sample URLs, the training module 62 is used to:

[0239] Determine at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to each of the sample URLs and the associated sample URLs. Based on at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to the URL, determine the initial sample features of the URL.

[0240] In some feasible implementations, for each of the above sample URLs, the training module 62 is used to:

[0241] Based on the relationship between the sample URL and each of the above-mentioned sample-related URLs, a sample URL relationship diagram is determined. The sample URL relationship diagram includes a first sample URL and a second sample URL. The first sample URL is the URL that is associated with only one URL among the sample URL and the above-mentioned sample-related URLs, excluding the sample URL itself. The second sample URL is the URL that is associated with the sample URL and the above-mentioned sample-related URLs, excluding the first sample URL itself.

[0242] For each of the above-mentioned second sample URLs, based on the initial sample features of the second sample URL and the first sample features of the second neighbor URLs of the second sample URL, the first sample features of the second sample URL are determined. The second neighbor URLs of the second sample URL are the other neighbor URLs of the second sample URL excluding the parent URL of the second sample URL. If the second neighbor URLs of the second sample URL include the first sample URL, then the first sample features of the first sample URL are the initial sample features of the first sample URL.

[0243] The target features of the sample URL are determined from the first sample features of each of the aforementioned second sample URLs.

[0244] In practice, the determination of the aforementioned website type can be performed through its built-in functional modules as described above. Figure 12 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0245] See Figure 15 , Figure 15This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 15 As shown, the electronic device 1000 in this embodiment may include: a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the electronic device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 15 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0246] exist Figure 15 In the illustrated electronic device 1000, the network interface 1004 provides network communication functionality; the user interface 1003 is primarily used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to implement the website type determination method provided in this embodiment. Specific implementation:

[0247] Obtain the URL to be tested;

[0248] Input the above-mentioned URL to be detected into the website type prediction model to obtain the website type prediction result of the above-mentioned URL to be detected, and determine the website type of the above-mentioned URL to be detected based on the above-mentioned website type prediction result.

[0249] Specifically, the processor 1001 is used for:

[0250] Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs;

[0251] Based on the initial features of each URL and the relationships between the URLs, the target features of the URLs to be detected are determined.

[0252] Based on the target characteristics of the URLs to be detected, the website type prediction results for the URLs to be detected are determined.

[0253] In some feasible implementations, the processor 1001 described above is used for:

[0254] A URL relationship graph is determined based on the association relationships between the aforementioned URLs. The URL relationship graph includes a first URL and a second URL. The first URL is the URL among the aforementioned URLs that has an association relationship with only one URL, excluding the URL to be detected. The second URL is the URL among the aforementioned URLs that is excluding the first URL.

[0255] For each of the above-mentioned second URLs, based on the initial characteristics of the second URL and the first characteristics of the first neighbor URLs of the second URL, the first characteristics of the second URL are determined. The first neighbor URLs of the second URL are the other neighbor URLs of the second URL excluding the parent URL of the second URL. If the first neighbor URLs of the second URL include the first URL, then the first characteristics of the first URL are the initial characteristics of the first URL.

[0256] The target features of the URL to be detected are determined from the first features of each of the aforementioned second URLs.

[0257] In some feasible implementations, for each of the aforementioned second URLs, the processor 1001 is configured to:

[0258] Based on the first feature of the first neighboring URL of the second URL, determine the neighbor aggregation feature corresponding to the second URL;

[0259] Based on the initial features of the second URL and the aforementioned neighbor aggregation features, the first feature of the second URL is determined.

[0260] In some feasible implementations, for each of the aforementioned second URLs, the processor 1001 is configured to:

[0261] Determine the URL type of the first neighbor URL of the second URL, and aggregate the first features of the first neighbor URLs corresponding to the same URL type to obtain the first aggregated features corresponding to each URL type;

[0262] Determine the weight features corresponding to each of the above URL types. For each of the above URL types, determine the second aggregate feature corresponding to that URL type based on the first aggregate feature and weight feature corresponding to that URL type.

[0263] Based on the second aggregation features of each location, the neighbor aggregation features corresponding to the second URL are determined.

[0264] In some feasible implementations, for each of the aforementioned second URLs, the processor 1001 is configured to:

[0265] The initial features of the second URL are concatenated with the aforementioned neighbor aggregation features to obtain the first feature of the second URL.

[0266] In some feasible implementations, the processor 1001 described above is used for:

[0267] For each URL, determine at least one of the URL features, attribute features, or webpage features corresponding to the URL, and determine the initial features of the URL based on at least one of the URL features, attribute features, or webpage features corresponding to the URL.

[0268] In some feasible implementations, the processor 1001 described above is used for:

[0269] Determine the attribute information for each URL, and based on the attribute information, determine the attribute features corresponding to that URL;

[0270] The aforementioned attribute information includes at least one of the following: the number of characters in the URL, whether it contains preset characters, whether the corresponding domain name is an Internet Protocol IP domain name, or whether the corresponding site contains a random string.

[0271] In some feasible implementations, the processor 1001 described above is used for:

[0272] Determine at least one of the text features or image features of the webpage corresponding to each URL, and determine the webpage features corresponding to the URL based on the aforementioned text features or image features.

[0273] The aforementioned text features are determined based on word vectors of the text information in the aforementioned webpage, and the aforementioned image features are determined based on image information in the aforementioned webpage.

[0274] In some feasible implementations, the processor 1001 described above is used for:

[0275] Determine the sampling depth and the corresponding number of samples;

[0276] The URL to be detected is determined as the initial sampling URL. Based on the sampling depth and the corresponding sampling quantity, as well as the initial sampling URL, URL sampling is performed to obtain the associated URLs that are related to the URL to be detected.

[0277] Optionally, the processor 1001 described above can also be used to implement the training method of the website prediction model provided in the embodiments of this application, specifically:

[0278] Obtain a training sample set. Each sample URL in the training sample set is labeled with a sample tag, which represents the real website type of the sample URL.

[0279] Input each of the above sample URLs into the initial model to obtain the website type prediction results for each of the above sample URLs. Based on the above website type prediction results, determine the predicted website type for the corresponding sample URLs.

[0280] Based on the sample tags and predicted website types of each of the above sample URLs, the training loss value is determined. The initial model is iteratively trained according to the training loss value and the training sample set until the training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model.

[0281] Specifically, the processor 1001 is used for:

[0282] Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the aforementioned associated URLs;

[0283] Based on the initial characteristics of the sample URL and each of the aforementioned associated sample URLs, as well as the association between the sample URL and each of the aforementioned associated sample URLs, the sample target characteristics of the sample URL are determined.

[0284] Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

[0285] In some feasible implementations, for each of the above sample URLs, the processor 1001 is used to:

[0286] Determine at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to each of the sample URLs and the associated sample URLs. Based on at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to the URL, determine the initial sample features of the URL.

[0287] In some feasible implementations, for each of the above sample URLs, the processor 1001 is used to:

[0288] Based on the relationship between the sample URL and each of the above-mentioned sample-related URLs, a sample URL relationship diagram is determined. The sample URL relationship diagram includes a first sample URL and a second sample URL. The first sample URL is the URL that is associated with only one URL among the sample URL and the above-mentioned sample-related URLs, excluding the sample URL itself. The second sample URL is the URL that is associated with the sample URL and the above-mentioned sample-related URLs, excluding the first sample URL itself.

[0289] For each of the above-mentioned second sample URLs, based on the initial sample features of the second sample URL and the first sample features of the second neighbor URLs of the second sample URL, the first sample features of the second sample URL are determined. The second neighbor URLs of the second sample URL are the other neighbor URLs of the second sample URL excluding the parent URL of the second sample URL. If the second neighbor URLs of the second sample URL include the first sample URL, then the first sample features of the first sample URL are the initial sample features of the first sample URL.

[0290] The target features of the sample URL are determined from the first sample features of each of the aforementioned second sample URLs.

[0291] It should be understood that in some feasible implementations, the processor 1001 described above may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0292] In specific implementation, the aforementioned electronic device 1000 can perform the above-described actions through its built-in functional modules. Figure 1 , Figure 2 , Figure 9 and / or Figure 12 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0293] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement... Figure 1 , Figure 2 , Figure 9 and / or Figure 12 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0294] The aforementioned computer-readable storage medium can be an internal storage unit of the apparatus and / or electronic device provided in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0295] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform... Figure 1 , Figure 2 , Figure 9 and / or Figure 12 The methods provided for each step in the process.

[0296] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0297] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0298] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for determining website type, characterized in that, The method includes: Obtain the URL to be tested; The URL to be detected is input into the website type prediction model to obtain the website type prediction result of the URL to be detected, and the website type of the URL to be detected is determined based on the website type prediction result. The website type prediction model determines the website type prediction result based on the following method: Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs; A URL relationship graph is determined based on the association relationships between the URLs. The URL relationship graph includes a first URL and a second URL. The first URL is a URL among the URLs that has an association with only one URL, excluding the URL to be detected. The second URL is a URL among the URLs that is excluding the first URL. For each second URL, a first feature of the second URL is determined based on the initial feature of the second URL and the first feature of the first neighbor URL of the second URL. The first neighbor URL of the second URL is the other neighbor URLs of the second URL, excluding the parent URL of the second URL. If the first neighbor URL of the second URL includes the first URL, then the first feature of the first URL is the initial feature of the first URL. The target feature of the URL to be detected is determined from the first features of each second URL. The website type prediction result of the website to be detected is determined based on the target features of the website to be detected.

2. The method according to claim 1, characterized in that, For each second URL, determining the first characteristic of the second URL based on the initial characteristics of the second URL and the first characteristics of the first neighboring URLs of the second URL includes: Based on the first feature of the first neighboring URL of the second URL, determine the neighbor aggregation feature corresponding to the second URL; Based on the initial characteristics of the second URL and the neighbor aggregation characteristics, the first characteristic of the second URL is determined.

3. The method according to claim 2, characterized in that, For each second URL, determining the neighbor aggregation features corresponding to the second URL based on the first features of the first neighbor URLs of the second URL includes: Determine the URL type of the first neighbor URL of the second URL, and aggregate the first features of the first neighbor URLs corresponding to the same URL type to obtain the first aggregated features corresponding to each URL type; Determine the weight features corresponding to each URL type. For each URL type, determine the second aggregate feature corresponding to that URL type based on the first aggregate feature and weight feature corresponding to that URL type. Based on the second aggregation features of each location, the neighbor aggregation features corresponding to the second URL are determined.

4. The method according to claim 2, characterized in that, For each second URL, the initial features based on that second URL and the neighbor aggregation features, The first characteristic for determining the second URL includes: The initial features of the second URL and the neighbor aggregation features are concatenated to obtain the first feature of the second URL.

5. The method according to claim 1, characterized in that, The initial features for determining the URL to be detected and the associated URL include: For each URL, determine at least one of the URL features, attribute features, or webpage features corresponding to the URL, and determine the initial features of the URL based on at least one of the URL features, attribute features, or webpage features corresponding to the URL.

6. The method according to claim 5, characterized in that, Determine the attribute characteristics corresponding to each URL, including: Determine the attribute information for each URL, and determine the attribute features corresponding to that URL based on the attribute information; The attribute information includes at least one of the following: the number of characters in the URL, whether it contains preset characters, whether the corresponding domain name is an Internet Protocol (IP) domain name, or whether the corresponding site contains a random string.

7. The method according to claim 5, characterized in that, Determine the characteristics of the webpage corresponding to each URL, including: Determine at least one of the text features or image features of the webpage corresponding to each URL, and determine the webpage features corresponding to the URL based on at least one of the text features or image features; The text features are determined based on word vectors of the text information in the webpage, and the image features are determined based on image information in the webpage.

8. The method according to claim 1, characterized in that, The determination of associated URLs that are related to the URL to be detected includes: Determine the sampling depth and the corresponding number of samples; The URL to be detected is determined as the initial sampling URL. Based on the sampling depth, the corresponding sampling quantity, and the initial sampling URL, URL sampling is performed to obtain associated URLs that are related to the URL to be detected.

9. A training method for a website type prediction model, characterized in that, The method includes: Obtain a training sample set, wherein each sample URL in the training sample set is labeled with a sample tag, and the sample tag represents the real website type of the sample URL; Each of the sample URLs is input into the initial model to obtain the website type prediction result for each of the sample URLs. Based on the website type prediction result, the predicted website type of the corresponding sample URL is determined. Based on the sample tags and predicted website types of each sample URL, a training loss value is determined. The initial model is iteratively trained according to the training loss value and the training sample set until the training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model. The initial model determines the website type prediction result for each sample URL based on the following method: Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the associated URLs; Based on the association between the sample URL and each of the associated sample URLs, a sample URL relationship graph is determined. This graph includes a first sample URL and a second sample URL. The first sample URL is the URL among the sample URL and the associated sample URLs that has an association with only one URL, excluding the sample URL itself. The second sample URL is the URL among the sample URL and the associated sample URLs that exclude the first sample URL. For each second sample URL, based on its initial sample features and the first sample features of its second neighbor URLs, a first sample feature is determined. The second neighbor URLs of the second sample URL are the other neighbor URLs besides its parent URL. If the second neighbor URL of the second sample URL includes the first sample URL, then the first sample feature of the first sample URL is its initial sample feature. The target sample feature of each second sample URL is determined from its first sample features. Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

10. The method according to claim 9, characterized in that, For each of the sample URLs, determining the initial characteristics of the sample URL and its associated URLs includes: Determine at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to each of the sample URLs and the associated sample URLs. Based on at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to the URL, determine the initial sample features of the URL.

11. A website type determination device, characterized in that, The device includes: The URL acquisition module is used to obtain the URL to be detected. The website type determination module is used to input the URL to be detected into the website type prediction model, obtain the website type prediction result of the URL to be detected, and determine the website type of the URL to be detected based on the website type prediction result. Specifically, the website type prediction model is used for: Identify associated URLs that are related to the URL to be detected, and determine the initial characteristics of the URL to be detected and the associated URLs; A URL relationship graph is determined based on the association relationships between the URLs. The URL relationship graph includes a first URL and a second URL. The first URL is a URL among the URLs that has an association with only one URL, excluding the URL to be detected. The second URL is a URL among the URLs that is excluding the first URL. For each second URL, a first feature of the second URL is determined based on the initial feature of the second URL and the first feature of the first neighbor URL of the second URL. The first neighbor URL of the second URL is the other neighbor URLs of the second URL, excluding the parent URL of the second URL. If the first neighbor URL of the second URL includes the first URL, then the first feature of the first URL is the initial feature of the first URL. The target feature of the URL to be detected is determined from the first features of each second URL. The website type prediction result of the website to be detected is determined based on the target features of the website to be detected.

12. The apparatus according to claim 11, characterized in that, For each of the second URLs, when the website type determination module determines the first characteristic of the second URL based on the initial characteristics of the second URL and the first characteristics of the first neighboring URLs of the second URL, it is used to: Based on the first feature of the first neighboring URL of the second URL, determine the neighbor aggregation feature corresponding to the second URL; Based on the initial characteristics of the second URL and the neighbor aggregation characteristics, the first characteristic of the second URL is determined.

13. The apparatus according to claim 12, characterized in that, For each second URL, when the website type determination module determines the neighbor aggregation feature corresponding to the second URL based on the first feature of the first neighbor URLs of the second URL, it is used to: Determine the URL type of the first neighbor URL of the second URL, and aggregate the first features of the first neighbor URLs corresponding to the same URL type to obtain the first aggregated features corresponding to each URL type; Determine the weight features corresponding to each URL type. For each URL type, determine the second aggregate feature corresponding to that URL type based on the first aggregate feature and weight feature corresponding to that URL type. Based on the second aggregation features of each location, the neighbor aggregation features corresponding to the second URL are determined.

14. The apparatus according to claim 12, characterized in that, For each of the second URLs, when the website type determination module determines the first feature of the second URL based on its initial features and the neighbor aggregation features, it is used to: The initial features of the second URL and the neighbor aggregation features are concatenated to obtain the first feature of the second URL.

15. The apparatus according to claim 11, characterized in that, When the website type determination module determines the initial characteristics of the URL to be detected and the associated URL, it is used for: For each URL, determine at least one of the URL features, attribute features, or webpage features corresponding to the URL, and determine the initial features of the URL based on at least one of the URL features, attribute features, or webpage features corresponding to the URL.

16. The apparatus according to claim 15, characterized in that, When the website type determination module determines the attribute characteristics corresponding to each URL, it is used for: Determine the attribute information for each URL, and determine the attribute features corresponding to that URL based on the attribute information; The attribute information includes at least one of the following: the number of characters in the URL, whether it contains preset characters, whether the corresponding domain name is an Internet Protocol (IP) domain name, or whether the corresponding site contains a random string.

17. The apparatus according to claim 15, characterized in that, When the website type determination module determines the webpage characteristics corresponding to each URL, it is used for: Determine at least one of the text features or image features of the webpage corresponding to each URL, and determine the webpage features corresponding to the URL based on at least one of the text features or image features; The text features are determined based on word vectors of the text information in the webpage, and the image features are determined based on image information in the webpage.

18. The apparatus according to claim 11, characterized in that, When the website type determination module determines an associated website that is related to the website to be detected, it is used for: Determine the sampling depth and the corresponding number of samples; The URL to be detected is determined as the initial sampling URL. Based on the sampling depth, the corresponding sampling quantity, and the initial sampling URL, URL sampling is performed to obtain associated URLs that are related to the URL to be detected.

19. A training device for a website type prediction model, characterized in that, The device includes: The sample acquisition module is used to acquire a training sample set, wherein each sample URL in the training sample set is labeled with a sample tag, and the sample tag represents the real website type of the sample URL; The training module is used to input each of the sample URLs into the initial model, obtain the website type prediction result for each of the sample URLs, and determine the predicted website type of the corresponding sample URL based on the website type prediction result; The training evaluation module is used to determine the training loss value based on the sample tags and predicted website types of each sample URL, and to iteratively train the initial model according to the training loss value and the training sample set until the training loss value meets the training termination condition. The model at the end of training is then determined as the website type prediction model. Specifically, when determining the website type prediction result for each of the sample URLs, the aforementioned initial model is used for: Identify the associated URLs of the sample URL and determine the initial sample characteristics of the sample URL and the associated URLs; Based on the association between the sample URL and each of the associated sample URLs, a sample URL relationship graph is determined. This graph includes a first sample URL and a second sample URL. The first sample URL is the URL among the sample URL and the associated sample URLs that has an association with only one URL, excluding the sample URL itself. The second sample URL is the URL among the sample URL and the associated sample URLs that exclude the first sample URL. For each second sample URL, based on its initial sample features and the first sample features of its second neighbor URLs, a first sample feature is determined. The second neighbor URLs of the second sample URL are the other neighbor URLs besides its parent URL. If the second neighbor URL of the second sample URL includes the first sample URL, then the first sample feature of the first sample URL is its initial sample feature. The target sample feature of each second sample URL is determined from its first sample features. Based on the target features of the sample URL, the website type prediction result of the sample URL is determined.

20. The apparatus according to claim 19, characterized in that, For each of the sample URLs, when the training module determines the initial features of the sample URL and its associated URLs, it uses the following methods: Determine at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to each of the sample URLs and the associated sample URLs. Based on at least one of the sample URL features, sample attribute features, or sample webpage features corresponding to the URL, determine the initial sample features of the URL.

21. An electronic device, characterized in that, It includes a processor and a memory, which are interconnected; The memory is used to store computer programs; The processor is configured to perform the method as described in any one of claims 1 to 10 when the computer program is invoked.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method of any one of claims 1 to 10.

23. A computer program product comprising computer instructions stored in a computer-readable storage medium; a processor of an electronic device reading from the computer-readable storage medium and executing the computer instructions, causing the electronic device to perform the method of any one of claims 1 to 10.

Images