Risk website identification method, device and medium based on multi-level high-risk characteristics

Abstract

The present application discloses a risk website identification method, device and storage medium based on multi-level high-risk features. The method is implemented by: collecting access website information, which is provided with risk labels; extracting features from the access website information through a shallow feature extraction strategy to obtain target shallow features; extracting features from the access website information through a deep feature extraction strategy to obtain deep features of the access website, comparing the deep features of the access website with the deep features of the risk website to obtain comparison similarity features, and using the deep features of the access website and the comparison similarity features as target deep features; iteratively training a preset risk website identification model until the preset convergence conditions are met to obtain a trained risk website identification model; predicting the access website to be identified based on the trained risk website identification model to obtain the risk level of the access website to be identified. The correlation of risk features can be enhanced and the prediction accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method, device and medium for identifying risky websites based on multi-level high-risk features. Background Art

[0002] Existing websites usually use blacklists to determine whether the websites visited by users are malicious, phishing, or fraudulent websites. However, this method usually requires a lot of manual screening, resulting in high labor costs. Alternatively, it can be done through the URL (Uniform Resource Locator, which is a concise representation of the location and access method of resources available on the Internet, and is the address of standard resources on the Internet) feature, which has a fast recognition speed and can actively discover abnormal websites. However, due to the limited information contained in the URL, it is not possible to judge whether a website is an abnormal website or a normal website only from the URL. Alternatively, it is also possible to identify the risks of websites through neural networks, but existing neural network models usually only extract and predict single features, lack risk feature association, and have poor prediction accuracy. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for identifying risky websites based on multi-level high-risk features to solve at least one problem existing in the above-mentioned prior art.

[0004] In a first aspect, the embodiment of the present application is implemented by providing a method for identifying risky websites based on multi-level high-risk features, the method comprising the following steps:

[0005] Collecting access website information, wherein the access website information is provided with a risk label;

[0006] Extracting features from the accessed URL information using a shallow feature extraction strategy to obtain target shallow features;

[0007] Extracting features from the accessed website information through a deep feature extraction strategy to obtain deep features of the accessed website, comparing the deep features of the accessed website with the deep features of the risky website to obtain a comparison similarity feature, and using the deep features of the accessed website and the comparison similarity feature as target deep features;

[0008] Based on the shallow features of the target and the deep features of the target, iteratively train the preset risk website identification model until a preset convergence condition is met, thereby obtaining a trained risk website identification model;

[0009] The to-be-identified access website is predicted based on the trained risk website identification model to obtain the risk level of the to-be-identified access website.

[0010] In a second aspect, a risk website identification device based on multi-level high-risk features is provided, comprising:

[0011] A visited website information collection unit, used to collect visited website information, wherein the visited website information is provided with a risk tag;

[0012] A target shallow feature extraction unit, used for extracting features from the accessed website information through a shallow feature extraction strategy to obtain target shallow features;

[0013] A target deep feature extraction unit is used to extract features from the accessed website information through a deep feature extraction strategy to obtain deep features of the accessed website, compare the deep features of the accessed website with the deep features of the risky website to obtain a comparison similarity feature, and use the deep features of the accessed website and the comparison similarity feature as target deep features;

[0014] A model training unit, used for iteratively training a preset risk website identification model based on the target shallow features and the target deep features until a preset convergence condition is met, thereby obtaining a trained risk website identification model;

[0015] The prediction unit is used to predict the to-be-identified access website based on the trained risk website identification model to obtain the risk level of the to-be-identified access website.

[0016] In a third aspect, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored on the memory and running on the processor, wherein when the processor executes the computer-readable instructions, the method for identifying risky websites based on multi-level high-risk features as described above is implemented.

[0017] In a fourth aspect, a readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the risk website identification method based on multi-level high-risk features as described above is implemented.

[0018] The above-mentioned risk website identification method, device, computer equipment and storage medium based on multi-level high-risk features, and its method implementation includes: collecting access website information, the access website information is set with a risk label; extracting features from the access website information through a shallow feature extraction strategy to obtain target shallow features; extracting features from the access website information through a deep feature extraction strategy to obtain deep features of the access website, comparing the deep features of the access website with the deep features of the risk website to obtain comparison similarity features, and using the deep features of the access website and the comparison similarity features as target deep features; iteratively training a preset risk website identification model based on the target shallow features and the target deep features until a preset convergence condition is met, thereby obtaining a trained risk website identification model; predicting the access website to be identified based on the trained risk website identification model to obtain the risk level of the access website to be identified. In the embodiment of the present application, by introducing a multi-level high-risk feature extraction mechanism, deep features and shallow features are extracted to more efficiently screen out riskier websites. At the same time, multi-dimensional high-risk features are automatically extracted and identified from website content, user behavior and image data, and through deep neural network training, the correlation analysis between different risk features is enhanced to ensure that the system can accurately and timely detect and respond to potential online fraud. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 This is a flow chart of a method for identifying risky websites based on multi-level high-risk features in one embodiment of the present application;

[0021] Figure 2 This is a flow chart of a method for constructing a feature database in one embodiment of the present application;

[0022] Figure 3 This is a flow chart of a method for extracting shallow features of a target in one embodiment of the present application;

[0023] Figure 4 This is a flow chart of a method for downloading complete website information in one embodiment of the present application;

[0024] Figure 5 It is a flow chart of a method for extracting deep features of a target in one embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of the network architecture of a risky website identification model in one embodiment of the present application;

[0026] Figure 7 This is a schematic diagram of a structure of a risk website identification device based on multi-level high-risk features in one embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In one embodiment, if Figure 1 As shown, a method for identifying risky websites based on multi-level high-risk features is provided, comprising the following steps:

[0030] In step S110, the access website information is collected, and the access website information is set with a risk tag;

[0031] In an embodiment of the present application, the access URL information can be obtained by collecting IP URL data of each port in the local area network. For the risk label of the access URL, part of it may be the risk type URL marked by the authoritative channel, and the other part may come from the manual risk marking of the daily access URL. The authoritative channel may specifically be a party with review qualifications, such as the network operator to which the IP port belongs, to conduct risk review and marking. Or it may be a manual marking based on the access results when the user makes a daily visit. It should be noted that the risk label may specifically include the risk level, that is, it can be marked with different values ​​according to the degree of the risk level. For example, high risk can be marked as 2, medium risk can be marked as 1, low risk can be marked as 0, etc.

[0032] In step S120, feature extraction is performed on the accessed website information using a shallow feature extraction strategy to obtain target shallow features;

[0033] In the embodiment of the present application, a multi-level high-risk feature extraction strategy module can be constructed. The extraction method of the multi-level high-risk feature extraction strategy module can be specifically as follows: Figure 2As shown, by performing shallow information acquisition and deep information acquisition on the collected URL information, and performing shallow feature extraction and deep feature extraction based on the constructed shallow feature extraction strategy and deep feature extraction strategy respectively, complete website feature information is obtained for subsequent model training.

[0034] In the embodiment of the present application, the process of shallow feature extraction can be specifically as follows: Figure 3 As shown, the shallow target features may specifically include at least one of domain value information, website whois information, website length information, and website access information, or any combination thereof. Wherein, website whois information refers to the transmission protocol of information such as the IP address and owner of the domain name.

[0035] The specific method of obtaining the domain value information is as follows: a large amount of domain name sample data is collected, which may include high-value domain names, such as www.worldcoin.com, and low-value domain names, such as www.ajsfga.com. At the same time, current hot keyword data, such as gpt, ai, blockchain, etc., can be collected and saved in the hot keyword database for subsequent use. The collected access URLs can be segmented to extract their core parts. For example, the core domain name of www.worldcoin.com can be worldcoin, and the character length of the core domain name is determined, such as the character length of worldcoin is 9. Then, the core domain name can be compared with the pre-constructed high-value dictionary and hot keyword data, and the core domain name can be scored for high value and hot keyword according to the comparison results. At the same time, according to the obtained character length, the character length can be scored according to the preset rules, for example, the longer the character length, the higher the score. For character features, the character meaning can be analyzed to score it. For example, if the core domain name has a clear meaning, it means that it has a higher value. At this time, it can be given a higher score. Then, a weighted sum is performed on the obtained dictionary score, character length score, hot keyword score and character feature score to obtain a value score, which is used as the domain name value information.

[0036] For the whois information of the website, the whois information of the designated website can be obtained by using automated requests. Specifically, it may include the creation time of the website, the expiration date of the website, the organization and location of creation, and other related information. Based on this related information, further analysis can be performed to obtain characteristic information such as the year of establishment and registration location of the website.

[0037] The URL length can be a corresponding int value, which is the length of the URL domain name.

[0038] The website access information mainly includes two parts. The first part may be the number of visitors, and the second part may be the number of visits. The number of visitors and the number of visits to the website can be determined by connecting to the network operator and counting the number of visitors and the number of visits through the network operator's backend.

[0039] In step S130, feature extraction is performed on the accessed website information by using a deep feature extraction strategy to obtain deep features of the accessed website, the deep features of the accessed website are compared with the deep features of the risky website to obtain a comparison similarity feature, and the deep features of the accessed website and the comparison similarity feature are used as target deep features;

[0040] In an embodiment of the present application, the target deep features may specifically include at least one of HTML features, JS features, CSS features, multimedia resource file features, and overall risk item features, or any combination thereof.

[0041] In an embodiment of the present application, the access URL information may specifically include HTML files, JS files, CSS files, multimedia resource files, and their website links and file structures, such as directory structures. Based on the feature extraction of the HTML file, it may specifically include: DOM structure extraction, HTML file can be HTML block (header, body, footer, etc.) text feature extraction, HTML function feature extraction, HTML code style feature extraction, etc., to obtain HTML features. For the feature extraction of JS files, it may specifically include: JS file name feature extraction, function method name feature extraction, JS code style feature extraction, JS reference library feature extraction, JS function feature extraction, etc.; for CSS files, CSS file name features, CSS selector features, CSS animation features, CSS font features, CSS color features, etc. are obtained to obtain JS features. For the feature extraction of multimedia resource files, it may specifically include: resource file structure feature extraction, image attribute feature extraction, resource file MD5 value feature extraction, image content feature extraction, image quality feature extraction, etc., to obtain multimedia resource file features. For the feature extraction of overall risk items, it may specifically include code encryption features, data encryption features, website text features, grammatical features, risk field features, etc.

[0042] The HTML features, JS features, CSS features, multimedia resource file features and overall risk item features collected above are aggregated to serve as deep features of the visited URL. Similarly, for risky URLs, deep features can also be extracted in the above manner to obtain deep features of risky URLs. Then, the deep features of the visited URL and the deep features of the risky URL are calculated for similarity. For example, similarity algorithms such as cosine similarity and Euclidean distance can be used to calculate the similarity to obtain a similarity value. Based on the similarity value, a comparative similarity feature is determined and aggregated with the deep features of the visited URL to jointly construct the target deep features.

[0043] In step S140, based on the target shallow features and the target deep features, the preset risk website identification model is iteratively trained until the preset convergence condition is met, thereby obtaining a trained risk website identification model;

[0044] In the embodiment of the present application, the target deep features are fused with the target shallow features and vectorized to construct a feature database for iterative training of the preset risk website identification model. Optionally, the target shallow features can be constructed as a first multidimensional vector, which may specifically include website value features, website establishment years features, registered address features, website length features, number of visitors features, number of visits features, etc. Similarly, the target deep features can be constructed as a second multidimensional vector, which may specifically include DOM structure features, HTML reference file features, HTML block text features, HTML function features, HTML code style features, JS file name features, function method name features, JS code style features, JS reference library features, JS function features, CSS file name features, CSS selector features, CSS animation features, CSS font features, CSS color features, etc., resource file structure features, image attribute features, resource file MD5 value features, image content features, image quality features, code encryption features, data encryption features, website text features, grammatical errors, risk field features, and similarity features with the black industry intelligence library, etc. Then, the first multidimensional vector is merged with the second multidimensional vector, and each feature is matched with a risk label, thereby forming a feature database for subsequent training of the model.

[0045] In an embodiment of the present application, the preset risk website identification model may be a deep neural network (DNN), which, as the core architecture of deep learning, has demonstrated outstanding capabilities in processing complex data and identifying complex patterns. DNN is composed of multiple layers of neurons, each layer of neurons extracts features from the previous layer through nonlinear transformations, thereby capturing the complex structure of the data layer by layer. In website risk identification, DNN can automatically identify high-risk features from web page content, user behavior, and image data through hierarchical feature extraction. It should be noted that other neural network models, such as CNN models, can also be used to construct the risk website identification model.

[0046] Optionally, the DNN model may specifically include an input layer, multiple hidden layers, for example, 4, and an output layer. A single group of multidimensional feature vectors may be arbitrarily selected and input into the input layer, and then pass through multiple hidden layers in sequence through the fully connected layer. Each layer may be activated by an activation function, and finally the risk level may be output through the output layer, and the loss value may be calculated by a loss function based on the output result and the risk label. Then, the next group of multidimensional feature vectors is selected from the remaining multidimensional feature vectors and input into the input layer, and the above steps are repeated until the preset convergence conditions are met, such as when the number of iterations reaches the maximum number of iterations, or when the loss value meets the preset loss conditions, the iteration is stopped, and a trained risk website identification model may be obtained.

[0047] In step S150, the risk level of the to-be-identified access website is obtained by predicting the to-be-identified access website based on the trained risk website identification model.

[0048] In an embodiment of the present application, after the trained risk website identification model is obtained, the to-be-identified access website address can be obtained, and the risk level prediction can be performed through the risk website identification model to obtain the risk level of the to-be-identified access website address.

[0049] In the embodiment of the present application, a risk warning can be issued based on the predicted risk level of the to-be-identified accessed website, such as outputting a prompt message "The accessed website is a high-risk website, please do not enter", or outputting a sound or light prompt. At the same time, for network operators, risk screening and monitoring of different websites can be implemented to avoid network security issues.

[0050] In an embodiment of the present application, a method for identifying risky websites based on multi-level high-risk features is provided, including: collecting access website information, wherein the access website information is provided with a risk label; extracting features from the access website information through a shallow feature extraction strategy to obtain target shallow features; extracting features from the access website information through a deep feature extraction strategy to obtain deep features of the access website, comparing the deep features of the access website with the deep features of the risky website to obtain comparison similarity features, and using the deep features of the access website and the comparison similarity features as target deep features; iteratively training a preset risk website identification model based on the target shallow features and the target deep features until a preset convergence condition is met to obtain a trained risk website identification model; predicting the access website to be identified based on the trained risk website identification model to obtain the risk level of the access website to be identified. In the embodiment of the present application, by introducing a multi-level high-risk feature extraction mechanism, deep features and shallow features are extracted to more efficiently screen out riskier websites. At the same time, multi-dimensional high-risk features are automatically extracted and identified from website content, user behavior and image data, and through deep neural network training, the correlation analysis between different risk features is enhanced to ensure that the system can accurately and timely detect and respond to potential online fraud.

[0051] In one embodiment of the present application, the shallow features include domain value information. The shallow feature extraction strategy is used to extract features from the access URL information to obtain target shallow features, including:

[0052] Extract domain name information based on URL information;

[0053] Perform word segmentation on domain name information and extract core domain names;

[0054] Calculate the dictionary score, character length score, hot keyword score, and character feature score of the core domain name respectively;

[0055] The dictionary score, character length score, keyword score and character feature score are weighted and summed to obtain the domain name value score.

[0056] Optionally, the domain name of the accessed website information and the current hot keyword data can be determined, for example, www.worldcoin.com represents a high-value domain name, www.ajsfga.com represents a low-value domain name, and hot keywords such as gpt, ai, blockchain, etc. The domain name can be segmented to extract the core domain name, for example, the core domain name of www.worldcoin.com is worldcoin, and its character length is calculated, for example, the character length of worldcoin can be 9. The core domain name is matched with a high-value dictionary; the high-value dictionary may include high-value domain names such as world, coin, and tech. At the same time, the current hot keywords and pre-constructed hot keywords, such as gpt, can also be matched; at the same time, the character features of the core domain name, such as the meaning of the characters, can also be analyzed. Then, according to the preset scoring rules, the dictionary score, character length score, hot keyword score, and character feature score of the core domain name are calculated, and the domain value score is obtained by weighted summing the dictionary score, character length score, keyword score, and character feature score.

[0057] The domain value score can be obtained through the following formula:

[0058] Domain Value Rating =

[0059] a×dictionary score+b×length score+c×hot keyword score+d×character feature score;

[0060] Among them, a, b, c, and d can be corresponding hyperparameters, which can be set manually and can be continuously adjusted to achieve the evaluation of the value of the domain name.

[0061] In one embodiment of the present application, the dictionary score, length score, hot keyword score and character feature score of the core domain name are calculated respectively, including:

[0062] Determine the character length of the core domain name, and score the character length of the core domain name according to a preset length scoring rule to obtain a character length score;

[0063] Compare the core domain name with the high-value dictionary and determine the dictionary score based on the comparison results according to the preset dictionary scoring rules;

[0064] Compare the hot keywords of the core domain name with the preset hot keywords, and determine the hot keyword score based on the comparison results according to the preset keyword scoring rules;

[0065] According to the preset character feature scoring rules, the characters of the core domain name are scored to obtain a character feature score.

[0066] It should be noted that when scoring the length of characters, the shorter the length, the higher the score. Optionally, a character length and score mapping table can be established, for example, the score corresponding to 5 characters is 10 points, the score corresponding to 8 characters is 7 points, etc. After obtaining the character length, the character length score can be obtained by querying the mapping table. For hot keywords, they are scored according to their popularity. If the popularity is high, the score is high, and if the popularity is low, the score is small. For example, a mapping table of hot keywords and scores can also be established. After comparing the current hot keywords with the pre-built hot keywords, the scores corresponding to the hot keywords with consistent comparisons are determined through the mapping table as the hot keyword scores. Similarly, for dictionary scoring, it can be scored according to the value of its core domain name. The higher the value, the higher the score. For example, a mapping table of high-value domain names and scores can be established. After comparing the core domain name with the high-value dictionary, the scores corresponding to the high-value domain names with consistent comparisons are determined through the mapping table as the dictionary score. For character feature scoring, words with clear meanings have higher values, while domain names containing too many special characters have lower values. Full-letter domain names such as worldcoin have higher scores than ajsfga.

[0067] In one embodiment of the present application, the deep features of the visited URLs are compared with the deep features of the risky URLs to obtain a comparison similarity feature, and the deep features of the visited URLs and the comparison similarity features are used as target deep features, including:

[0068] Acquire risky websites, and extract features from risky websites using a deep feature extraction strategy to obtain deep features of risky websites;

[0069] Calculate the similarity value between the deep features of the visited URL and the deep features of the risky URL by using a preset similarity algorithm;

[0070] Based on the similarity values, determining a comparative similarity feature;

[0071] The comparative similarity features are combined with the deep features of the visited URLs to serve as the target deep features.

[0072] The target deep features include at least one of HTML features, JS features, CSS features, multimedia resource file features, and overall risk item features.

[0073] Alternatively, see Figure 5, risky URLs can be URLs collected from authoritative channels and marked as risky URLs. For example, network operator platforms, network supervision platforms, etc. The risky URLs are feature extracted through deep feature extraction strategies to obtain deep features of risky URLs. Specifically, the complete information of the risky network can be downloaded, which may specifically include HTML files, JS files, CSS files, multimedia resource files, and their website links and file structures, such as directory structures. It should be noted that for HTML, JS, and CSS files, since they are the core functional files of the website, a core file recognition algorithm can be used to extract key core files. Among them, the core file recognition algorithm mainly uses methods such as file name keywords, key file locations, file sizes, and general framework file recognition for comprehensive judgment. Then, based on the HTML file, JS file, CSS file, multimedia resource file, and its website link and file structure, feature extraction is performed according to the corresponding feature extraction method to obtain DOM structure features, HTML reference file features, HTML block text features, HTML function features, HTML code style features, JS file name features, function method name features, JS code style features, JS reference library features, JS function features, CSS file name features, CSS selector features, CSS animation features, CSS font features, CSS color features, resource file structure features, image attribute features, resource file MD5 value features, image content features, image quality features, code encryption features, data encryption features, website text features, grammatical errors, risk field features, and similarity features with the black industry intelligence database, etc. The deep features of all risky websites are constructed into a black industry intelligence database.

[0074] Then, the similarity between the deep features of the risky URL and the deep features of the visited URL can be calculated by using a preset similarity algorithm, such as a cosine similarity algorithm or a Euclidean similarity algorithm. Taking the cosine similarity algorithm as an example, the similarity can be calculated by the following formula:

[0075]

[0076] Among them A i is the i-th element of vector A, B i is the i-th element of vector B.

[0077] For example, the html file content of website A is AAAA; the corresponding html file content of website B of the black market intelligence database is AABB; then the similarity feature of Html is cosine similarity (AAAA, AABB), and the corresponding comparison similarity feature can be a number, such as 0.96.

[0078] It should be noted that a similarity can be calculated for each deep feature of the file type, such as HTML similarity, JS similarity, CSS similarity, etc., which can be weighted and summed as the final comparative similarity feature. Alternatively, the comparative similarity feature of a URL can include multiple comparative similarities such as HTML similarity, JS similarity, CSS similarity, etc.

[0079] In one embodiment of the present application, feature extraction is performed on the accessed website information through a deep feature extraction strategy to obtain deep features of the accessed website, including:

[0080] Obtaining complete URL information corresponding to the access URL information, the complete URL information includes core files, multimedia resource files and directory structure, wherein the core files are extracted by a preset core file identification algorithm;

[0081] Deep feature extraction is performed on core files, multimedia resource files and directory structures respectively to obtain the deep features of the visited URLs.

[0082] See also Figure 4 , provides a flow chart of the complete URL information download process, and performs automated data download for the collected URL information. That is, the complete URL information can be obtained by simulating a request for the URL, then intercepting the information obtained by the browser and saving it locally. It can specifically include HTML files, JS files, CSS files, multimedia resource files, and their website links and file structures, such as directory structures. And classify them. For HTML, JS, and CSS files, since they are the core functional files of the website, the core file recognition algorithm can be used to extract the key core files. Among them, the core file recognition algorithm mainly uses methods such as file name keywords, key file locations, file sizes, and general framework file recognition for comprehensive judgment.

[0083] It should be noted that the core file identification algorithm specifically refers to determining whether the file is important and whether it is a core file by setting up a set of identification processes. For example, taking JS files as an example, the file name keyword score, key file location score, file size score, general framework file score and function score can be calculated separately, and then weighted summation is performed to obtain the final score. If the final score is greater than the preset threshold, it can be considered a core file.

[0084] Exemplarily, for file name keywords: if the file name contains common core js file words such as app, index, home, etc., it is more valuable and can be given extra points; key file location: if it is a js file in the main directory, extra points will be given, if not, 0 points will be given; file size: if the file size is less than 1kb, 0 points will be given, and if it is larger than 1kb, extra points will be given; general framework file identification: if the js is found to be a general framework js file, such as the js file package of jquery, then it will be discarded and considered unimportant; split the js file, analyze the number of functions in it, the length of each function, the maximum depth of the function, the single-loop complexity of the function, the number of hard-coded functions, the number of routes and other function parameters, and weight each function to calculate its score.

[0085] See also Figure 5 , for the extracted complete URL information, deep feature extraction can be performed. For example, the feature extraction of the HTML file can specifically include: DOM structure extraction, HTML block (header, body, footer, etc.) text feature extraction, HTML function feature extraction, HTML code style feature extraction, etc., to obtain HTML features. For the feature extraction of JS files, it can specifically include: JS file name feature extraction, function method name feature extraction, JS code style feature extraction, JS reference library feature extraction, JS function feature extraction, etc.; for CSS files, CSS file name features, CSS selector features, CSS animation features, CSS font features, CSS color features, etc. are obtained to obtain JS features. For the feature extraction of multimedia resource files, it can specifically include: resource file structure feature extraction, image attribute feature extraction, resource file MD5 value feature extraction, image content feature extraction, image quality feature extraction, etc., to obtain multimedia resource file features. For the feature extraction of overall risk items, it can specifically include code encryption features, data encryption features, website text features, grammatical errors features, risk field features, etc.

[0086] In one embodiment of the present application, based on the shallow features and deep features of the target, the preset risk website identification model is iteratively trained, including:

[0087] Step a: vectorize the shallow features and deep features of the target respectively, and construct a feature database, which includes multiple groups of multi-dimensional feature vector groups;

[0088] Step b: randomly select a group of multi-dimensional feature vectors and input them into a preset risk website identification model for prediction to obtain prediction results;

[0089] Step c: Calculate the loss value based on the prediction results and risk labels;

[0090] Step d: When the loss value is greater than a preset threshold, another set of multidimensional feature vector groups is selected from the remaining multidimensional feature vector groups, and is input into a preset risk website identification model for prediction to obtain a prediction result;

[0091] Repeat the above steps bd until the preset convergence condition is met and the iteration stops.

[0092] Optionally, the extracted shallow features are fused with the deep features, and a feature database is constructed, which can be connected to a preset risk identification model so as to iteratively train the preset risk identification model. Specifically, a group of multidimensional feature vectors can be randomly selected from the feature database and input into the preset risk identification model for prediction. The prediction result is the risk level. Based on the risk level, the back propagation algorithm and the error between the predicted risk level and the risk label can be used to calculate the loss function value through a preset loss function, such as a cross entropy function. The parameters of the preset risk identification model can be adjusted based on the loss function value, and a group of multidimensional feature vectors can be selected from the remaining multidimensional feature vectors again, and input into the preset risk identification model for prediction. Repeat the above steps until the number of iterations reaches the maximum number of iterations, such as 1000 times, or when the loss function value is less than the threshold loss value, the iteration is completed, and a trained risk identification model can be obtained.

[0093] The risk identification model can be built based on the Deep Neural Network (DNN), which, as the core architecture of deep learning, has demonstrated excellent capabilities in processing complex data and identifying complex patterns. DNN consists of multiple layers of neurons, each layer of neurons extracts features from the previous layer through nonlinear transformation, thereby capturing the complex structure of the data layer by layer. In website risk identification, DNN can automatically identify high-risk features from web page content, user behavior and image data through hierarchical feature extraction.

[0094] See also Figure 6 , the model may include an input layer, multiple hidden layers and an output layer. Taking 4 hidden layers as an example, it may include the first hidden layer hidden layer1 with 32 nodes, the second hidden layer hidden layer2 with 64 nodes, the hidden layer hidden layer3 with 32 nodes and the hidden layer hidden layer4 with 16 nodes. Among them, activation functions such as sigmoid function can be used between each layer for activation processing, as shown below:

[0095]

[0096] Among them, x is the input value, and S(x) is the result after nonlinear transformation.

[0097] It should be noted that the back propagation algorithm can be used in this model, and the loss function corresponding to the error calculation is the cross entropy function. The specific formula is as follows:

[0098]

[0099] Among them, the true distribution is y i , the network output distribution is The total number of categories is n.

[0100] The hidden layer 4 of 16 nodes will eventually pass through the Softmax function to obtain the final output layer, where the function expression of Softmax is:

[0101]

[0102] Among them, z i is the output value of the ith node, and C is the number of output nodes, that is, the number of categories:

[0103] The final output result of the output layer is the risk level, such as high-risk website, medium-risk website, low-risk website, and the corresponding output result values ​​are 0, 1, and 2.

[0104] In the embodiment of the present application, by introducing a multi-level high-risk feature extraction mechanism, deep features and shallow features are extracted to more efficiently screen out riskier websites. At the same time, multi-dimensional high-risk features are automatically extracted and identified from website content, user behavior and image data, and through deep neural network training, the correlation analysis between different risk features is enhanced to ensure that the system can accurately and timely detect and respond to potential online fraud.

[0105] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] In one embodiment, a risk website identification device based on multi-level high-risk features is provided, and the risk website identification device based on multi-level high-risk features corresponds to the risk website identification method based on multi-level high-risk features in the above embodiment. Figure 7 As shown, the risk website identification device based on multi-level high-risk features includes an access website information collection unit 10, a target shallow feature extraction unit 20, a target deep feature extraction unit 30, a model training unit 40 and a prediction unit 50. The functional modules are described in detail as follows:

[0107] The access website information collection unit 10 is used to collect the access website information, and the access website information is provided with a risk tag;

[0108] A target shallow feature extraction unit 20 is used to extract features from the accessed website information through a shallow feature extraction strategy to obtain target shallow features;

[0109] The target deep feature extraction unit 30 is used to extract features from the accessed website information through the deep feature extraction strategy to obtain the deep features of the accessed website, compare the deep features of the accessed website with the deep features of the risky website to obtain the comparison similarity features, and use the deep features of the accessed website and the comparison similarity features as the target deep features;

[0110] The model training unit 40 is used to iteratively train the preset risk website identification model based on the target shallow features and the target deep features until the preset convergence conditions are met, thereby obtaining a trained risk website identification model;

[0111] The prediction unit 50 is used to predict the to-be-identified access website based on the trained risk website identification model to obtain the risk level of the to-be-identified access website.

[0112] In one embodiment of the present application, the target shallow features include at least one of domain name value information, website whois information, website length, and website access information.

[0113] In one embodiment of the present application, the shallow features include domain value information, and the target shallow feature extraction unit 20 includes:

[0114] Extract domain name information based on URL information;

[0115] Perform word segmentation on domain name information and extract core domain names;

[0116] Calculate the dictionary score, character length score, hot keyword score, and character feature score of the core domain name respectively;

[0117] The dictionary score, character length score, keyword score and character feature score are weighted and summed to obtain the domain name value score.

[0118] In one embodiment of the present application, the target shallow feature extraction unit 20 includes:

[0119] Determine the character length of the core domain name, and score the character length of the core domain name according to a preset length scoring rule to obtain a character length score;

[0120] Compare the core domain name with the high-value dictionary and determine the dictionary score based on the comparison results according to the preset dictionary scoring rules;

[0121] Compare the hot keywords of the core domain name with the preset hot keywords, and determine the hot keyword score based on the comparison results according to the preset keyword scoring rules;

[0122] According to the preset character feature scoring rules, the characters of the core domain name are scored to obtain a character feature score.

[0123] In one embodiment of the present application, the target deep feature extraction unit 30 is further used to:

[0124] Acquire risky websites, and extract features from risky websites using a deep feature extraction strategy to obtain deep features of risky websites;

[0125] Calculate the similarity value between the deep features of the visited URL and the deep features of the risky URL by using a preset similarity algorithm;

[0126] Based on the similarity values, determining a comparative similarity feature;

[0127] The comparative similarity features are combined with the deep features of the visited URLs to serve as the target deep features.

[0128] In one embodiment of the present application, the target deep feature extraction unit 30 is further used to:

[0129] Obtaining complete URL information corresponding to the access URL information, the complete URL information includes core files, multimedia resource files and directory structure, wherein the core files are extracted by a preset core file identification algorithm;

[0130] Deep feature extraction is performed on core files, multimedia resource files and directory structures respectively to obtain the deep features of the visited URLs.

[0131] In one embodiment of the present application, the target deep features include at least one of HTML features, JS features, CSS features, multimedia resource file features, and overall risk item features.

[0132] In one embodiment of the present application, the model training model 40 is used to:

[0133] Step a: vectorize the shallow features and deep features of the target respectively, and construct a feature database, which includes multiple groups of multi-dimensional feature vector groups;

[0134] Step b: randomly select a group of multi-dimensional feature vectors and input them into a preset risk website identification model for prediction to obtain prediction results;

[0135] Step c: Calculate the loss value based on the prediction results and risk labels;

[0136] Step d: When the loss value is greater than a preset threshold, another set of multidimensional feature vector groups is selected from the remaining multidimensional feature vector groups, and is input into a preset risk website identification model for prediction to obtain a prediction result;

[0137] Repeat the above steps bd until the preset convergence condition is met and the iteration stops.

[0138] In the embodiment of the present application, by introducing a multi-level high-risk feature extraction mechanism, deep features and shallow features are extracted to more efficiently screen out riskier websites. At the same time, multi-dimensional high-risk features are automatically extracted and identified from website content, user behavior and image data, and through deep neural network training, the correlation analysis between different risk features is enhanced to ensure that the system can accurately and timely detect and respond to potential online fraud.

[0139] For the specific definition of the risk website identification device based on multi-level high-risk features, please refer to the definition of the risk website identification method based on multi-level high-risk features above, which will not be repeated here. Each module in the above-mentioned risk website identification device based on multi-level high-risk features can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0140] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, a method for identifying risky websites based on multi-level high-risk features is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0141] In an embodiment of the present application, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the risk website identification method based on multi-level high-risk features as described above are implemented.

[0142] In an embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the steps of the general parking lot group modeling method as described above are implemented.

[0143] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0145] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for identifying risky websites based on multi-level high-risk features, characterized in that: The method comprises: Collecting access website information, wherein the access website information is provided with a risk label; Extracting features from the accessed URL information using a shallow feature extraction strategy to obtain target shallow features; Extracting features from the accessed website information through a deep feature extraction strategy to obtain deep features of the accessed website, comparing the deep features of the accessed website with the deep features of the risky website to obtain a comparison similarity feature, and using the deep features of the accessed website and the comparison similarity feature as target deep features; Based on the shallow features of the target and the deep features of the target, iteratively train the preset risk website identification model until a preset convergence condition is met, thereby obtaining a trained risk website identification model; The to-be-identified access website is predicted based on the trained risk website identification model to obtain the risk level of the to-be-identified access website.

2. The risk website identification method based on multi-level high-risk features as claimed in claim 1, characterized in that: The target shallow feature includes at least one of domain name value information, website whois information, website length, and website access information.

3. The risk website identification method based on multi-level high-risk features according to claim 1 or 2, characterized in that: The shallow features include domain value information. The shallow feature extraction strategy is used to extract features from the access URL information to obtain target shallow features, including: Based on the website information, extracting domain name information; Perform word segmentation on the domain name information and extract the core domain name; Calculate the dictionary score, character length score, hot keyword score and character feature score of the core domain name respectively; The dictionary score, the character length score, the keyword score and the character feature score are weighted and summed respectively to obtain a domain name value score.

4. The risk website identification method based on multi-level high-risk features as claimed in claim 3 is characterized in that: The respectively calculating the dictionary score, length score, hot keyword score and character feature score of the core domain name includes: Determining the character length of the core domain name, and scoring the character length of the core domain name according to a preset length scoring rule to obtain the character length score; Comparing the core domain name with the high-value dictionary, and determining the dictionary score based on the comparison result according to the preset dictionary scoring rules; Comparing the hot keywords of the core domain name with the preset hot keywords, and determining the score of the hot keywords based on the comparison result according to the preset keyword scoring rule; According to a preset character feature scoring rule, the characters of the core domain name are scored to obtain the character feature score.

5. The risk website identification method based on multi-level high-risk features as claimed in claim 1, characterized in that: The step of comparing the deep features of the visited URL with the deep features of the risky URL to obtain a comparison similarity feature, and using the deep features of the visited URL and the comparison similarity feature as target deep features includes: Acquire risky websites, and extract features from the risky websites using the deep feature extraction strategy to obtain deep features of the risky websites; Calculate the similarity value between the deep features of the visited website and the deep features of the risky website by using a preset similarity algorithm; Based on the similarity value, determining the comparison similarity feature; The comparison similarity feature is combined with the deep feature of the visited URL to serve as the target deep feature.

6. The risk website identification method based on multi-level high-risk features according to claim 1 or 5, characterized in that: The step of extracting features from the accessed website information by using a deep feature extraction strategy to obtain deep features of the accessed website includes: Obtaining complete website information corresponding to the access website information, wherein the complete website information includes core files, multimedia resource files, and directory structures, wherein the core files are extracted by a preset core file identification algorithm; Deep feature extraction is performed on the core file, the multimedia resource file and the directory structure respectively to obtain the deep features of the accessed website.

7. The risk website identification method based on multi-level high-risk features according to claim 1 or 5, characterized in that: The target deep features include at least one of HTML features, JS features, CSS features, multimedia resource file features, and overall risk item features.

8. The risk website identification method based on multi-level high-risk features as claimed in claim 1, characterized in that: The iterative training of the preset risk website identification model based on the shallow features of the target and the deep features of the target includes: Step a: vectorizing the shallow features and deep features of the target respectively, and constructing a feature database, wherein the feature database includes multiple groups of multi-dimensional feature vector groups; Step b: randomly selecting a group of multi-dimensional feature vectors and inputting them into the preset risk website identification model for prediction to obtain prediction results; Step c: Calculating the loss value based on the prediction result and the risk label; Step d: when the loss value is greater than a preset threshold, another set of multidimensional feature vector groups is selected from the remaining multidimensional feature vector groups, and is input into the preset risk website identification model for prediction to obtain a prediction result; Repeat steps b to d until the preset convergence condition is met, and then the iteration stops.

9. A risk website identification device based on multi-level high-risk features, characterized in that: The system comprises: A visited website information collection unit, used to collect visited website information, wherein the visited website information is provided with a risk tag; A target shallow feature extraction unit, used for extracting features from the accessed website information through a shallow feature extraction strategy to obtain target shallow features; A target deep feature extraction unit is used to extract features from the accessed website information through a deep feature extraction strategy to obtain deep features of the accessed website, compare the deep features of the accessed website with the deep features of the risky website to obtain a comparison similarity feature, and use the deep features of the accessed website and the comparison similarity feature as target deep features; A model training unit, used for iteratively training a preset risk website identification model based on the target shallow features and the target deep features until a preset convergence condition is met, thereby obtaining a trained risk website identification model; The prediction unit is used to predict the to-be-identified access website based on the trained risk website identification model to obtain the risk level of the to-be-identified access website.

10. A readable storage medium having computer readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the risk website identification method based on multi-level high-risk features as described in any one of claims 1 to 7 is implemented.

Images