Fraudulent website detection method, device, equipment and storage medium
By obtaining and analyzing the newly added domain name log information in the user's Internet log, using the classification and analysis model of fraud-related websites and K-mean clustering analysis, the problem of rapid update of domain names and short survival cycles in the existing technology is solved, and the fraud-related problem is achieved Accurate analysis and backtracking analysis of websites improve network security.
Patent Information
- Application Number
- CN202311540726.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-11-20
AI Technical Summary
When detecting fraud-related websites, the prior art cannot effectively deal with the problem of rapid update of domain names and short survival cycles, and there is insufficient accuracy in relying on domain name registrar information.
By obtaining the new domain name logs in the user's Internet log information, extracting the relative link feature vector and inputting the classification and analysis model of fraud-related websites, combining K-mean clustering analysis and whitelist matching, automatic analysis and backtracking analysis of fraud-related websites is achieved.
Accurate analysis and judgment of fraud-related websites, including changes in website content, keeping the template unchanged and frequent changes in web text, improving network security guarantees.
Smart Images

Figure CN117254983B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network security technology, and in particular to a method, device, equipment and storage medium for detecting fraudulent websites. Background Art
[0002] In the prior art, fraudulent URLs are detected through detection methods based on website URL features and web page content features. However, URL comparison relies on a known blacklist URL address library and cannot adapt to scenarios where fraudulent domain names are quickly updated. Detection methods based on HTML page rendering screenshot similarity and content feature comparison usually rely on web crawling and cannot analyze fraudulent domain names with short survival cycles. Fraudulent URLs are detected through detection methods based on domain name registration information, but there is a problem of relying on domain name registrar information. Currently, fraud suspects purchase domain names from third-party domain name registration agents, resulting in poor analysis accuracy. Summary of the invention
[0003] The present invention provides a method, device, equipment and storage medium for detecting fraudulent URLs, aiming to realize automatic judgment on whether a newly added domain name in a user's online log information is a fraudulent URL, so as to accurately and effectively judge fraudulent URLs in various situations such as the content of the website homepage changes but the website template remains unchanged, the website has been shut down, the text content of the webpage changes frequently, etc.
[0004] The present invention provides a method for detecting fraudulent websites, comprising:
[0005] Obtain log information of several newly added domain names within the preset detection time;
[0006] Based on any of the newly added domain name log information, obtaining a relative link feature vector corresponding to the newly added domain name log information;
[0007] The relative link feature vector is input into a fraudulent website classification and analysis model to obtain a fraudulent analysis result output by the fraudulent website classification and analysis model to determine the fraudulent status of the newly added domain name log information.
[0008] According to a fraudulent website detection method provided by the present invention, after inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0009] Obtaining a preset number of clusters, a list of fraudulent websites, and a list of incremental domain name logs within a preset backtracking time, wherein the preset number of clusters is the K value in the K-means clustering algorithm;
[0010] Based on the preset number of clusters, the list of fraud-related web addresses, and the incremental domain name log list, a K-means cluster analysis is performed to obtain K incremental clustering results and K fraud-related clustering results;
[0011] Each of the incremental clustering results and each of the fraud-related clustering results are pushed to security experts, so that the security experts can classify the list of fraud-related websites with labels and use them as training samples to train the classification and analysis model for fraud-related websites.
[0012] According to a fraud-related website detection method provided by the present invention, cluster analysis is performed based on the preset number of categories, the fraud-related website list and the incremental domain name log list to obtain K incremental clustering results and K fraud-related clustering results, including:
[0013] Based on the fraud-related website list and the incremental domain name log list, a number of incremental relative links and a number of fraud-related relative links are obtained;
[0014] Matching each of the incremental relative links and each of the fraud-related relative links with a preset whitelist relative link set;
[0015] Based on the preset number of clusters, K-means clustering analysis is performed on each of the matched incremental relative links and each of the matched fraud-related relative links to obtain the K incremental clustering results and the K fraud-related clustering results.
[0016] According to a fraudulent website detection method provided by the present invention, the step of obtaining a relative link feature vector corresponding to any of the newly added domain name log information based on the newly added domain name log information includes:
[0017] Extracting a plurality of newly added relative link features from any of the newly added domain name log information;
[0018] Each of the newly added relative link features is compared and matched with a preset fraud-related relative link feature set to generate the relative link feature vector.
[0019] According to a fraudulent website detection method provided by the present invention, the step of obtaining log information of a plurality of newly added domain names within a preset detection time includes:
[0020] Obtaining user online log information within a preset detection time, and extracting a host field in the user online log information;
[0021] Matching the host field with a preset whitelist URL library to filter whitelist domain name log information in the user's Internet access log information;
[0022] The host field obtained after filtering is compared and matched with a preset stock domain name library to obtain the plurality of newly added domain name log information.
[0023] According to a fraudulent website detection method provided by the present invention, before inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0024] Obtaining several groups of fraudulent website samples and training sample labels corresponding to the several groups of fraudulent website samples;
[0025] According to each of the fraud-related website samples and the training sample labels corresponding to each of the fraud-related website samples, the initial fraud-related website classification and analysis model is iteratively trained to obtain the fraud-related website classification and analysis model.
[0026] According to a fraudulent website detection method provided by the present invention, after inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0027] If the fraud-related analysis result corresponding to the newly added domain name log information is a fraud-related website result, the newly added domain name log information is matched with a preset malicious website library;
[0028] If the match fails, generating fraudulent domain name information based on the newly added domain name log information, and adding the fraudulent domain name information to the preset malicious URL library;
[0029] The fraud-related domain name information is pushed to a management personnel so that the management personnel can manage the newly added domain name log information based on the fraud-related domain name information.
[0030] The present invention also provides a fraudulent website detection device, comprising:
[0031] The acquisition module is used to obtain log information of several newly added domain names within a preset detection time;
[0032] A feature module, used to obtain a relative link feature vector corresponding to any of the newly added domain name log information based on the newly added domain name log information;
[0033] The analysis module is used to input the relative link feature vector into the fraud-related website classification analysis model to obtain the fraud-related analysis result output by the fraud-related website classification analysis model to determine the fraud-related situation of the newly added domain name log information.
[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting fraudulent websites as described above is implemented.
[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for detecting fraudulent websites.
[0036] The fraudulent website detection method, device, equipment and storage medium provided by the present invention obtain a plurality of newly added domain name log information within a preset detection time, and then obtain a relative link feature vector corresponding to the newly added domain name log information based on any of the newly added domain name log information, thereby inputting the relative link feature vector into a fraudulent website classification and judgment model, obtaining a fraudulent judgment result output by the fraudulent website classification and judgment model, and then determining the fraudulent situation of the newly added domain name log information, thereby realizing the judgment of whether the newly added domain name in the user's Internet log information is a fraudulent website, and can accurately and effectively judge the fraudulent website in various situations such as the website homepage content changes but the website template remains unchanged, the website has been shut down, the webpage text content changes frequently, etc., and at the same time, it also realizes the functions of backtracking analysis of fraudulent website and early warning of fraudulent website, effectively ensuring network security. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 It is a flow chart of the fraudulent website detection method provided by the present invention;
[0039] Figure 2 It is a structural schematic diagram of the fraudulent website detection device provided by the present invention;
[0040] Figure 3 is a schematic diagram of the structure of an electronic device provided by the present invention;
[0041] Figure 4 It is a system module diagram of the network anti-fraud big data analysis platform provided by the present invention;
[0042] Figure 5 It is a partial function implementation diagram of the network anti-fraud big data analysis platform provided by the present invention. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0044] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.
[0045] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when...".
[0046] Figure 1 FIG. 1 is a flow chart of the method for detecting fraudulent websites provided by the present invention. Figure 1 As shown, the fraudulent website detection method includes:
[0047] Step S11, obtaining log information of a number of newly added domain names within a preset detection time;
[0048] It should be noted that the preset detection time refers to the pre-set detection time, for example, 10 minutes, 24 hours and one week, etc., which can be set according to the actual situation and is not set here. In addition, the newly added domain name log information refers to the domain name log information newly added to the user's Internet log within the preset detection time, including domain name, host field, user information, timestamp, request method (HTTP request method) and request path (URL path) and other information. In addition, a statistical report on newly added domain name log information is also provided each time the newly added domain name log information is obtained, including incremental domain name, date, number of visiting users, number of access log entries, relative link secondary distribution (relative connection, number of access log entries) and other information.
[0049] Specifically, the user's Internet access log information within a preset detection time is obtained, and the host field in the user's Internet access log information is extracted, and then the host field is matched with a preset whitelist URL library to filter the whitelist domain name log information in the user's Internet access log information, and then the host field obtained after filtering is compared and matched with the preset existing domain name library to obtain the several newly added domain name log information.
[0050] Step S12, based on any of the newly added domain name log information, obtaining a relative link feature vector corresponding to the newly added domain name log information;
[0051] It should be noted that the relative link feature vector refers to a feature vector obtained between the relative link features extracted from the newly added domain name log information and the relative link features extracted from the fraud-related domain name log information, wherein the relative link feature can be in the form of " / client_aps", etc., which is not limited here.
[0052] Specifically, a plurality of newly added relative link features of any of the newly added domain name log information are extracted, and then each of the newly added relative link features is compared and matched with a preset fraud-related relative link feature set, thereby generating a relative link feature vector.
[0053] Step S13, inputting the relative link feature vector into a fraudulent website classification and analysis model, obtaining a fraudulent analysis result output by the fraudulent website classification and analysis model, so as to determine the fraudulent status of the newly added domain name log information.
[0054] It should be noted that the classification and judgment model of the fraudulent website is obtained by multiple iterative training based on several groups of fraudulent website samples and several groups of training sample labels corresponding to the fraudulent website samples. The fraudulent judgment results include normal website results and fraudulent website results. In addition, a statistical report of the fraudulent domain names of the fraudulent website obtained after each judgment is also provided, including information such as the fraudulent website, the type of fraudulent website, the date, the number of visiting users, and the number of access logs.
[0055] Specifically, the relative link feature vector is input into a fraudulent website classification and analysis model to obtain a fraudulent analysis result output by the fraudulent website classification and analysis model, so that the fraudulent status of the newly added domain name log information can be determined based on the fraudulent analysis result.
[0056] The embodiment of the present invention obtains several newly added domain name log information within a preset detection time, and then obtains the relative link feature vector corresponding to the newly added domain name log information based on any of the newly added domain name log information, thereby inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain the fraudulent judgment result output by the fraudulent website classification and judgment model, and then determines the fraudulent situation of the newly added domain name log information, thereby realizing the judgment of whether the newly added domain name in the user's Internet log information is a fraudulent website, and can accurately and effectively judge the fraudulent website in various situations such as the website homepage content changes but the website template remains unchanged, the website has been shut down, the web page text content changes frequently, etc., and at the same time, it also realizes the functions of backtracking analysis of fraudulent website and early warning of fraudulent website, effectively ensuring network security.
[0057] In one embodiment of the present invention, after inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0058] Obtain a preset number of clusters, a list of fraudulent web addresses, and an incremental domain name log list within a preset backtracking time, wherein the preset number of clusters is the K value in the K-means clustering algorithm; perform K-means clustering analysis based on the preset number of clusters, the list of fraudulent web addresses, and the incremental domain name log list to obtain K incremental clustering results and K fraudulent clustering results; push each of the incremental clustering results and each of the fraudulent clustering results to a security expert, so that the security expert can use them as training samples to train the fraudulent web address classification and analysis model after labeling the list of fraudulent web addresses.
[0059] It should be noted that the preset number of clusters is the K value in the K-means clustering algorithm. For example, when using the K-means clustering algorithm to classify data, the K value represents the number of categories to be divided into, that is, the preset number of clusters. If K is set to 5, then the K-means clustering algorithm will assign all data to 5 different clusters. The preset number of clusters can be set according to actual conditions and is not limited here.
[0060] It should be further explained that the preset backtracking time refers to the preset backtracking analysis time, including the backtracking start time and the backtracking end time. The backtracking start time and the backtracking end time can be set according to the actual situation and are not set here. The incremental domain name log list refers to the domain name log information list of all newly added domain names within the preset backtracking time, including information such as incremental domain names and incremental relative links. The list of fraudulent URLs refers to a list of various types of fraudulent URLs that need to be backtracked and analyzed, including information such as fraud type, fraudulent domain names, number of log records, and fraudulent relative links. In addition, the incremental clustering results include grouping labels, the number of incremental domain names, incremental domain name sets, and incremental domain name feature sets, etc. The fraud-related clustering results include grouping labels, the number of fraud-related domain names, the fraud-related domain name sets, and the fraud-related domain name feature sets, etc.
[0061] Specifically, a preset number of categories and a list of fraudulent web addresses are obtained, and an incremental domain name log list within a preset backtracking time is obtained, and then based on the list of fraudulent web addresses and the incremental domain name log list, a number of incremental relative links and a number of fraudulent relative links are obtained, so as to match each of the incremental relative links and each of the fraudulent relative links with a preset whitelist relative link set, and then based on the preset number of categories, K-means clustering analysis is performed on each of the matched incremental relative links and each of the matched fraudulent relative links, so as to obtain K incremental clustering results and K fraudulent clustering results.
[0062] Furthermore, each of the incremental clustering results and each of the fraud-related clustering results are pushed to security experts, so that after analyzing the incremental clustering results and the fraud-related clustering results, the security experts manually label and classify the fraud-related URLs in each type of fraud-related clustering results and then merge them to obtain a set of fraud-related relative link features of all fraud-related URLs in the fraud-related URL list, and then use the set of fraud-related relative link features as training samples to train the fraud-related URL classification and analysis model.
[0063] For example, if the incremental domain name collection in one category of incremental clustering results is "qlwunef.cn, okcti.com.cn", the incremental domain name feature set is " / favicon.ico, / "; if the fraud-related domain name collection in one category of fraud-related clustering results of counterfeit phishing (purchase of game props) is "www3.lbeuhis0wa.space, m1mmlmm.y6q2egm.buzz", the fraud-related domain name feature set is " / 51sale.asp, / ajax_server3.asp, / getcode.asp, / favicon.ico, / reg_ok.asp". It can be seen that both results contain the relative link feature " / favicon.ico", but the relative link feature is obviously a common feature of the website. The security expert can then remove the relative link feature from the fraud-related relative link feature set, and obtain a list of fraud-related URLs corresponding to such fraud-related clustering results as "www3.lbeuhis0wa.space, m1mmlmm.y6 q2egm.buzz", and a set of fraud-related relative link features as " / 51sale.asp, / ajax_server3.asp, / getcode.asp, / reg_ok.asp". Based on the fraud-related relative link feature set and the list of fraud-related URLs, the fraud-related websites corresponding to such fraud-related clustering results are manually labeled and classified as "fraudURL.fishing.modelA".
[0064] The embodiment of the present invention obtains a preset number of categories, a list of fraudulent web addresses, and an incremental domain name log list within a preset backtracking time, and then performs K-means clustering analysis based on the preset number of categories, the list of fraudulent web addresses, and the incremental domain name log list to obtain K incremental clustering results and K fraud-related clustering results, thereby pushing each of the incremental clustering results and each of the fraud-related clustering results to a security expert for the security expert to label the list of fraudulent web addresses, and then providing training samples such as fraudulent web address classification labels and fraudulent relative link feature sets for the training of a fraudulent web address classification and judgment model, thereby effectively reducing the workload of classification labeling and feature selection for model training, and effectively improving the robustness of the model and the accuracy of the model output results.
[0065] In one embodiment of the present invention, cluster analysis is performed based on the preset number of clusters, the fraud-related website list, and the incremental domain name log list to obtain K incremental clustering results and K fraud-related clustering results, including:
[0066] Based on the fraud-related URL list and the incremental domain name log list, a number of incremental relative links and a number of fraud-related relative links are obtained; each of the incremental relative links and each of the fraud-related relative links is matched with a preset whitelist relative link set; based on the preset number of clusters, K-means clustering analysis is performed on each of the matched incremental relative links and each of the matched fraud-related relative links to obtain the K incremental clustering results and the K fraud-related clustering results.
[0067] It should be noted that the incremental relative link refers to the relative link corresponding to an incremental domain name in the incremental domain name log list, and the fraud-related relative link refers to the relative link corresponding to a fraud-related domain name in the fraud-related URL list. The relative link is a link method used in Hypertext Markup Language (HTML) or other markup languages, such as " / about.html", " / another-page.html", etc.
[0068] It should be further explained that the preset whitelist relative link set refers to a set of relative links that most URLs have, such as the relative link " / index.html" of the homepage.
[0069] Specifically, the fraud-related domain name log information corresponding to each fraud-related web address in the fraud-related web address list is extracted from the incremental domain name log list, and then the fraud-related relative links in the fraud-related domain name log information and the incremental relative links in the incremental domain name log list are respectively extracted. In order to prevent the common relative link features common to some web addresses from interfering with the accuracy of the classification and judgment model of fraud-related web addresses, and to accurately screen out the relative link features corresponding to each web address, each of the incremental relative links and each of the fraud-related relative links are matched with a preset whitelist relative link set. For example, please refer to Table 1 and Table 2. Table 1 is a list of fraud-related domain name relative links after extracting fraud-related relative links from the fraud-related domain name log information in one embodiment of the present invention. Table 2 is a list of incremental domain name relative links after extracting incremental relative links from the incremental domain name log list in one embodiment of the present invention.
[0070] Furthermore, according to the preset number of clusters, K-means clustering analysis is performed on the matched incremental relative links and the fraud-related relative links, respectively, to obtain K incremental clustering results and K fraud-related clustering results. Please refer to Tables 3 and 4. Table 3 is the incremental clustering result obtained for Table 2 in one embodiment of the present invention.
[0071] Table 4 is the fraud-related clustering result obtained for Table 1 in one embodiment of the present invention.
[0072] The embodiment of the present invention obtains a plurality of incremental relative links and a plurality of fraud-related relative links based on the fraud-related website list and the incremental domain name log list, and then matches each of the incremental relative links and each of the fraud-related relative links with a preset whitelist relative link set, and based on the preset number of clusters, performs K-means clustering analysis on each of the matched incremental relative links and each of the matched fraud-related relative links, respectively, to obtain the K incremental clustering results and the K fraud-related clustering results, thereby effectively improving the efficiency and accuracy of the backtracking analysis, and can group the fraud-related websites through the K-means clustering analysis algorithm, which is convenient for subsequent classification and analysis. At the same time, after multiple backtracking analyses of fraudulent website addresses, the changing patterns of online fraud activities can also be discovered, which is conducive to preventing new fraud methods.
[0073]
[0074]
[0075]
[0076]
[0077] In one embodiment of the present invention, obtaining a relative link feature vector corresponding to the newly added domain name log information based on any of the newly added domain name log information includes:
[0078] Extract a plurality of newly added relative link features of any of the newly added domain name log information; compare and match each of the newly added relative link features with a preset fraud-related relative link feature set to generate the relative link feature vector.
[0079] It should be noted that the newly added relative link feature refers to the relative link feature corresponding to the domain name in the newly added domain name log information. The preset fraud-related relative link feature set is a relative link feature set of known fraud-related URLs, and the preset fraud-related relative link feature set will be automatically and continuously updated to expand the matching range, thereby improving the accuracy of the newly added relative link feature.
[0080] Specifically, extract a number of newly added relative link features of any of the newly added domain name log information, and then compare and match each of the newly added relative link features with a preset fraud-related relative link feature set to generate a relative link feature vector. For example, in one embodiment, compare and match each of the newly added relative link features with a preset fraud-related relative link feature set. If the newly added relative link feature successfully matches the preset fraud-related relative link feature set, the corresponding feature vector is 1; if the newly added relative link feature fails to match the preset fraud-related relative link feature set, the corresponding feature vector is 0, and then merge all feature vectors to generate a relative link feature vector. For details, please refer to Table 5, which is a matching table of relative link feature vectors of some newly added domain names in one embodiment of the present invention.
[0081]
[0082] The embodiment of the present invention generates a relative link feature vector by extracting a number of newly added relative link features of any of the newly added domain name log information, and then comparing and matching each of the newly added relative link features with a preset fraud-related relative link feature set, thereby generating a relative link feature vector by extracting the relative link features of the newly added domain name, and then accurately identifying fraud-related URLs based on the fact that fraud suspects usually use website templates to quickly build fraud-related websites while website templates (corresponding to relative link addresses) change relatively slowly, thereby facilitating timely discovery and disposal of various fraud-related URLs and ensuring network security.
[0083] In one embodiment of the present invention, the step of obtaining log information of a plurality of newly added domain names within a preset detection time includes:
[0084] Obtain user online log information within a preset detection time, and extract the host field in the user online log information; match the host field with a preset whitelist URL library to filter the whitelist domain name log information in the user online log information; compare and match the host field obtained after filtering with a preset stock domain name library to obtain the plurality of newly added domain name log information.
[0085] It should be noted that the user online log information includes user information, host field and target URL, etc., wherein the host field is a field in the HTTP request header, which is used to indicate the domain name or IP address of the request target server. For example, the user online log information sample is as follows:
[0086] {MSISDN: 138xxxx7890, [mobile phone number]
[0087] HOST:iflow.uczzd.cn, [Host field]
[0088] URL: http: / / iflow.uczzd.cn / iflow / api / v1 / client_aps, [target URL]
[0089] IP: 106.11.14.4, [target IP address]
[0090] PORT:80, [target IP port]
[0091] Context-Type: application / json, [Content Type]
[0092] Context-Length: 31243, [Content length]
[0093] RAT:2, [network type, 2 represents 2G, 5 represents 3G, 6 represents 4G, 10 represents 5G]
[0094] Message_Status: 200 [HTTP response code]
[0095] Start_time:1638943363475652, [access start time]
[0096] End_time:1638943491311540, [Access deadline]}
[0097] It should be further explained that the preset whitelist URL library records various common trusted HOST domain names or IP addresses, which are used to quickly screen the Internet access logs generated by most of the trusted domain names in the existing network to improve the analysis and matching performance. In addition, the preset stock domain name library records the stock newly added domain names based on the historical user Internet access log information and the newly added domain name information, and continuously updates automatically to screen the newly added domain name log information in the user Internet access log, including the stock domain name, IP address, port, storage time, etc. In addition, the preset stock domain name library also provides functions such as domain name query.
[0098] Specifically, the user's online log information within the preset detection time is obtained, and the host field in the user's online log information is extracted, and then the host field is matched with the preset whitelist URL library. When the match is successful, the whitelist domain name log information in the user's online log information is filtered out. When the match fails, the host field obtained after filtering is compared and matched with the preset stock domain name library. If the match is successful, the domain name log information that already exists in the user's online log information is filtered out, and then the several newly added domain name log information is obtained.
[0099] In addition, each of the newly added domain name log information is saved in a new domain name temporary table. Before the next execution of the step of obtaining the user Internet log information within the preset detection time, the new domain name temporary table is merged into the preset existing domain name library, and the new domain name temporary table is cleared at the same time to realize automatic update of the new domain name.
[0100] The embodiment of the present invention obtains user Internet log information within a preset detection time, extracts the host field in the user Internet log information, and then matches the host field with a preset whitelist URL library to filter the whitelist domain name log information in the user Internet log information, and compares and matches the host field obtained after filtering with a preset stock domain name library to obtain the multiple newly added domain name log information, thereby realizing quasi-real-time log stream technology, and then through the quasi-real-time log stream technology, the real-time data collection and low-latency transmission of the newly added domain name log information are realized, thereby laying a foundation for the rapid analysis of the newly added website that the user has just visited.
[0101] In one embodiment of the present invention, before inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0102] Obtain several groups of fraudulent website samples and training sample labels corresponding to the groups of fraudulent website samples; iteratively train the initial fraudulent website classification and analysis model according to each of the fraudulent website samples and the training sample labels corresponding to the fraudulent website samples to obtain the fraudulent website classification and analysis model.
[0103] It should be noted that each of the fraudulent URL samples includes known fraudulent URL samples and training samples such as the fraudulent relative link feature set and fraudulent URL list obtained after the backtracking analysis process to ensure the robustness of the model.
[0104] Specifically, several groups of fraudulent website samples and the training sample labels corresponding to the several groups of fraudulent website samples are obtained, and then each of the fraudulent website samples is input into the initial fraudulent website classification and judgment model to obtain the predicted value output by the initial fraudulent website classification and judgment model, wherein the support vector machine (SVM) classification algorithm can be used, and then based on the predicted value and the training sample labels corresponding to each of the fraudulent website samples, the model loss value is calculated using the loss function. In this embodiment, the loss function can be set according to actual needs and is not specifically limited here. After the model loss value is calculated, the training process ends, and the error back propagation algorithm is used to update the model parameters in the initial fraudulent website classification and judgment model, and then the next training is carried out. During the training process, it is determined whether the updated initial fraudulent website classification and judgment model meets the preset training end conditions. If it does, the updated initial fraudulent website classification and judgment model is used as the fraudulent website classification and judgment model. If it does not meet the conditions, the model training continues, wherein the preset training end conditions include loss convergence and reaching the maximum number of iterations.
[0105] The embodiment of the present invention obtains several groups of fraudulent website address samples and training sample labels corresponding to the several groups of fraudulent website address samples, and then iteratively trains the initial fraudulent website address classification and analysis model according to each of the fraudulent website address samples and the training sample labels corresponding to the fraudulent website address samples to obtain the fraudulent website address classification and analysis model, thereby increasing the robustness of the model and effectively improving the accuracy of the model output results.
[0106] In one embodiment of the present invention, after inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes:
[0107] If the fraud-related analysis result corresponding to the newly added domain name log information is a fraud-related URL result, the newly added domain name log information is matched with a preset malicious URL library; if the match fails, fraud-related domain name information is generated based on the newly added domain name log information, and the fraud-related domain name information is added to the preset malicious URL library; the fraud-related domain name information is pushed to the management personnel, so that the management personnel can manage the newly added domain name log information based on the fraud-related domain name information.
[0108] It should be noted that the preset malicious URL library refers to the log information library of known fraudulent URLs, including fraudulent URLs, IP addresses, ports, domain name sources, fraudulent URL categories, fraudulent URL template categories, and matching types, etc. In addition, the fraudulent URL results include research and judgment categories, fraudulent URL categories, and fraudulent URL template categories, etc. For example, in one embodiment, the fraudulent research and judgment results output by the fraudulent URL classification research and judgment model can refer to Table 6, which is the fraudulent research and judgment results output by the fraudulent URL classification research and judgment model in one embodiment of the present invention.
[0109]
[0110] It should be further explained that the fraudulent domain name information includes the fraudulent domain name, port, IP, fraudulent web page and browser rendering effect screenshots, etc. Optionally, the fraudulent domain name information can be generated through the "Python+Selenium+WebDriver+Chrome+Xvfb" crawler framework, simulating the mobile browser to access the fraudulent domain name and crawl data, and save the HTML web page and browser rendering effect screenshots corresponding to the fraudulent domain name as supplementary evidence for the fraudulent research and judgment results, so as to facilitate security experts to conduct a second manual review to improve the accuracy of the fraudulent research and judgment results. Among them, in order to effectively save the Chinese content of the HTML web page, the crawler framework will read the charset character set attribute of the HTML web page as the character encoding for saving the web page, including <meta charset="UTF-8"> , <meta http-equiv="content-type" content="text / html;charset=UTF-8 "> Etc. The fraud-related domain name information can also be generated according to other situations, which are not limited here.
[0111] Specifically, if the fraud-related judgment result corresponding to the newly added domain name log information is a fraud-related website result, the newly added domain name log information is matched with the preset malicious website library, wherein the host field of the fraud-related website or the URL field of the fraud-related website can be used for matching to improve the matching speed. Further, if the match fails, that is, the newly added domain name log information does not exist in the preset malicious website library, then the fraud-related domain name information is generated according to the newly added domain name in the newly added domain name log information, and the fraud-related domain name information is added to the preset malicious website library, and then the fraud-related domain name information is pushed to the management personnel, so that the management personnel can perform early warning management on the newly added domain name log information according to the fraud-related domain name information, wherein the push can be performed by sending emails, APP notifications, and mini-program notifications, etc., which are not limited here.
[0112] In the embodiment of the present invention, if the fraud-related analysis result corresponding to the newly added domain name log information is a fraud-related website result, the newly added domain name log information is matched with a preset malicious website library, and then when the match fails, fraud-related domain name information is generated based on the newly added domain name log information, and the fraud-related domain name information is added to the preset malicious website library, so that the fraud-related domain name information is pushed to the management personnel, and then the management personnel perform early warning management on the newly added domain name log information based on the fraud-related domain name information, so that preventive response measures can be taken more quickly and specifically, the risk of being attacked is reduced, and losses are controlled, and at the same time, it is conducive to the improvement of the network security knowledge base.
[0113] The fraudulent website detection method provided in this embodiment can be applied to a network anti-fraud big data analysis platform, wherein the network anti-fraud big data analysis platform includes functions such as centralized management, data analysis, and data collection. For details, please refer to Figure 4 and Figure 5 , Figure 4 This is a system module diagram of the network anti-fraud big data analysis platform provided by the present invention. Figure 5 It is a partial function implementation diagram of the network anti-fraud big data analysis platform provided by the present invention.
[0114] The following is a description of the fraudulent website detection device provided by the present invention. The fraudulent website detection device described below and the fraudulent website detection method described above can be referenced to each other.
[0115] Figure 2 is a schematic diagram of the structure of the fraudulent website detection device provided by the present invention. Figure 2 As shown, a fraudulent website detection device according to an embodiment of the present invention includes:
[0116] The acquisition module 21 is used to acquire log information of a number of newly added domain names within a preset detection time;
[0117] A feature module 22, configured to obtain a relative link feature vector corresponding to any of the newly added domain name log information based on the newly added domain name log information;
[0118] The analysis module 23 is used to input the relative link feature vector into the fraudulent website classification analysis model to obtain the fraudulent analysis result output by the fraudulent website classification analysis model to determine the fraudulent situation of the newly added domain name log information.
[0119] The fraudulent website detection device is also used for:
[0120] Obtaining a preset number of clusters, a list of fraudulent websites, and a list of incremental domain name logs within a preset backtracking time, wherein the preset number of clusters is the K value in the K-means clustering algorithm;
[0121] Based on the preset number of clusters, the list of fraud-related web addresses, and the incremental domain name log list, a K-means cluster analysis is performed to obtain K incremental clustering results and K fraud-related clustering results;
[0122] Each of the incremental clustering results and each of the fraud-related clustering results are pushed to security experts, so that the security experts can classify the list of fraud-related websites with labels and use them as training samples to train the classification and analysis model for fraud-related websites.
[0123] The fraudulent website detection device is also used for:
[0124] Based on the fraud-related website list and the incremental domain name log list, a number of incremental relative links and a number of fraud-related relative links are obtained;
[0125] Matching each of the incremental relative links and each of the fraud-related relative links with a preset whitelist relative link set;
[0126] Based on the preset number of clusters, K-means clustering analysis is performed on each of the matched incremental relative links and each of the matched fraud-related relative links to obtain the K incremental clustering results and the K fraud-related clustering results.
[0127] The fraudulent website detection device is also used for:
[0128] Extracting a plurality of newly added relative link features from any of the newly added domain name log information;
[0129] Each of the newly added relative link features is compared and matched with a preset fraud-related relative link feature set to generate the relative link feature vector.
[0130] The fraudulent website detection device is also used for:
[0131] Obtaining user online log information within a preset detection time, and extracting a host field in the user online log information;
[0132] Matching the host field with a preset whitelist URL library to filter whitelist domain name log information in the user's Internet access log information;
[0133] The host field obtained after filtering is compared and matched with a preset stock domain name library to obtain the plurality of newly added domain name log information.
[0134] The fraudulent website detection device is also used for:
[0135] Obtaining several groups of fraudulent website samples and training sample labels corresponding to the several groups of fraudulent website samples;
[0136] According to each of the fraud-related website samples and the training sample labels corresponding to each of the fraud-related website samples, the initial fraud-related website classification and analysis model is iteratively trained to obtain the fraud-related website classification and analysis model.
[0137] The fraudulent website detection device is also used for:
[0138] If the fraud-related analysis result corresponding to the newly added domain name log information is a fraud-related website result, the newly added domain name log information is matched with a preset malicious website library;
[0139] If the match fails, the fraud-related domain name information is generated based on the newly added domain name log information, and the fraud-related domain name information is added to the preset malicious URL library;
[0140] The fraud-related domain name information is pushed to a management personnel so that the management personnel can manage the newly added domain name log information based on the fraud-related domain name information.
[0141] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.
[0142] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a memory 320, a communications interface 330 and a communication bus 340, wherein the processor 310, the memory 320 and the communications interface 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 320 to execute a fraudulent website detection method, the method comprising: obtaining a number of newly added domain name log information within a preset detection time; based on any of the newly added domain name log information, obtaining a relative link feature vector corresponding to the newly added domain name log information; inputting the relative link feature vector into a fraudulent website classification and judgment model, obtaining a fraudulent judgment result output by the fraudulent website classification and judgment model, so as to determine the fraudulent situation of the newly added domain name log information.
[0143] In addition, the logic instructions in the above-mentioned memory 320 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0144] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the fraudulent website address detection method provided by the above-mentioned methods, the method comprising: obtaining a number of newly added domain name log information within a preset detection time; based on any of the newly added domain name log information, obtaining a relative link feature vector corresponding to the newly added domain name log information; inputting the relative link feature vector into a fraudulent website address classification and analysis model, obtaining a fraudulent analysis result output by the fraudulent website address classification and analysis model, so as to determine the fraudulent status of the newly added domain name log information.
[0145] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting fraudulent websites, characterized in that: include: Obtain log information of several newly added domain names within the preset detection time; Based on any of the newly added domain name log information, obtaining a relative link feature vector corresponding to the newly added domain name log information; Inputting the relative link feature vector into a fraudulent website classification and judgment model to obtain a fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent status of the newly added domain name log information; After inputting the relative link feature vector into the fraudulent website classification and judgment model to obtain the fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes: Obtaining a preset number of groups, a list of fraudulent websites, and an incremental domain name log list within a preset backtracking time, wherein the preset number of groups is the K value in the K-means clustering algorithm; the incremental domain name log list refers to a list of domain name log information of all newly added domain names within the preset backtracking time, including incremental domain names and incremental relative links; Based on the preset number of clusters, the list of fraud-related web addresses, and the incremental domain name log list, a K-means cluster analysis is performed to obtain K incremental clustering results and K fraud-related clustering results; Pushing each of the incremental clustering results and each of the fraud-related clustering results to security experts, so that the security experts can classify the list of fraud-related websites with labels and use them as training samples to train the classification and judgment model of fraud-related websites; The cluster analysis is performed based on the preset number of clusters, the fraud-related website list, and the incremental domain name log list to obtain K incremental clustering results and K fraud-related clustering results, including: Based on the fraudulent website list and the incremental domain name log list, a plurality of incremental relative links and a plurality of fraudulent relative links are obtained; the incremental relative link refers to a relative link corresponding to an incremental domain name in the incremental domain name log list, the fraudulent relative link refers to a relative link corresponding to a fraudulent domain name in the fraudulent website list, and the relative link is a link method used in Hypertext Markup Language (HTML) or other markup languages; Matching each of the incremental relative links and each of the fraud-related relative links with a preset whitelist relative link set; Based on the preset number of clusters, K-means cluster analysis is performed on each of the matched incremental relative links and each of the matched fraud-related relative links to obtain the K incremental clustering results and the K fraud-related clustering results; The obtaining, based on any of the newly added domain name log information, a relative link feature vector corresponding to the newly added domain name log information includes: Extracting a plurality of newly added relative link features of any of the newly added domain name log information; the newly added relative link features refer to relative link features corresponding to the domain names in the newly added domain name log information; Compare and match each of the newly added relative link features with a preset fraud-related relative link feature set to generate the relative link feature vector; Among them, comparing and matching each of the newly added relative link features with the preset fraud-related relative link feature set to generate the relative link feature vector includes: Compare and match each of the newly added relative link features with a preset fraud-related relative link feature set; If the newly added relative link feature successfully matches the preset fraud-related relative link feature set, the corresponding feature vector is 1; If the newly added relative link feature fails to match the preset fraud-related relative link feature set, the corresponding feature vector is 0; All feature vectors are merged to generate a relative link feature vector.
2. The method for detecting fraudulent websites according to claim 1, characterized in that: The obtaining of a plurality of newly added domain name log information within a preset detection time includes: Obtaining user online log information within a preset detection time, and extracting a host field in the user online log information; Matching the host field with a preset whitelist URL library to filter whitelist domain name log information in the user's Internet access log information; The host field obtained after filtering is compared and matched with a preset stock domain name library to obtain the plurality of newly added domain name log information.
3. The method for detecting fraudulent websites according to claim 1, characterized in that: Before inputting the relative link feature vector into the fraudulent website classification and judgment model to obtain the fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes: Obtaining several groups of fraudulent website samples and training sample labels corresponding to the several groups of fraudulent website samples; According to each of the fraud-related website samples and the training sample labels corresponding to each of the fraud-related website samples, the initial fraud-related website classification and analysis model is iteratively trained to obtain the fraud-related website classification and analysis model.
4. The method for detecting fraudulent websites according to claim 1, characterized in that: After inputting the relative link feature vector into the fraudulent website classification and judgment model to obtain the fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes: If the fraud-related analysis result corresponding to the newly added domain name log information is a fraud-related website result, the newly added domain name log information is matched with a preset malicious website library; If the match fails, generating fraudulent domain name information based on the newly added domain name log information, and adding the fraudulent domain name information to the preset malicious URL library; The fraud-related domain name information is pushed to a management personnel so that the management personnel can manage the newly added domain name log information based on the fraud-related domain name information.
5. A device for detecting fraudulent websites, characterized in that: include: The acquisition module is used to obtain log information of several newly added domain names within a preset detection time; A feature module, used to obtain a relative link feature vector corresponding to any of the newly added domain name log information based on the newly added domain name log information; A judgment module is used to input the relative link feature vector into a fraudulent website classification judgment model to obtain a fraudulent judgment result output by the fraudulent website classification judgment model to determine the fraudulent situation of the newly added domain name log information; After inputting the relative link feature vector into the fraudulent website classification and judgment model to obtain the fraudulent judgment result output by the fraudulent website classification and judgment model to determine the fraudulent situation of the newly added domain name log information, the method further includes: Obtaining a preset number of groups, a list of fraudulent websites, and an incremental domain name log list within a preset backtracking time, wherein the preset number of groups is the K value in the K-means clustering algorithm; the incremental domain name log list refers to a list of domain name log information of all newly added domain names within the preset backtracking time, including incremental domain names and incremental relative links; Based on the preset number of clusters, the list of fraud-related web addresses, and the incremental domain name log list, a K-means cluster analysis is performed to obtain K incremental clustering results and K fraud-related clustering results; Pushing each of the incremental clustering results and each of the fraud-related clustering results to security experts, so that the security experts can classify the list of fraud-related websites with labels and use them as training samples to train the classification and judgment model of fraud-related websites; The cluster analysis is performed based on the preset number of clusters, the fraud-related website list, and the incremental domain name log list to obtain K incremental clustering results and K fraud-related clustering results, including: Based on the fraudulent website list and the incremental domain name log list, a plurality of incremental relative links and a plurality of fraudulent relative links are obtained; the incremental relative link refers to a relative link corresponding to an incremental domain name in the incremental domain name log list, the fraudulent relative link refers to a relative link corresponding to a fraudulent domain name in the fraudulent website list, and the relative link is a link method used in Hypertext Markup Language (HTML) or other markup languages; Matching each of the incremental relative links and each of the fraud-related relative links with a preset whitelist relative link set; Based on the preset number of clusters, K-means cluster analysis is performed on each of the matched incremental relative links and each of the matched fraud-related relative links to obtain the K incremental clustering results and the K fraud-related clustering results; The obtaining, based on any of the newly added domain name log information, a relative link feature vector corresponding to the newly added domain name log information includes: Extracting a plurality of newly added relative link features of any of the newly added domain name log information; the newly added relative link features refer to relative link features corresponding to the domain names in the newly added domain name log information; Compare and match each of the newly added relative link features with a preset fraud-related relative link feature set to generate the relative link feature vector; Among them, comparing and matching each of the newly added relative link features with the preset fraud-related relative link feature set to generate the relative link feature vector includes: Compare and match each of the newly added relative link features with a preset fraud-related relative link feature set; If the newly added relative link feature successfully matches the preset fraud-related relative link feature set, the corresponding feature vector is 1; If the newly added relative link feature fails to match the preset fraud-related relative link feature set, the corresponding feature vector is 0; All feature vectors are merged to generate a relative link feature vector.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method for detecting fraudulent websites as described in any one of claims 1 to 4 is implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting fraudulent websites as described in any one of claims 1 to 4 is implemented.
