Abnormal network link identification method and device, electronic equipment, medium and product

Abstract

The application provides an abnormal network link identification method and device, electronic equipment, medium and product, relates to the technical field of network security, and fuses the application program types identified by at least one application program identification module, obtains an abnormal application program identification result of an application program link, reduces the deviation of a single application program identification module, and improves the overall identification precision and overall identification efficiency of the abnormal application program identification result. The application fuses the URL types identified by at least one URL identification module, obtains an abnormal URL identification result of a network protocol link, reduces the deviation of a single URL identification module, and improves the overall identification precision and overall identification efficiency of the abnormal URL identification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to an abnormal network link identification method and device, electronic equipment, medium and product. BACKGROUND

[0002] The existing network anomaly data identification only targets a single data source of abnormal data, cannot comprehensively cover all features of abnormal data, and has low identification efficiency. When processing large-scale network data, the existing network anomaly data identification has poor instantaneity and is difficult to cope with rapidly changing network anomaly data.

[0003] In summary, the existing network anomaly data identification has low efficiency. SUMMARY

[0004] The present application provides an abnormal network link identification method and device, electronic equipment, medium and product, to solve the defect of low efficiency of network anomaly data identification in the prior art, and to improve the identification efficiency of network anomaly data.

[0005] In a first aspect, the present application provides an abnormal network link identification method, comprising: acquiring a data type of a network link; when the data type is an application program link, inputting the network link into an application program fusion identification model to obtain an abnormal application program identification result output by the application program fusion identification model; wherein the application program fusion identification model comprises at least one application program identification module, the application program identification module is used for identifying an application program type of the network link, and the application program fusion identification model fuses at least one application program type to obtain the abnormal application program identification result; when the data type is a network protocol link with an explicit website address, inputting the network link into a website address fusion identification model to obtain an abnormal website address identification result output by the website address fusion identification model; wherein the website address fusion identification model comprises at least one website address identification module, the website address identification module is used for identifying a website address type of the network link, and the website address fusion identification model fuses at least one website address type to obtain the abnormal website address identification result.

[0006] In one embodiment, the data type of the network link is acquired, including: when the domain name of the network link is clear and the network address is not clear, acquiring an extracted feature set of the network link and a feature weight of each extracted feature of the extracted feature set, the extracted features including at least a domain name length, a judgment result of whether the domain name is an IP type, domain name information entropy, a number of vowel letters in the domain name, a number of digits in the domain name, a number of times the domain name appears in a specific time window of a specific type of website, and a judgment result of whether the network link is a content distribution network; performing weighted summation on the plurality of extracted features and the corresponding feature weights to obtain a feature value of the network link; and performing normalization processing on the feature value to obtain a danger score of the network link, and if the danger score exceeds a set score, completing the network address of the network link to acquire the data type.

[0007] In one embodiment, the network address of the network link is completed to acquire the data type, including: based on the danger score, acquiring a maximum crawler access number of the network link; acquiring an initial network address set of the network link, and based on a number of initial network addresses of the initial network address set, acquiring an initial crawler access number of the network link; when the initial crawler access number is less than the maximum crawler access number, acquiring a plurality of associated network addresses of the initial network address, and if the domain name of the associated network address is the same as the domain name of the network link and the associated network address is not in the initial network address set, adding the associated network address to the initial network address set to update the initial network address set and increase the initial crawler access number by 1; when the initial crawler access number increases to the maximum crawler access number or the initial network address set is no longer updated, taking the last updated initial network address set as a completed network address set of the network link; and based on each completed network address in the completed network address set of the network link, acquiring the data type.

[0008] In one embodiment, the network address fusion recognition model is used to fuse at least one network address type to obtain an abnormal network address recognition result, including: acquiring a first recognition weight of each network address type, the first recognition weight being a weight of a network address recognition module recognizing the network address type; based on the first recognition weight, acquiring a first ranking score of the network link belonging to each network address type, the first ranking score being larger when the first recognition weight is larger; based on the tuning configuration information of the network address recognition module, adjusting the first ranking score to obtain an adjusted first ranking score; taking a network address type corresponding to the highest adjusted first ranking score as a recognized network address type of the network link; and based on a comparison result of the recognized network address type and a set normal network address type, acquiring an abnormal network address recognition result of the network link.

[0009] In an embodiment, the application program fusion identification model is used for fusion of at least one application program type to obtain an abnormal application program identification result, including: obtaining a second identification weight of each application program type, the second identification weight being a weight of an application program identification module identifying the application program type; obtaining a second ranking score of the network link belonging to each application program type based on the second identification weight, the second ranking score being larger when the second identification weight is larger; adjusting the second ranking score based on the tuning configuration information of the application program identification module to obtain an adjusted second ranking score; taking the application program type corresponding to the highest adjusted second ranking score as the identified application program type of the network link; and obtaining the abnormal application program identification result of the network link based on a comparison result of the identified application program type and the set normal application program type.

[0010] In an embodiment, the tuning configuration information of the URL identification module includes: for a network link, when the URL types identified by at least two URL identification modules are the same, obtaining the same URL type, and adjusting the first ranking score corresponding to the same URL type to be the highest first ranking score; and for a network link, when the URL types identified by at least three URL identification modules are the set normal URL type, adjusting the first ranking score corresponding to the set normal URL type to be the highest first ranking score.

[0011] In a second aspect, the present application provides an abnormal network link identification device, including: a data type determination module, configured to obtain a data type of a network link; an abnormal application program identification module, configured to input the network link into an application program fusion identification model when the data type is an application program link, and obtain an abnormal application program identification result output by the application program fusion identification model; wherein the application program fusion identification model includes at least one application program identification module, the application program identification module is configured to identify an application program type of the network link, and the application program fusion identification model is configured to fuse at least one application program type to obtain the abnormal application program identification result; and an abnormal URL identification module, configured to input the network link into a URL fusion identification model when the data type is a URL explicit network protocol link, and obtain an abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model includes at least one URL identification module, the URL identification module is configured to identify a URL type of the network link, and the URL fusion identification model is configured to fuse at least one URL type to obtain the abnormal URL identification result.

[0012] In a third aspect, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the above abnormal network link identification methods when executing the computer program.

[0013] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements any of the above-described abnormal network link identification methods.

[0014] In a fifth aspect, the present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements any of the above-described abnormal network link identification methods.

[0015] The abnormal network link identification method, device, electronic device, medium and product provided by the present application fuse the application program types identified by the at least one application program identification module, obtain the abnormal application program identification result of the application program link, reduce the deviation of a single application program identification module, and improve the overall identification precision and overall identification efficiency of the abnormal application program identification result. The abnormal network link identification method, device, electronic device, medium and product provided by the present application fuse the website types identified by the at least one website identification module, obtain the abnormal website identification result of the network protocol link, reduce the deviation of a single website identification module, and improve the overall identification precision and overall identification efficiency of the abnormal website identification result. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0017] Figure 1 is one of the flowcharts of the abnormal network link identification method provided by the present application.

[0018] Figure 2 is another flowchart of the abnormal network link identification method provided by the present application.

[0019] Figure 3 is a structural schematic diagram of the abnormal network link identification device provided by the present application.

[0020] Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] The application will be described below Figures 1-4 The application provides an abnormal network link identification method, device and electronic equipment.

[0023] Figure 1 is one of the flowcharts of the abnormal network link identification method provided by the application, as Figure 1 shown, the abnormal network link identification method includes S100 to S300, and each step is as follows.

[0024] S100: Obtain the data type of the network link.

[0025] As Figure 2 shown, first, collect network traffic data, capture and process the passing data packets through the Deep Packet Inspection (DPI) technology, obtain the network link, including the website request of HTTP and HTTPS protocol. Data cleaning is performed on the network link, and the data cleaning step aims to remove invalid or redundant information and retain data useful for fraud resource identification. The cleaning process may include removing duplicate websites, filtering irrelevant protocol information, filtering white lists, etc., to ensure the accuracy and reliability of the data.

[0026] Through Kafka message queue and other technologies, the network link after data cleaning is sent to the data analysis module. The data analysis module performs data analysis on the network link after data cleaning, and obtains the data type of the network link. For example, the network link with.apk suffix usually points to an Application (APP) download link, which is an APP link. For non-APP links, further distinguish between HTTP protocol data and HTTPS protocol data. The domain name (Host) and Uniform Resource Identifier (URI) of the HTTP protocol data can be identified, which is a network protocol link with a clear website. The HTTPS protocol data can only identify the domain name, and cannot identify the website, which is a network protocol link with a clear domain name but an unclear website.

[0027] S200: When the data type is an application link, input the network link into an application fusion identification model to obtain an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result.

[0028] A plurality of sample application program (APP) links and sample abnormal application program identification results thereof are obtained in advance. A preset application program fusion identification model is obtained according to at least one preset application program identification module. The sample APP links are labeled according to the sample abnormal application program identification results, to obtain APP link training samples carrying labels. The APP link training samples carrying labels are input into the preset application program fusion identification model, until the error of the predicted abnormal application program identification result output by the preset application program fusion identification model is less than a second set error, to obtain an application program fusion identification model.

[0029] When the data type is an application program (APP) link, the network link is input into the application program fusion identification model, to obtain an abnormal application program identification result output by the application program fusion identification model. The application program fusion identification model includes at least one application program identification module, and each application program identification module identifies one application program type of the network link. The application program fusion identification model fusion analyzes at least one application program type, to obtain a final APP link of the network link. The application program fusion identification model compares the final APP link with a normal APP link (white list), to obtain the abnormal application program identification result. For example, if the final APP link belongs to the normal APP link, it is determined that the final APP link is the normal APP link. If the final APP link does not belong to the normal APP link, it is determined that the final APP link is an abnormal APP link.

[0030] S300: When the data type is a network protocol link with an explicit website address, the network link is input into a website address fusion identification model, to obtain an abnormal website address identification result output by the website address fusion identification model; wherein the website address fusion identification model includes at least one website address identification module, the website address identification module is used to identify a website address type of the network link, and the website address fusion identification model fusion analyzes at least one website address type, to obtain the abnormal website address identification result.

[0031] A plurality of sample network protocol links with explicit website addresses and sample abnormal website address identification results thereof are obtained. A preset website address fusion identification model is obtained according to at least one preset website address identification module. The sample network protocol links are labeled according to the sample abnormal website address identification results, to obtain website protocol link training samples carrying labels. The website protocol link training samples carrying labels are input into the preset website address fusion identification model, until the error of the predicted abnormal website address identification result output by the preset website address fusion identification model is less than a third set error, to obtain a website address fusion identification model.

[0032] When the data type is a network protocol link with an explicit network address, the network link is input into a network address fusion recognition model to obtain an abnormal network address recognition result output by the network address fusion recognition model. The network address fusion recognition model includes at least one network address recognition module, and each network address recognition module recognizes one network address type of the network link. The network address fusion recognition model analyzes at least one network address type to obtain a final network address type of the network link. The network address fusion recognition model filters the final network address type according to a normal and legal network address to obtain an abnormal network address recognition result of the final network address type.

[0033] The abnormal network link recognition method provided by the embodiment of the application fuses the application program types recognized by the at least one application program recognition module to obtain an abnormal application program recognition result of the application program link, reduces the deviation of a single application program recognition module, and improves the overall recognition accuracy and overall recognition efficiency of the abnormal application program recognition result. The network protocol link is fused by the at least one network address recognition module to obtain an abnormal network address recognition result, reduces the deviation of a single network address recognition module, and improves the overall recognition accuracy and overall recognition efficiency of the abnormal network address recognition result.

[0034] Based on the above embodiment, the data type of the network link is obtained, including S110 to S130, and each step is specifically as follows.

[0035] S110: When the domain name of the network link is explicit and the network address is not explicit, the feature set of the network link and the feature weight of each feature in the feature set are obtained, and the features include at least the length of the domain name, the judgment result of whether the domain name is an IP type, the information entropy of the domain name, the number of vowel letters in the domain name, the number of numbers in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and the judgment result of whether the network link is a content distribution network.

[0036] S120: The multiple features and the corresponding feature weights are weighted and summed to obtain a feature value of the network link.

[0037] S130: The feature value is normalized to obtain a danger score of the network link. If the danger score exceeds a set score, the network address of the network link is completed to obtain the data type.

[0038] A plurality of sample network links with explicit domain names but non-explicit network addresses are obtained in advance. A sample danger score of the sample network link is obtained. The sample network link is labeled according to the label of the sample danger score to obtain a training sample carrying a label. The preset model is trained according to the training sample carrying the label until the output error of the preset model is less than a first set error to obtain a risk assessment model.

[0039] If the data analysis module determines that the domain name of the network link is clear, but the URL is unclear, a pre-trained risk assessment model can be used to extract features from the network link, resulting in an extracted feature set for the network link.

[0040] The risk assessment model extracts features from web links to obtain a feature set, which includes multiple features. These features include, but are not limited to, domain name length, whether the domain is an IP address, domain information entropy, number of vowels in the domain name, number of numbers in the domain name, the number of times the domain appears on a specific type of website within a specific time window, and the result of determining whether the web link is a Content Delivery Network (CDN). Specific types of websites include customer service websites, banking websites, lending websites, download websites, warning websites, conference websites, and dating websites, etc. The number of times the domain appears on a specific type of website within a specific time window includes the DPI (Data Points Per Hour) of the domain appearing with customer service websites within the same 5-minute, 30-minute, 1-hour, etc., and the DPI of the domain appearing with banking websites within the same 5-minute, 30-minute, 1-hour, etc. A risk score for the web link is then calculated.

[0041] ;

[0042] in, Rate the danger of web links. For normalization function, To extract the number of features from the feature set. For the first One extracted feature, For the first The weights of each extracted feature.

[0043] Determining the weights of extracted features includes, but is not limited to, logistic regression, support vector machine, decision tree, random forest, gradient boosting tree, etc.

[0044] The risk assessment model uses extracted feature sets to perform an initial risk screening of web links, determining a risk score for each link. If a link's risk score is higher than a set threshold, it is considered potentially suspicious. The model then completes the URL of the link and obtains the data type of the completed URL.

[0045] Furthermore, if the domain name of the URL link is clear but the URL is unclear, and the risk score of the link is less than the set score, then the link is determined to be a normal link.

[0046] The application obtains the danger score by weighted sum of the extracted features in the extracted feature set, and improves the accuracy of obtaining the danger score. The application obtains the danger score by the extracted feature set for preliminary screening of the network link with an unclear URL, can effectively filter out most normal network links in the early stage, and focuses on more in-depth analysis of high-risk network links, and also provides a reference for subsequent completion of the URL of the network link.

[0047] Based on the above embodiment, the URL of the network link is completed to obtain the data type, including S131 to S135, and each step is specifically as follows.

[0048] S131: obtaining the maximum crawler access number of the network link based on the danger score.

[0049] S132: obtaining an initial URL set of the network link, and obtaining an initial crawler access number of the network link based on the number of initial URLs in the initial URL set.

[0050] S133: when the initial crawler access number is less than the maximum crawler access number, obtaining a plurality of associated URLs of the initial URL, if the domain name of the associated URL is the same as the domain name of the network link and the associated URL is not in the initial URL set, adding the associated URL to the initial URL set to update the initial URL set, and adding 1 to the initial crawler access number.

[0051] S134: when the initial crawler access number increases to the maximum crawler access number, or the initial URL set is no longer updated, taking the last updated initial URL set as the completed URL set of the network link.

[0052] S135: obtaining the data type based on each completed URL in the completed URL set of the network link.

[0053] If the danger score is greater than the set score, the network link is retained, and subsequent URL completion operation is performed on the network link. If the danger score is less than or equal to the set score, the network link is determined as a normal network link, and the normal network link is removed.

[0054] For the network link with an explicit domain name and an unclear URL, for example, the captured URI of the HTTPS protocol link is incomplete (only the domain name), which greatly affects the identification of abnormal network links. Because many network protocol links cannot be determined whether they are abnormal network links only by the domain name.

[0055] For the network link with an explicit domain name and an unclear URL, a crawler technology is used to obtain all associated completed URLs under the domain name.

[0056] According to the danger score of the network link, the maximum crawler access number of the network link is calculated.

[0057] ;

[0058] wherein, is the maximum crawler access number of the network link with a danger score of , is the danger score, is a position parameter for determining at which point the danger score starts to have a significant impact on the maximum crawler access number, is a slope parameter, is the maximum crawler access number of the highest-risk website.

[0059] An initial set of website addresses of the network link is obtained, and the initial set of website addresses includes multiple initial website addresses of the network link. The number of initial website addresses is equal to the initial crawler access number of the network link. If the initial crawler access number of the network link is less than the maximum crawler access number, the associated website addresses of each initial website address are obtained, and the associated website addresses are added to the initial set of website addresses to update the initial set of website addresses, and the initial crawler access number of the initial set of website addresses is further updated.

[0060] When the initial crawler access number of the network link increases to the maximum crawler access number or all associated website addresses of the initial website addresses are traversed, the update of the initial set of website addresses is terminated. The last updated initial set of website addresses is taken as a complete set of website addresses of the network link. According to one complete website address in the complete set of website addresses, one data type of the network link is obtained. According to multiple complete website addresses in the complete set of website addresses, multiple data types of the network link are obtained. If the data type corresponding to the complete website address is an APP link, the complete website address and the network link (domain name) are input into an application fusion identification model to obtain an abnormal application identification result output by the application fusion identification model. If the data type corresponding to the complete website address is a network protocol link, the complete website address and the network link (domain name) are input into a website fusion identification model to obtain an abnormal website identification result output by the website fusion identification model.

[0061] The application obtains the maximum crawler access number according to the danger score, which is beneficial to further obtain the complete set of website addresses of the network link. The website of the network link is completed according to the maximum crawler access number, which improves the comprehensiveness of the complete set of website addresses.

[0062] Based on the above embodiment, the website fusion identification model is used to fuse at least one website type to obtain an abnormal website identification result, including S310 to S350, and each step is specifically as follows.

[0063] S310: Obtain a first identification weight of each website type, and the first identification weight is a weight of a website identification module identifying a website type.

[0064] S320: Obtain a first ranking score of the network link belonging to each URL type based on the first identification weight, and the first ranking score is larger when the first identification weight is larger.

[0065] S330: Adjust the first ranking score based on the tuning configuration information of the URL identification module to obtain an adjusted first ranking score.

[0066] S340: Take the URL type corresponding to the highest adjusted first ranking score as the identified URL type of the network link.

[0067] S350: Obtain an abnormal URL identification result of the network link based on the comparison result of the identified URL type and the set normal URL type.

[0068] When the data type of the network link is a URL-specific network protocol link, or the data type of the network link with a completed URL is a URL-specific network protocol link, input the network link (or the network link with a completed URL) into the URL fusion identification model. Each URL identification module of the URL fusion identification model performs URL identification on the network link, and finds a matching preset URL type from the preset URL type set as a URL type of the network link.

[0069] Different URL identification modules analyze and identify the network link from different angles (for example, URL identification module 1 may focus on website code structure analysis, URL identification module 2 may focus on domain name and IP address reputation analysis, URL identification module 3 may focus on website content analysis, etc.), and the output URL types may be different.

[0070] For a network link, obtain a URL type set through each URL identification module. Perform uniform format conversion on the URL type set. Obtain a preset identification weight of each preset URL type for each URL identification module, and the larger the preset identification weight, the higher the accuracy of the URL identification module in identifying the preset URL type. Obtain a first identification weight of each URL type after uniform format conversion according to the preset identification weight of each preset URL type for each URL identification module. Determine a first ranking score of each URL type according to the first identification weight of each URL type. For example, directly take the first identification weight as the first ranking score. For example, the first identification weight is 0.9, and the first ranking score is 0.9.

[0071] The tuning configuration information of the URL identification module includes: for a network link, when the URL types identified by at least two URL identification modules are the same, obtaining the same URL type, and adjusting the first ranking score corresponding to the same URL type to the highest first ranking score; for a network link, when the URL types identified by at least three URL identification modules are set normal URL types, adjusting the first ranking score corresponding to the set normal URL type to the highest first ranking score.

[0072] The set of tuning configuration information {r1, r2,..., r i ,..., r t} of the website identification module is set, and the tuning configuration information r i of each website identification module defines a classification decision under certain conditions. The first ranking score of each website type is adjusted according to the tuning configuration information of each website identification module.

[0073] For example, if website identification module 1 and website identification module 2 both mark a network link as an investment and financial management type (same website type), the first ranking score corresponding to the investment and financial management type is adjusted to the highest first ranking score (for example, 100 points), and the network link is further marked as an investment and financial management type.

[0074] If website identification module 1, website identification module 2, and website identification module 3 all mark a network link as a normal website type, the first ranking score corresponding to the normal website type is adjusted to the highest first ranking score, and the network link is further marked as a normal website type.

[0075] Further, the tuning configuration information of the website identification module further includes: if the time when the network link first appears in the DPI data is within 72 hours, then the first ranking score of website identification module 2 and website identification module 3 for the specified preset website type is increased by 0.5.

[0076] For each network link, the website type corresponding to the highest adjusted first ranking score is taken as the identified website type of the network link. If the identified website type does not belong to the set of normal website types, it is determined that the network link is an abnormal website, and a binary tuple consisting of {network link, identified website type} is obtained. If the identified website type belongs to the set of normal website types, it is determined that the network link is a normal website. The abnormal website identification result of the network link is stored in the database.

[0077] The present application obtains the first ranking score of each website type through the first identification weight, and realizes the preliminary determination of the identified website type of the network link. Adjusting the first ranking score according to the tuning configuration information of the website identification module is conducive to further determining the accuracy of the identified website type of the network link.

[0078] Based on the above embodiment, the application fusion identification model is used to fuse at least one application program type to obtain an abnormal application program identification result, including S210 to S250, and each step is as follows.

[0079] S210: Obtain a second identification weight of each application program type, and the second identification weight is the weight of the application program identification module identifying the application program type.

[0080] S220: Based on the second identification weight, obtain the second ranking score of the network link belonging to each application type. The larger the second identification weight, the larger the second ranking score.

[0081] S230: Based on the optimization configuration information of the application identification module, the second ranking score is adjusted to obtain the adjusted second ranking score.

[0082] S240: The application type corresponding to the highest adjusted second-ranked score is used as the identification application type for the network link.

[0083] S250: Based on the comparison results between the identified application type and the set normal application type, obtain the abnormal application identification results of the network link.

[0084] When the data type of the web link is an application (APP) link, or when the data type of the web link with the completed URL is an APP link, the web link (or the web link with the completed URL) is input into the application fusion recognition model. Each application recognition module of the application fusion recognition model performs APP recognition on the web link, finds a matching preset application type from the preset application type set, and uses it as an application type for the web link.

[0085] Different application identification modules analyze and identify network links from different perspectives, and the output application types may differ.

[0086] For a given network link, obtain the application type set through each application identification module. Perform a unified format conversion on the application type set. Obtain the preset identification weight for each preset application type by each application identification module. Based on the preset identification weight for each preset application type by each application identification module, obtain the second identification weight for each application type after unified format conversion. Determine the second ranking score for each application type based on the second identification weight for each application type.

[0087] Set the optimization configuration information set {r'1, r'2, ..., r'} for the application identification module. i ,...,r' t}, the tuning configuration information r' for each application identification module i The classification decision was defined under certain conditions. The second ranking score for each application type was adjusted based on the optimization configuration information of each application identification module.

[0088] For network links (APP links), the application type corresponding to the highest adjusted second-ranked score is used as the identified application type for the network link. If the identified application type does not belong to the set normal application type, the network link is determined to be an abnormal network link, and a tuple consisting of {network link, identified application type} is obtained. If the identified application type belongs to the set normal application type, the network link is determined to be a normal APP. The abnormal application identification result of the network link is then stored in the database.

[0089] This application obtains a second ranking score for each application type through a second identification weight, thus achieving a preliminary determination of the application type of network links. Adjusting the second ranking score based on the optimization configuration information of the application identification module helps to further determine the accuracy of identifying the application type of APP links.

[0090] The abnormal network link identification device provided in this application is described below. The abnormal network link identification device described below can be referred to in correspondence with the abnormal network link identification method described above.

[0091] like Figure 3 As shown, an abnormal network link identification device includes: a data type determination module 301, used to obtain the data type of the network link.

[0092] The abnormal application identification module 302 is used to input the network link into the application fusion identification model when the data type is an application link, and obtain the abnormal application identification result output by the application fusion identification model; wherein, the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result.

[0093] The abnormal URL identification module 303 is used to input the network link into the URL fusion identification model when the data type is a network protocol link with a clear URL, and obtain the abnormal URL identification result output by the URL fusion identification model; wherein, the URL fusion identification model includes at least one URL identification module, which is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain the abnormal URL identification result.

[0094] The abnormal network link identification device provided by the embodiment of the application fuses the application program types identified by the at least one application program identification module, obtains the abnormal application program identification result of the application program link, reduces the deviation of the single application program identification module, and improves the overall identification precision and overall identification efficiency of the abnormal application program identification result. The abnormal network link identification device fuses the website types identified by the at least one website identification module, obtains the abnormal website identification result of the network protocol link, reduces the deviation of the single website identification module, and improves the overall identification precision and overall identification efficiency of the abnormal website identification result.

[0095] In one embodiment, the data type determination module 301 is configured to: when the domain name of the network link is explicit and the website is not explicit, obtain the extracted feature set of the network link and the feature weight of each extracted feature of the extracted feature set, the extracted features including at least the length of the domain name, the judgment result of whether the domain name is an IP type, the domain name information entropy, the number of vowel letters in the domain name, the number of numbers in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and the judgment result of whether the network link is a content distribution network; weighted sum of the plurality of extracted features and the corresponding feature weights to obtain the feature value of the network link; and normalizing the feature value to obtain the danger score of the network link, if the danger score exceeds the set score, the website of the network link is completed to obtain the data type.

[0096] In one embodiment, the data type determination module 301 is configured to: based on the danger score, obtain the maximum crawler access number of the network link; obtain the initial website set of the network link, and based on the number of initial websites in the initial website set, obtain the initial crawler access number of the network link; when the initial crawler access number is less than the maximum crawler access number, obtain a plurality of associated websites of the initial website, if the domain name of the associated website is the same as the domain name of the network link, and the associated website is not in the initial website set, add the associated website to the initial website set to update the initial website set, and increase the initial crawler access number by 1; when the initial crawler access number increases to the maximum crawler access number, or the initial website set is no longer updated, the last updated initial website set is taken as the completed website set of the network link; and based on each completed website in the completed website set of the network link, obtain the data type.

[0097] In an embodiment, the abnormal website identification module 303 is configured to: obtain a first identification weight of each website type, the first identification weight being a weight of a website identification module identifying the website type in identifying the website type; obtain a first ranking score of the network link belonging to each website type based on the first identification weight, the first ranking score being larger when the first identification weight is larger; adjust the first ranking score based on the tuning configuration information of the website identification module to obtain an adjusted first ranking score; take the website type corresponding to the highest adjusted first ranking score as the identified website type of the network link; and obtain an abnormal website identification result of the network link based on a comparison result of the identified website type and the set normal website type.

[0098] In an embodiment, the abnormal application program identification module 302 is configured to: obtain a second identification weight of each application program type, the second identification weight being a weight of an application program identification module identifying the application program type in identifying the application program type; obtain a second ranking score of the network link belonging to each application program type based on the second identification weight, the second ranking score being larger when the second identification weight is larger; adjust the second ranking score based on the tuning configuration information of the application program identification module to obtain an adjusted second ranking score; take the application program type corresponding to the highest adjusted second ranking score as the identified application program type of the network link; and obtain an abnormal application program identification result of the network link based on a comparison result of the identified application program type and the set normal application program type.

[0099] In an embodiment, the tuning configuration information of the website identification module comprises: for a network link, when the website types identified by at least two website identification modules are the same, obtaining the same website type, and adjusting the first ranking score corresponding to the same website type to be the highest first ranking score; for a network link, when the website types identified by at least three website identification modules are the set normal website type, adjusting the first ranking score corresponding to the set normal website type to be the highest first ranking score.

[0100] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 4As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can invoke a logical instruction in the memory 430 to execute an abnormal network link identification method, which includes obtaining a data type of a network link; when the data type is an application link, inputting the network link into an application fusion identification model to obtain an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify an application type of the network link, the application fusion identification model fuses at least one application type to obtain the abnormal application identification result; when the data type is a network protocol link with an explicit website, inputting the network link into a website fusion identification model to obtain an abnormal website identification result output by the website fusion identification model; wherein the website fusion identification model includes at least one website identification module, the website identification module is used to identify a website type of the network link, and the website fusion identification model fuses at least one website type to obtain the abnormal website identification result.

[0101] In addition, the logical instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0102] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable the computer to execute the abnormal network link identification method provided by the above method, the method comprising: obtaining a data type of a network link; when the data type is an application program link, inputting the network link into an application program fusion identification model to obtain an abnormal application program identification result output by the application program fusion identification model; wherein the application program fusion identification model comprises at least one application program identification module, the application program identification module is configured to identify an application program type of the network link, and the application program fusion identification model fuses at least one application program type to obtain the abnormal application program identification result; when the data type is a network protocol link with an explicit website address, inputting the network link into a website address fusion identification model to obtain an abnormal website address identification result output by the website address fusion identification model; wherein the website address fusion identification model comprises at least one website address identification module, the website address identification module is configured to identify a website address type of the network link, and the website address fusion identification model fuses at least one website address type to obtain the abnormal website address identification result.

[0103] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executable by a processor to implement the abnormal network link identification method provided by the above method, the method comprising: obtaining a data type of a network link; when the data type is an application program link, inputting the network link into an application program fusion identification model to obtain an abnormal application program identification result output by the application program fusion identification model; wherein the application program fusion identification model comprises at least one application program identification module, the application program identification module is configured to identify an application program type of the network link, and the application program fusion identification model fuses at least one application program type to obtain the abnormal application program identification result; when the data type is a network protocol link with an explicit website address, inputting the network link into a website address fusion identification model to obtain an abnormal website address identification result output by the website address fusion identification model; wherein the website address fusion identification model comprises at least one website address identification module, the website address identification module is configured to identify a website address type of the network link, and the website address fusion identification model fuses at least one website address type to obtain the abnormal website address identification result.

[0104] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0105] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0106] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features thereof; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An abnormal network link identification method, characterized by, The method comprises the following steps: When the domain name of the network link is clear and the URL is not clear, the extraction feature set of the network link and the feature weight of each extraction feature in the extraction feature set are obtained; The feature values of the network link are obtained by weighted summation of the plurality of extraction features and the corresponding feature weights; The feature values are normalized to obtain the risk score of the network link. If the risk score exceeds the set score, the URL of the network link is completed to obtain the completed URL set of the network link. Based on each completed URL in the completed URL set, the data type of the network link is obtained; When the data type is an application link, the network link is input into an application fusion recognition model to obtain an abnormal application recognition result output by the application fusion recognition model; wherein the application fusion recognition model comprises at least one application recognition module, the application recognition module is used to identify the application type of the network link, and the application fusion recognition model fuses at least one application type to obtain the abnormal application recognition result; When the data type is a network protocol link with a clear URL, the network link is input into a URL fusion recognition model to obtain an abnormal URL recognition result output by the URL fusion recognition model; wherein the URL fusion recognition model comprises at least one URL recognition module, the URL recognition module is used to identify the URL type of the network link, and the URL fusion recognition model fuses at least one URL type to obtain the abnormal URL recognition result. 2.The method of claim 1, wherein, The data type of the network link is obtained, comprising: The extraction features at least include domain name length, judgment result of whether the domain name is IP type, domain name information entropy, number of domain name vowel letters, number of domain name digits, number of domain name appearing in a specific time window of a specific type of website, and judgment result of whether the network link is a content distribution network. 3.The method of claim 2, wherein, The completed URL set is obtained based on the following method: Based on the risk score, the maximum crawler access number of the network link is obtained; The initial URL set of the network link is obtained, and the initial crawler access number of the network link is obtained based on the number of initial URLs in the initial URL set; When the initial crawler access number is less than the maximum crawler access number, a plurality of associated URLs of the initial URL are obtained. If the domain name of the associated URL is the same as the domain name of the network link, and the associated URL is not in the initial URL set, the associated URL is added to the initial URL set to update the initial URL set, and the initial crawler access number is increased by 1; When the initial crawler access number increases to the maximum crawler access number, or the initial URL set is no longer updated, the last updated initial URL set is taken as the completed URL set of the network link. 4.The method of claim 1, wherein, The URL fusion recognition model is used to fuse at least one URL type to obtain the abnormal URL recognition result, comprising: obtaining a first identification weight of each of the URL types, the first identification weight being a weight of a URL identification module identifying the URL type; obtaining a first ranking score of the network link belonging to each of the URL types based on the first identification weight, the first ranking score being larger when the first identification weight is larger; adjusting the first ranking score based on the tuning configuration information of the URL identification module to obtain an adjusted first ranking score; taking a URL type corresponding to a highest adjusted first ranking score as an identified URL type of the network link; obtaining the abnormal URL identification result of the network link based on a comparison result of the identified URL type and a set normal URL type.

5. The abnormal network link identification method of claim 1, wherein, The application program fusion identification model is used for fusing at least one of the application program types to obtain the abnormal application program identification result, and includes: obtaining a second identification weight of each of the application program types, the second identification weight being a weight of an application program identification module identifying the application program type; obtaining a second ranking score of the network link belonging to each of the application program types based on the second identification weight, the second ranking score being larger when the second identification weight is larger; adjusting the second ranking score based on the tuning configuration information of the application program identification module to obtain an adjusted second ranking score; taking an application program type corresponding to a highest adjusted second ranking score as an identified application program type of the network link; obtaining the abnormal application program identification result of the network link based on a comparison result of the identified application program type and a set normal application program type.

6. The abnormal network link identification method of claim 4, wherein, The tuning configuration information of the URL identification module includes: for one of the network links, when the URL types identified by at least two of the URL identification modules are the same, obtaining the same URL type, and adjusting a first ranking score corresponding to the same URL type to be a highest first ranking score; for one of the network links, when the URL types identified by at least three of the URL identification modules are the set normal URL type, adjusting a first ranking score corresponding to the set normal URL type to be the highest first ranking score.

7. An abnormal network link identification apparatus characterized by comprising: The application program fusion identification model includes: a data type determination module, configured to obtain an extracted feature set of a network link and a feature weight of each extracted feature in the extracted feature set when a domain name of the network link is clear and a URL of the network link is unclear; and obtain a feature value of the network link by weighted summation of a plurality of the extracted features and the corresponding feature weights; performing normalization processing on the feature value to obtain a danger score of the network link, and if the danger score exceeds a set score, complementing a URL of the network link to obtain a complemented URL set of the network link, and obtaining a data type of the network link based on each complemented URL in the complemented URL set. An abnormal application program identification module is configured to input the network link into an application program fusion identification model when the data type is an application program link, and obtain an abnormal application program identification result output by the application program fusion identification model; wherein the application program fusion identification model comprises at least one application program identification module, the application program identification module is configured to identify an application program type of the network link, and the application program fusion identification model fuses at least one application program type to obtain the abnormal application program identification result; An abnormal website identification module is configured to input the network link into a website fusion identification model when the data type is a network protocol link with an explicit website, and obtain an abnormal website identification result output by the website fusion identification model; wherein the website fusion identification model comprises at least one website identification module, the website identification module is configured to identify a website type of the network link, and the website fusion identification model fuses at least one website type to obtain the abnormal website identification result.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the abnormal network link identification method in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the abnormal network link identification method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the abnormal network link identification method in any one of claims 1 to 6.

Images