Abnormal network link identification method and device, electronic equipment, medium and product

Abstract

The invention provides an abnormal network link identification method and device, electronic equipment, a medium and a product, and relates to the technical field of network security. Application program types identified by at least one application program identification module are fused to obtain an abnormal application program identification result of an application program link; the deviation of a single application program identification module is reduced, and the overall identification precision and the overall identification efficiency of an abnormal application program identification result are improved. By fusing the website types identified by the at least one website identification module, the abnormal website identification result of the network protocol link is obtained, the deviation of a single website identification module is reduced, and the overall identification precision and the overall identification efficiency of the abnormal website identification result are improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to an abnormal network link identification method, device, electronic device, medium and product. Background Art

[0002] The existing recognition of network anomaly data only targets a single data source of anomaly data, which cannot fully cover all the characteristics of anomaly data, and the recognition efficiency of anomaly data is low. When processing large-scale network data, the existing recognition of network anomaly data has poor immediacy and is difficult to cope with rapidly changing network anomaly data.

[0003] In summary, the efficiency of existing network abnormal data identification is low. Summary of the invention

[0004] The present application provides a method, device, electronic device, medium and product for identifying abnormal network links, so as to solve the defect of low efficiency in identifying abnormal network data in the prior art and improve the efficiency of identifying abnormal network data.

[0005] In a first aspect, the present application provides a method for identifying abnormal network links, comprising: obtaining a data type of a network link; when the data type is an application link, inputting the network link into an application fusion identification model, and obtaining an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain an abnormal application identification result; when the data type is a network protocol link with a clear URL, inputting the network link into a URL fusion identification model, and obtaining an abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model includes at least one URL identification module, the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain an abnormal URL identification result.

[0006] In one embodiment, obtaining the data type of a network link includes: when the domain name of the network link is clear and the URL is unclear, obtaining an extracted feature set of the network link and a feature weight of each extracted feature in the extracted feature set, wherein the extracted features include at least the domain name length, a judgment result of whether the domain name is an IP type, domain name information entropy, the number of vowels in the domain name, the number of digits in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and a judgment result of whether the network link is a content distribution network; performing weighted summation on multiple extracted features and corresponding feature weights to obtain a feature value of the network link; normalizing the feature value to obtain a risk score of the network link, and if the risk score exceeds a set score, completing the URL of the network link to obtain the data type.

[0007] In one embodiment, completing the URL of a network link to obtain a data type includes: obtaining a maximum crawler access count for the network link based on a risk score; obtaining an initial URL set for the network link, and obtaining an initial crawler access count for the network link based on the number of initial URLs in the initial URL set; when the initial crawler access count is less than the maximum crawler access count, obtaining multiple associated URLs of the initial URL, and if the domain name of the associated URL is the same as the domain name of the network link and the associated URL is not in the initial URL set, adding the associated URL to the initial URL set to update the initial URL set, and increasing the initial crawler access count by 1; when the initial crawler access count increases to the maximum crawler access count, or the initial URL set is no longer updated, using the last updated initial URL set as a completed URL set for the network link; and obtaining a data type based on each completed URL in the completed URL set for the network link.

[0008] In one embodiment, a URL fusion identification model is used to fuse at least one URL type to obtain an abnormal URL identification result, including: obtaining a first identification weight for each URL type, the first identification weight being the weight of a URL identification module for identifying the URL type; based on the first identification weight, obtaining a first ranking score of the network link belonging to each URL type, the larger the first identification weight, the larger the first ranking score; based on the tuning configuration information of the URL identification module, adjusting the first ranking score to obtain an adjusted first ranking score; using the URL type corresponding to the highest adjusted first ranking score as the identified URL type of the network link; based on the comparison result of the identified URL type and the set normal URL type, obtaining the abnormal URL identification result of the network link.

[0009] In one embodiment, an application fusion identification model is used to fuse at least one application type to obtain an abnormal application identification result, including: obtaining a second identification weight for each application type, the second identification weight being the weight of an application identification module that identifies the application type; based on the second identification weight, obtaining a second ranking score of the network link belonging to each application type, the larger the second identification weight, the larger the second ranking score; based on the tuning configuration information of the application identification module, adjusting the second ranking score to obtain an adjusted second ranking score; using the application type corresponding to the highest adjusted second ranking score as the identified application type of the network link; and obtaining an abnormal application identification result of the network link based on a comparison result of the identified application type and a set normal application type.

[0010] In one embodiment, the tuning configuration information of the URL identification module includes: for a network link, when the URL types identified by at least two URL identification modules are the same, the same URL type is obtained, and the first ranking score corresponding to the same URL type is adjusted to the highest first ranking score; for a network link, when the URL type identified by at least three URL identification modules is a set normal URL type, the first ranking score corresponding to the set normal URL type is adjusted to the highest first ranking score.

[0011] In a second aspect, the present application provides an abnormal network link identification device, including: a data type determination module, used to obtain the data type of the network link; an abnormal application identification module, used to input the network link into an application fusion identification model when the data type is an application link, and obtain the abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result; an abnormal website address identification module, used to input the network link into a website address fusion identification model when the data type is a network protocol link with a clear website address, and obtain the abnormal website address identification result output by the website address fusion identification model; wherein the website address fusion identification model includes at least one website address identification module, the website address identification module is used to identify the website address type of the network link, and the website address fusion identification model fuses at least one website address type to obtain the abnormal website address identification result.

[0012] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned abnormal network link identification methods is implemented.

[0013] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned abnormal network link identification methods.

[0014] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements any of the above-mentioned abnormal network link identification methods when executed by a processor.

[0015] The abnormal network link identification method, device, electronic device, medium and product provided by the present application obtain abnormal application identification results of application links by fusing the application types identified by at least one application identification module, thereby reducing the deviation of a single application identification module and improving the overall identification accuracy and overall identification efficiency of abnormal application identification results. The abnormal website identification results of network protocol links are obtained by fusing the website types identified by at least one website identification module, thereby reducing the deviation of a single website identification module and improving the overall identification accuracy and overall identification efficiency of abnormal website identification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is one of the flow charts of the abnormal network link identification method provided in this application.

[0018] Figure 2 This is the second flow chart of the abnormal network link identification method provided by this application.

[0019] Figure 3 It is a structural diagram of the abnormal network link identification device provided in this application.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] Combine the following Figure 1-Figure 4 The present invention describes an abnormal network link identification method, device and electronic device.

[0023] Figure 1 This is one of the flow charts of the abnormal network link identification method provided by this application, such as Figure 1 As shown, the abnormal network link identification method includes S100 to S300, and each step is specifically as follows.

[0024] S100: Acquire the data type of the network link.

[0025] like Figure 2 As shown in the figure, firstly, network traffic data is collected. Through the Deep Packet Inspection (DPI) technology, the system captures and processes the passing data packets to obtain network links, including URL requests of HTTP and HTTPS protocols. Data cleaning is performed on network links. The data cleaning step is to remove invalid or redundant information and retain data that is useful for identifying fraudulent resources. The cleaning process may include removing duplicate URLs, filtering irrelevant protocol information, filtering whitelists, etc. to ensure the accuracy and reliability of the data.

[0026] Through technologies such as Kafka message queues, the network links after data cleaning are sent to the data analysis module. The data analysis module performs data analysis on the network links after data cleaning to obtain the data type of the network links. For example, a network link with the .apk suffix usually points to an application (Application, APP) download link, which is an APP link. For non-APP links, they are further divided into HTTP protocol data and HTTPS protocol data. The domain name (Host) and URL (Uniform Resource Identifier, URI) of HTTP protocol data can both be identified, and it is a network protocol link with a clear URL. HTTPS protocol data can only identify the domain name, not the URL, and is a network protocol link with a clear domain name but an unclear URL.

[0027] S200: When the data type is an application link, the network link is input into the application fusion identification model to obtain the abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result.

[0028] Acquire multiple sample application (APP) links and their sample abnormal application recognition results in advance. Acquire a preset application fusion recognition model according to at least one preset application recognition module. Label the sample APP links according to the sample abnormal application recognition results to obtain labeled APP link training samples. Input the labeled APP link training samples into the preset application fusion recognition model until the error of the predicted abnormal application recognition result output by the preset application fusion recognition model is less than the second set error, and obtain the application fusion recognition model.

[0029] When the data type is an application (APP) link, the network link is input into the application fusion identification model to obtain the abnormal application identification result output by the application fusion identification model. The application fusion identification model includes at least one application identification module, each of which identifies an application type of the network link. The application fusion identification model integrates and analyzes at least one application type to obtain the final application (APP) link of the network link. The application fusion identification model compares the final APP link with the normal APP link (white list) to obtain the abnormal application identification result. For example, if the final APP link belongs to a normal APP link, the final APP link is determined to be a normal APP link. If the final APP link does not belong to a normal APP link, the final APP link is determined to be an abnormal APP link.

[0030] S300: When the data type is a network protocol link with a clear URL, the network link is input into a URL fusion recognition model to obtain an abnormal URL recognition result output by the URL fusion recognition model; wherein the URL fusion recognition model includes at least one URL recognition module, the URL recognition module is used to identify the URL type of the network link, and the URL fusion recognition model fuses at least one URL type to obtain an abnormal URL recognition result.

[0031] Obtain sample network protocol links with clear URLs and sample abnormal URL recognition results. Obtain a preset URL fusion recognition model based on at least one preset URL recognition module. Label the sample network protocol links based on the sample abnormal URL recognition results to obtain URL protocol link training samples carrying labels. Input the URL protocol link training samples carrying labels into the preset URL fusion recognition model until the error of the predicted abnormal URL recognition result output by the preset URL fusion recognition model is less than the third set error, thereby obtaining the URL fusion recognition model.

[0032] When the data type is a network protocol link with a clear URL, the network link is input into the URL fusion recognition model to obtain the abnormal URL recognition result output by the URL fusion recognition model. The URL fusion recognition model includes at least one URL recognition module, each URL recognition module identifies a URL type of the network link. The URL fusion recognition model analyzes at least one URL type to obtain the final URL type of the network link. The URL fusion recognition model filters the final URL type according to the normal legal URL to obtain the abnormal URL recognition result of the final URL type.

[0033] The abnormal network link identification method provided in the embodiment of the present application obtains the abnormal application identification result of the application link by fusing the application type identified by at least one application identification module, thereby reducing the deviation of a single application identification module and improving the overall identification accuracy and overall identification efficiency of the abnormal application identification result. The abnormal website identification result of the network protocol link is obtained by fusing the website type identified by at least one website identification module, thereby reducing the deviation of a single website identification module and improving the overall identification accuracy and overall identification efficiency of the abnormal website identification result.

[0034] Based on the above embodiment, obtaining the data type of the network link includes S110 to S130, and each step is specifically as follows.

[0035] S110: When the domain name of the network link is clear but the URL is unclear, obtain an extracted feature set of the network link and a feature weight of each extracted feature in the extracted feature set, wherein the extracted features at least include the domain name length, a judgment result of whether the domain name is of IP type, domain name information entropy, the number of vowels in the domain name, the number of digits in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and a judgment result of whether the network link is a content distribution network.

[0036] S120: performing weighted summation on the multiple extracted features and the corresponding feature weights to obtain the feature value of the network link.

[0037] S130: normalizing the characteristic values ​​to obtain a risk score of the network link. If the risk score exceeds a set score, completing the URL of the network link to obtain the data type.

[0038] Pre-acquire multiple sample network links with clear domain names but unclear URLs. Obtain sample risk scores for the sample network links. Label the sample network links according to the labels of the sample risk scores to obtain labeled training samples. Train the preset model according to the labeled training samples until the output error of the preset model is less than the first set error, thereby obtaining a risk assessment model.

[0039] If the data analysis module determines that the domain name of the network link is clear but the URL is unclear, the network link is feature extracted through a pre-trained risk assessment model to obtain an extracted feature set of the network link.

[0040] The risk assessment model extracts features from the network link to obtain an extracted feature set, which includes multiple extracted features. The extracted features include, but are not limited to, the length of the domain name, the judgment result of whether the domain name is an IP type, the information entropy of the domain name, the number of vowels in the domain name, the number of digits in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and the judgment result of whether the network link is a content delivery network (CDN). Specific types of websites include customer service websites, banking websites, loan websites, download websites, warning websites, conference websites, dating websites, etc. The number of times the domain name appears in a specific time window of a specific type of website, including the DPI number of the domain name and the customer service website appearing in the same 5 minutes, 30 minutes, 1 hour..., the DPI number of the domain name and the banking website appearing in the same 5 minutes, 30 minutes, 1 hour..., etc. Calculate the risk score of the network link.

[0041] ; in, Score the risk of the network link. is the normalization function, is the number of features extracted from the feature set, For the Extract features, For the The weight of the extracted features.

[0042] Determining the weights of extracted features includes but is not limited to logistic regression, support vector machine, decision tree, random forest, gradient boosting tree, etc.

[0043] The risk assessment model performs a preliminary risk screening of the network link based on the extracted feature set and determines the risk score of the network link. When the risk score of the network link is greater than the set score, it indicates that the network link may be an abnormal network link. The network link's URL is completed and the data type of the network link after the completed URL is obtained.

[0044] Furthermore, if the domain name of the website link is clear, the website address is unclear, and the risk score of the network link is less than the set score, the network link is determined to be a normal network link.

[0045] This application obtains the risk score by weighted summing the extracted features in the extracted feature set, thereby improving the accuracy of obtaining the risk score. This application obtains the risk score by preliminarily screening the network links with unclear URLs through the extracted feature set, which can effectively filter out most of the normal network links at an early stage, and focus on more in-depth analysis of high-risk network links. At the same time, it also provides a reference for the subsequent completion of the URL of the network link.

[0046] Based on the above embodiment, the URL of the network link is completed to obtain the data type, including S131 to S135, and each step is specifically as follows.

[0047] S131: Based on the risk score, obtain the maximum number of crawler visits to the network link.

[0048] S132: Acquire an initial URL set of the network link, and acquire an initial crawler access count of the network link based on the number of initial URLs in the initial URL set.

[0049] S133: When the initial crawler access count is less than the maximum crawler access count, multiple associated URLs of the initial URL are obtained. If the domain name of the associated URL is the same as the domain name of the network link and the associated URL is not in the initial URL set, the associated URL is added to the initial URL set to update the initial URL set, and the initial crawler access count is increased by 1.

[0050] S134: When the initial crawler access count increases to the maximum crawler access count, or the initial URL set is no longer updated, the last updated initial URL set is used as the complete URL set of the network link.

[0051] S135: Acquire a data type for each completed URL in the completed URL set based on the network link.

[0052] If the risk score is greater than the set score, the network link is retained and the subsequent URL completion operation is performed on the network link. If the risk score is less than or equal to the set score, the network link is determined to be a normal network link and the normal network link is removed.

[0053] For network links with clear domain names but unclear URLs, such as HTTPS protocol links, the captured URI is incomplete (only the domain name), which has a great impact on the identification of abnormal network links. This is because many network protocol links cannot be judged as abnormal network links only by the domain name.

[0054] For web links with clear domain names but unclear URLs, crawler technology is used to obtain all the associated complete URLs under the domain name.

[0055] According to the risk score of the network link, the maximum number of crawler visits to the network link is calculated.

[0056] ; in, The risk score is The maximum number of crawler visits to a network link, To score the risk, is a position parameter that determines the point at which the risk score begins to have a significant impact on the maximum number of crawler visits. is the slope parameter, is the maximum number of crawler visits to the URL with the highest risk.

[0057] An initial URL set of a network link is obtained, and the initial URL set includes multiple initial URLs of the network link. The number of initial URLs is equal to the initial crawler access count of the network link. If the initial crawler access count of the network link is less than the maximum crawler access count, the associated URL of each initial URL is obtained, and the associated URL is added to the initial URL set to update the initial URL set, and further update the initial crawler access count of the initial URL set.

[0058] When the number of initial crawler visits to the network link increases to the maximum number of crawler visits or all associated URLs of the initial URLs are traversed, the update of the initial URL set is terminated. The last updated initial URL set is used as the supplementary URL set of the network link. According to a supplementary URL in the supplementary URL set, a data type of the network link is obtained. According to multiple supplementary URLs in the supplementary URL set, multiple data types of the network link are obtained. If the data type corresponding to the supplementary URL is an APP link, the supplementary URL and the network link (domain name) are input into the application fusion recognition model to obtain the abnormal application recognition result output by the application fusion recognition model. If the data type corresponding to the supplementary URL is a network protocol link, the supplementary URL and the network link (domain name) are input into the URL fusion recognition model to obtain the abnormal URL recognition result output by the URL fusion recognition model.

[0059] The present application obtains the maximum crawler access times according to the risk score, which is conducive to further obtaining the complete URL set of the network link. The URL of the network link is completed according to the maximum crawler access times, which improves the comprehensiveness of the complete URL set.

[0060] Based on the above embodiment, the URL fusion recognition model is used to fuse at least one URL type to obtain abnormal URL recognition results, including S310 to S350, and each step is specifically as follows.

[0061] S310: Obtain a first identification weight for each URL type, where the first identification weight is a weight used by a URL identification module to identify the URL type.

[0062] S320: Based on the first identification weight, obtain a first ranking score of each URL type of the network link. The greater the first identification weight, the greater the first ranking score.

[0063] S330: Adjusting the first ranking score based on the optimization configuration information of the URL identification module to obtain an adjusted first ranking score.

[0064] S340: Using the URL type corresponding to the highest adjusted first ranking point as the identified URL type of the network link.

[0065] S350: Based on the comparison result of the identified URL type and the set normal URL type, obtain the abnormal URL identification result of the network link.

[0066] When the data type of the network link is a network protocol link with a clear URL, or the data type of the network link of the completed URL is a network protocol link with a clear URL, the network link (or the completed URL and the network link) is input into the URL fusion recognition model. Each URL recognition module of the URL fusion recognition model performs URL recognition on the network link, and finds a matching preset URL type from the preset URL type set as a URL type of the network link.

[0067] Different URL identification modules analyze and identify network links from different perspectives (for example, URL identification module 1 may focus on website code structure analysis, URL identification module 2 may focus on domain name and IP address reputation analysis, URL identification module 3 may focus on website content analysis, etc.), and the output URL types may be different.

[0068] For a network link, a set of URL types is obtained through each URL identification module. The URL type set is converted to a unified format. The preset identification weight of each URL identification module for each preset URL type is obtained. The larger the preset identification weight, the higher the accuracy of the URL identification module in identifying the preset URL type. According to the preset identification weight of each URL identification module for each preset URL type, the first identification weight of each URL type after the unified format conversion is obtained. According to the first identification weight of each URL type, the first ranking score of each URL type is determined. For example, the first identification weight is directly used as the first ranking score. For example, the first identification weight is 0.9, and the first ranking score is 0.9.

[0069] The tuning configuration information of the URL identification module includes: for a network link, when the URL types identified by at least two URL identification modules are the same, the same URL type is obtained, and the first ranking score corresponding to the same URL type is adjusted to the highest first ranking score; for a network link, when the URL type identified by at least three URL identification modules is a set normal URL type, the first ranking score corresponding to the set normal URL type is adjusted to the highest first ranking score.

[0070] Set the URL recognition module tuning configuration information set {r1, r2, ..., r i , ..., r t}, the tuning configuration information of each URL recognition module i Defines classification decisions under certain conditions. Adjusts the first ranking score of each URL type based on the tuning configuration information of each URL recognition module.

[0071] For example, if URL identification module 1 and URL identification module 2 simultaneously mark a certain network link as investment and financial management category (same URL type), the first ranking score corresponding to the investment and financial management category is adjusted to the highest first ranking score (for example, 100 points), and further, the network link is marked as investment and financial management category.

[0072] If URL recognition module 1, URL recognition module 2 and URL recognition module 3 simultaneously mark a network link as a normal URL type, the first ranking score corresponding to the normal URL type is adjusted to the highest first ranking score, and further, the network link is marked as a normal URL type.

[0073] Furthermore, the tuning configuration information of the URL identification module also includes: if the network link first appears in the DPI data within 72 hours, the URL identification module 2 and the URL identification module 3 add 0.5 to the first ranking score of the specified preset URL type.

[0074] For each network link, the URL type corresponding to the highest adjusted first ranking is used as the identified URL type of the network link. If the identified URL type does not belong to the set normal URL type, the network link is determined to be an abnormal URL, and a tuple consisting of {network link, identified URL type} is obtained. If the identified URL type belongs to the set normal URL type, the network link is determined to be a normal URL. The abnormal URL identification result of the network link is stored in the database.

[0075] The present application obtains the first ranking score of each URL type through the first identification weight, thereby realizing a preliminary determination of the URL type of the network link. Adjusting the first ranking score according to the tuning configuration information of the URL identification module is conducive to further determining the accuracy of the URL type of the network link.

[0076] Based on the above embodiment, the application fusion identification model is used to fuse at least one application type to obtain abnormal application identification results, including S210 to S250, and each step is specifically as follows.

[0077] S210: Obtain a second identification weight for each application type, where the second identification weight is a weight used by an application identification module to identify the application type.

[0078] S220: Based on the second identification weight, obtain a second ranking score of the network link belonging to each application type, and the greater the second identification weight, the greater the second ranking score.

[0079] S230: Adjusting the second ranking score based on the tuning configuration information of the application identification module to obtain an adjusted second ranking score.

[0080] S240: Using the application type corresponding to the highest adjusted second ranking point as the identified application type of the network link.

[0081] S250: Based on the comparison result of the identified application type and the set normal application type, obtain the abnormal application identification result of the network link.

[0082] When the data type of the network link is an application (APP) link, or the data type of the network link of the completed URL is an APP link, the network link (or the completed URL and the network link) is input into the application fusion recognition model. Each application recognition module of the application fusion recognition model performs APP recognition on the network link, and finds a matching preset application type from the preset application type set as an application type of the network link.

[0083] Different application identification modules analyze and identify network links from different perspectives, and the output application types may be different.

[0084] For a network link, an application type set is obtained through each application identification module. The application type set is converted into a unified format. A preset identification weight of each preset application type is obtained from each application identification module. According to the preset identification weight of each preset application type by each application identification module, a second identification weight of each application type after the unified format conversion is obtained. According to the second identification weight of each application type, a second ranking score of each application type is determined.

[0085] Set the tuning configuration information set {r'1, r'2, ..., r' i , ..., r' t}, the tuning configuration information r' of each application identification module i The classification decision under certain conditions is defined. The second ranking score of each application type is adjusted according to the tuning configuration information of each application identification module.

[0086] For network links (APP links), the application type corresponding to the highest adjusted second ranking score is used as the identified application type of the network link. If the identified application type does not belong to the set normal application type, the network link is determined to be an abnormal network link, and a tuple consisting of {network link, identified application type} is obtained. If the identified application type belongs to the set normal application type, the network link is determined to be a normal APP. The abnormal application identification results of the network link are stored in the database.

[0087] The present application obtains the second ranking score of each application type through the second identification weight, thereby realizing a preliminary determination of the identification application type of the network link. Adjusting the second ranking score according to the tuning configuration information of the application identification module is conducive to further determining the accuracy of the identification application type of the APP link.

[0088] The abnormal network link identification device provided by the present application is described below. The abnormal network link identification device described below and the abnormal network link identification method described above can be referenced to each other.

[0089] like Figure 3 As shown, an abnormal network link identification device includes: a data type determination module 301, which is used to obtain the data type of the network link.

[0090] The abnormal application identification module 302 is used to input the network link into the application fusion identification model when the data type is an application link, and obtain the abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result.

[0091] The abnormal URL identification module 303 is used to input the network link into the URL fusion identification model when the data type is a network protocol link with a clear URL, and obtain the abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model includes at least one URL identification module, the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain the abnormal URL identification result.

[0092] The abnormal network link identification device provided in the embodiment of the present application obtains the abnormal application identification result of the application link by fusing the application type identified by at least one application identification module, thereby reducing the deviation of a single application identification module and improving the overall identification accuracy and overall identification efficiency of the abnormal application identification result. By fusing the URL type identified by at least one URL identification module, the abnormal URL identification result of the network protocol link is obtained, thereby reducing the deviation of a single URL identification module and improving the overall identification accuracy and overall identification efficiency of the abnormal URL identification result.

[0093] In one embodiment, the data type determination module 301 is used to: when the domain name of the network link is clear and the URL is unclear, obtain the extracted feature set of the network link and the feature weight of each extracted feature in the extracted feature set, the extracted features at least including the domain name length, the judgment result of whether the domain name is an IP type, the domain name information entropy, the number of vowels in the domain name, the number of digits in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and the judgment result of the network link being a content distribution network; perform weighted summation on multiple extracted features and corresponding feature weights to obtain a feature value of the network link; perform normalization on the feature value to obtain a risk score of the network link, and if the risk score exceeds the set score, complete the URL of the network link to obtain the data type.

[0094] In one embodiment, the data type determination module 301 is used to: obtain the maximum number of crawler visits to the network link based on the risk score; obtain the initial URL set of the network link, and obtain the initial crawler visit number of the network link based on the number of initial URLs in the initial URL set; when the initial crawler visit number is less than the maximum crawler visit number, obtain multiple associated URLs of the initial URL, if the domain name of the associated URL is the same as the domain name of the network link, and the associated URL is not in the initial URL set, add the associated URL to the initial URL set to update the initial URL set, and increase the initial crawler visit number by 1; when the initial crawler visit number increases to the maximum crawler visit number, or the initial URL set is no longer updated, use the last updated initial URL set as the complete URL set of the network link; and obtain the data type based on each complete URL in the complete URL set of the network link.

[0095] In one embodiment, the abnormal URL identification module 303 is used to: obtain a first identification weight for each URL type, where the first identification weight is the weight of the URL identification module for identifying the URL type; based on the first identification weight, obtain a first ranking score of the network link belonging to each URL type, and the larger the first identification weight, the larger the first ranking score; based on the tuning configuration information of the URL identification module, adjust the first ranking score to obtain an adjusted first ranking score; use the URL type corresponding to the highest adjusted first ranking score as the identified URL type of the network link; based on the comparison result of the identified URL type and the set normal URL type, obtain the abnormal URL identification result of the network link.

[0096] In one embodiment, the abnormal application identification module 302 is used to: obtain a second identification weight for each application type, where the second identification weight is the weight of the application identification module that identifies the application type to identify the application type; based on the second identification weight, obtain a second ranking score for each application type of the network link, and the larger the second identification weight, the larger the second ranking score; based on the tuning configuration information of the application identification module, adjust the second ranking score to obtain an adjusted second ranking score; use the application type corresponding to the highest adjusted second ranking score as the identified application type of the network link; based on the comparison result of the identified application type and the set normal application type, obtain the abnormal application identification result of the network link.

[0097] In one embodiment, the tuning configuration information of the URL identification module includes: for a network link, when the URL types identified by at least two URL identification modules are the same, the same URL type is obtained, and the first ranking score corresponding to the same URL type is adjusted to the highest first ranking score; for a network link, when the URL type identified by at least three URL identification modules is a set normal URL type, the first ranking score corresponding to the set normal URL type is adjusted to the highest first ranking score.

[0098] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the abnormal network link identification method, which includes: obtaining the data type of the network link; when the data type is an application link, inputting the network link into the application fusion identification model, and obtaining the abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result; when the data type is a network protocol link with a clear URL, inputting the network link into the URL fusion identification model, and obtaining the abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model includes at least one URL identification module, the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain the abnormal URL identification result.

[0099] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0100] On the other hand, the present application also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the abnormal network link identification method provided by the above methods, which includes: obtaining the data type of the network link; when the data type is an application link, inputting the network link into an application fusion identification model, and obtaining the abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, and the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain the abnormal application identification result; when the data type is a network protocol link with a clear URL, the network link is input into a URL fusion identification model, and the abnormal URL identification result output by the URL fusion identification model is obtained; wherein the URL fusion identification model includes at least one URL identification module, and the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain the abnormal URL identification result.

[0101] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the abnormal network link identification method provided by the above-mentioned methods, the method comprising: obtaining a data type of the network link; when the data type is an application link, inputting the network link into an application fusion identification model, and obtaining an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model comprises at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one application type to obtain an abnormal application identification result; when the data type is a network protocol link with a clear URL, inputting the network link into a URL fusion identification model, and obtaining an abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model comprises at least one URL identification module, the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one URL type to obtain an abnormal URL identification result.

[0102] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying abnormal network links, characterized in that: include: Get the data type of the network link; When the data type is an application link, the network link is input into an application fusion identification model to obtain an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used to identify the application type of the network link, and the application fusion identification model fuses at least one of the application types to obtain the abnormal application identification result; When the data type is a network protocol link with a clear URL, the network link is input into a URL fusion recognition model to obtain an abnormal URL recognition result output by the URL fusion recognition model; wherein the URL fusion recognition model includes at least one URL recognition module, and the URL recognition module is used to identify the URL type of the network link, and the URL fusion recognition model fuses at least one of the URL types to obtain the abnormal URL recognition result.

2. The abnormal network link identification method according to claim 1, characterized in that: The data type of the network link obtained includes: When the domain name of the network link is clear and the website address is unclear, obtaining an extraction feature set of the network link and a feature weight of each extraction feature of the extraction feature set, wherein the extraction feature at least includes the domain name length, a judgment result of whether the domain name is an IP type, domain name information entropy, the number of vowels in the domain name, the number of digits in the domain name, the number of times the domain name appears in a specific time window of a specific type of website, and a judgment result of whether the network link is a content distribution network; Performing weighted summation on a plurality of the extracted features and the corresponding feature weights to obtain a feature value of the network link; The characteristic value is normalized to obtain a risk score of the network link. If the risk score exceeds a set score, the URL of the network link is completed to obtain the data type.

3. The abnormal network link identification method according to claim 2, characterized in that: The completing the URL of the network link to obtain the data type includes: Based on the risk score, obtaining a maximum number of crawler visits to the network link; Acquire an initial URL set of the network link, and acquire an initial crawler access count of the network link based on the number of initial URLs in the initial URL set; When the initial crawler access count is less than the maximum crawler access count, obtaining multiple associated URLs of the initial URL, and if the domain name of the associated URL is the same as the domain name of the network link, and the associated URL is not in the initial URL set, adding the associated URL to the initial URL set to update the initial URL set, and increasing the initial crawler access count by 1; When the initial crawler access count increases to the maximum crawler access count, or the initial URL set is no longer updated, the last updated initial URL set is used as the complete URL set of the network link; Based on each completed URL in the completed URL set of the network link, the data type is obtained.

4. The abnormal network link identification method according to claim 1, characterized in that: The URL fusion recognition model is used to fuse at least one of the URL types to obtain the abnormal URL recognition result, including: Obtaining a first identification weight of each of the URL types, where the first identification weight is a weight of the URL type identified by a URL identification module that identifies the URL type; Based on the first identification weight, obtaining a first ranking score of each of the URL types for the network link, wherein the greater the first identification weight, the greater the first ranking score; Adjusting the first ranking score based on the optimization configuration information of the URL identification module to obtain an adjusted first ranking score; The URL type corresponding to the highest adjusted first ranking point is used as the identification URL type of the network link; Based on the comparison result of the identified URL type and the set normal URL type, the abnormal URL identification result of the network link is obtained.

5. The abnormal network link identification method according to claim 1, characterized in that: The application fusion identification model is used to fuse at least one of the application types to obtain the abnormal application identification result, including: Obtaining a second identification weight for each of the application types, where the second identification weight is a weight for an application identification module that identifies the application type to identify the application type; Based on the second identification weight, obtaining a second ranking score of each application type for which the network link belongs, wherein the greater the second identification weight, the greater the second ranking score; Adjusting the second ranking score based on the tuning configuration information of the application identification module to obtain an adjusted second ranking score; using the application type corresponding to the highest adjusted second ranking point as the identified application type of the network link; Based on the comparison result of the identified application type and the set normal application type, the abnormal application identification result of the network link is obtained.

6. The abnormal network link identification method according to claim 4, characterized in that: The tuning configuration information of the website identification module includes: For one of the network links, when the URL types identified by at least two of the URL identification modules are the same, the same URL type is obtained, and the first ranking score corresponding to the same URL type is adjusted to the highest first ranking score; For one of the network links, when the URL types identified by at least three of the URL identification modules are the set normal URL types, the first ranking score corresponding to the set normal URL type is adjusted to the highest first ranking score.

7. An abnormal network link identification device, characterized in that: include: A data type determination module, used to obtain the data type of the network link; an abnormal application identification module, used for, when the data type is an application link, inputting the network link into an application fusion identification model, and obtaining an abnormal application identification result output by the application fusion identification model; wherein the application fusion identification model includes at least one application identification module, the application identification module is used for identifying the application type of the network link, and the application fusion identification model fuses at least one of the application types to obtain the abnormal application identification result; The abnormal URL identification module is used to input the network link into the URL fusion identification model when the data type is a network protocol link with a clear URL, and obtain the abnormal URL identification result output by the URL fusion identification model; wherein the URL fusion identification model includes at least one URL identification module, and the URL identification module is used to identify the URL type of the network link, and the URL fusion identification model fuses at least one of the URL types to obtain the abnormal URL identification result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the abnormal network link identification method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the abnormal network link identification method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the abnormal network link identification method according to any one of claims 1 to 6 is implemented.

Images