A Dark Link Detection Method, Device, Equipment and Storage Medium

Abstract

The present application discloses a dark link detection method, device, equipment and storage medium, relating to the field of network security technology. The method includes: obtaining the access log of a website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation; extracting the uniform resource locators of search engines from the access log based on the access source IP to obtain a corresponding list of uniform resource locators; using preset rules to group the list of uniform resource locators to obtain several groups, and determining the semantic readability of the uniform resource locators in the several groups; detecting a target group with suspicious dark links from the several groups based on the semantic readability of the uniform resource locators, and performing corresponding processing on the suspicious dark links in the target group. It can be seen that through analyzing and accessing the access log, the present application can detect whether there are dark links on the website more comprehensively and pertinently, and improve the detection rate.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular, to a dark link detection method, device, equipment and storage medium. Background Art

[0002] A dark link, also known as a "black link" or "hidden link", refers to an external link that is invisible but can be recognized by a search engine and has its weight calculated. An intruder can implant a dark link in a web page through illegal technology, aiming to improve the search engine ranking of the website pointed to by the dark link and make a profit therefrom. The implantation of dark links not only affects the normal operation of the website, but also spreads a large amount of illegal information to the public, causing great harm. At present, the detection of web page dark links all starts directly from the content of the web page itself. These methods all have certain advantages and disadvantages, and due to the inability to obtain the web page link address or the lack of feature rules in the page, there are problems of missed reports, making it difficult to achieve comprehensive coverage. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a dark link detection method, device, equipment and storage medium, which can detect whether a website has dark links more comprehensively and targeted, and can improve the detection rate. The specific scheme is as follows:

[0004] In a first aspect, the present application provides a dark link detection method, including:

[0005] Obtain the access log of a website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation;

[0006] Extract the uniform resource locator of the search engine from the access log based on the access source IP to obtain a corresponding list of uniform resource locators;

[0007] Use preset rules to group the list of uniform resource locators to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups;

[0008] Detect the target group with suspected dark links from the several groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspected dark links in the target group.

[0009] Optionally, the extracting the uniform resource locator of the search engine from the access log based on the access source IP to obtain a corresponding list of uniform resource locators includes:

[0010] Match the threat intelligence with the access source IP to match the target IP in the threat intelligence from the access source IP, and use the target IP to extract the uniform resource locators of search engines from the access log to obtain a corresponding list of uniform resource locators.

[0011] Optionally, the extracting the uniform resource locators of search engines from the access log based on the access source IP to obtain a corresponding list of uniform resource locators includes:

[0012] Extract the uniform resource locators of search engines from the access log based on the access source IP to obtain initial uniform resource locators;

[0013] Perform standardization processing and deduplication processing on the initial uniform resource locators in sequence to obtain a corresponding list of uniform resource locators.

[0014] Optionally, the performing standardization processing and deduplication processing on the initial uniform resource locators in sequence to obtain a corresponding list of uniform resource locators includes:

[0015] Perform data standardization processing on the initial uniform resource locators by deleting the parameters and values related to query operations in the path part of the initial uniform resource locators to obtain standardized uniform resource locators;

[0016] Perform deduplication operations on the standardized uniform resource locators, and sort the deduplicated uniform resource locators in alphabetical order to obtain sorted uniform resource locators;

[0017] Delete the uniform resource locators with the number of characters less than the first preset number from the sorted uniform resource locators to obtain a corresponding list of uniform resource locators.

[0018] Optionally, the grouping the list of uniform resource locators using preset rules to obtain several groups includes:

[0019] Extract several characters in front of each uniform resource locator in the list of uniform resource locators as the prefix of the uniform resource locator;

[0020] Group the uniform resource locators with the same prefix into the same group, and discard the groups with the number of uniform resource locators in the group less than the preset value to obtain several groups corresponding to different prefixes.

[0021] Optionally, the determining the semantic readability of the uniform resource locators in the several groups includes:

[0022] Remove the prefix of the uniform resource locator in the several groups, segment the remaining part of the uniform resource locator, and mark the uniform resource locators with the number of segments not less than the second preset number as semantically readable.

[0023] Optionally, detecting a target group with a suspicious dark link from the several groups based on the semantic readability of the uniform resource locator includes:

[0024] Detect the uniform resource locators marked as semantically readable in the several groups, and determine whether the number of uniform resource locators marked as semantically readable in each group exceeds one;

[0025] If not, it is determined that there is a suspicious dark link in the corresponding group.

[0026] In a second aspect, the present application provides a dark link detection device, including:

[0027] A log acquisition module, configured to acquire an access log of a website within a preset time; the access log includes an access source IP and a uniform resource locator related to an access operation;

[0028] An extraction module, configured to extract the uniform resource locators of search engines from the access log based on the access source IP to obtain a corresponding list of uniform resource locators;

[0029] A grouping module, configured to group the list of uniform resource locators by using a preset rule to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups;

[0030] A dark link detection module, configured to detect a target group with a suspicious dark link from the several groups based on the semantic readability of the uniform resource locator, and perform corresponding processing on the suspicious dark link in the target group.

[0031] In a third aspect, the present application provides an electronic device, the electronic device includes a processor and a memory; wherein, the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the foregoing dark link detection method.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium, and the computer program implements the foregoing dark link detection method when executed by a processor.

[0033] When this application conducts dark link detection, it first obtains the access logs of the website within a preset time; the access logs include the access source IP and the uniform resource locator related to the access operation; based on the access source IP, it extracts the uniform resource locators of search engines from the access logs to obtain a corresponding list of uniform resource locators; it uses preset rules to group the list of uniform resource locators to obtain several groups, and determines the semantic readability of the uniform resource locators in the several groups; based on the semantic readability of the uniform resource locators, it detects the target group with suspicious dark links from the several groups, and performs corresponding processing on the suspicious dark links in the target group. It can be seen that this application extracts the suspicious dark link uniform resource locators through the access logs of search engines, analyzes the data of the website's access logs, identifies the abnormal drainage behavior of orphan links after dark link implantation, so as to identify the web pages where the website has been implanted with dark links. In this way, it can comprehensively and quickly detect whether a website has been implanted with dark links, and is particularly effective for the detection of orphan links. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0035] Figure 1 It is a flowchart of a dark link detection method provided by this application;

[0036] Figure 2 It is a schematic diagram of manual access to review suspicious dark links provided by this application;

[0037] Figure 3 It is a flowchart of a dark link detection method provided by this application;

[0038] Figure 4 It is a flowchart of a specific dark link detection method provided by this application;

[0039] Figure 5 It is a flowchart of a specific dark link detection method provided by this application;

[0040] Figure 6 It is a schematic diagram of the structure of a dark link detection device provided by this application;

[0041] Figure 7 It is a structural diagram of an electronic device provided by this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] In the prior art, current web page dark link detection is directly triggered from the content of the web page itself. These methods have certain advantages and disadvantages, but it is very difficult to achieve comprehensive coverage. To solve this technical problem, the present application provides a dark link detection method that can avoid the false negative problem caused by the inability to obtain the web page link address or the lack of feature rules in the page.

[0044] See Figure 1 As shown, an embodiment of the present invention discloses a dark link detection method, including:

[0045] Step S11: Obtain the access log of the website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation.

[0046] In this embodiment, the present application detects the web page with dark links based on the behavior analysis of the access log, so it is necessary to obtain the access log of the website to be inspected for analysis. For example, obtaining the access log of the website for one day, through the analysis and access of the access log, can detect whether there are dark links on this website more comprehensively and specifically, and can improve the detection rate. Especially for the implantation of single dark links, it can also be effectively detected.

[0047] Step S12: Extract the uniform resource locators of search engines from the access log based on the access source IP to obtain a corresponding list of uniform resource locators.

[0048] In this embodiment, threat intelligence is matched with the access source IP to match the target IP in the threat intelligence from the access source IP, and the access uniform resource locator of the search engine is extracted. Then, the uniform resource locator of the search engine is extracted from the access log using the target IP, and the initial uniform resource locator is sequentially subjected to standardization processing and deduplication processing to obtain a corresponding list of uniform resource locators. It can be understood that a search engine is a retrieval technology that retrieves specified information from the Internet according to user needs and certain algorithms and feeds it back to the user using specific strategies. Search engines rely on a variety of technologies, such as web crawler technology, retrieval sorting technology, web page processing technology, big data processing technology, natural language processing technology, etc., to provide fast and highly relevant information services for information retrieval users. The core modules of search engine technology generally include crawlers, indexes, retrievals, and sorting, etc. At the same time, a series of other auxiliary modules can be added to create a better network usage environment for users. The search engine will analyze and process the content in the web page and calculate the weight. According to the searched content, the web page content with a higher weight will be displayed first. At this time, hidden links will be implanted into some websites with a higher weight, and when searching in the search engine, the content related to the hidden links will be displayed.

[0049] Step S13: Group the list of uniform resource locators using a preset rule to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups.

[0050] In this embodiment, the first N characters in front of each uniform resource locator in the list of uniform resource locators are extracted as prefixes. In a specific embodiment, " / poc", " / xiazai", and " / xwzx" are respectively extracted. For the group with " / poc" as the prefix, there is only one URL in the group, so this group is discarded. The following groups are obtained, and the uniform resource locators with the same prefix are divided into the same group to obtain several groups.

[0051] Prefix group of " / xiazai"

[0052] / xiazai / yy1t7rzxw7.php / xiazai0fkvgbays9.shtml / xiazai29963 / 56509.phtml / xiazai38ssu0zwoo.asp

[0053] / xiazai4tk7a.asp

[0054] / xiazai61fuyzy2f0.asp

[0055] / xiazai65594 / 79380 /

[0056] / xiazai6smn3f5qz1.asp

[0057] / xiazai73574 / 60474.html / xiazai74105cxcp0.phtml / xiazaib8ow90vkrf.asp

[0058] / xiazaidni7h.asp

[0059] / xiazaidr62i.asp

[0060] / xiazaihdr6bxo5va.php

[0061] / xiazaiivbuz.asp

[0062] / xiazaij7q921x0vt.asp

[0063] / xiazaimjmbvk2ots.asp

[0064] / xiazaio74umfxncv.asp

[0065] / xiazairxgoo.asp

[0066] / xiazaitn8kzh7r9u.asp

[0067] / xiazaitszni.asp

[0068] / xiazaiuxesx.asp

[0069] / xiazaiwhd91hwmhw.asp “ / xwzx” prefix group

[0070] / xwzx0h63qh1atm.htm

[0071] / xwzx0phu0fiz87.asp

[0072] / xwzx415ll.asp

[0073] / xwzx50261 / 69755.phtml / xwzx6181796433.asp

[0074] / xwzx8ghqon1arx.xls

[0075] / xwzxaxdwm.php

[0076] / xwzxb78fs8rzr6.asp

[0077] / xwzxbuezsl54xa.asp

[0078] / xwzxgksl78khsj.asp

[0079] / xwzxhahfhibmcc.asp

[0080] / xwzxmepz3

[0081] / xwzxvzv2u9vfnw.doc

[0082] / xwzxwhx6viwldy.asp

[0083] / xwzxz0pcu681la.asp

[0084] After that, determine the semantic readability of the uniform resource locators in the several groups. It can be understood that for any string composed of English words, its semantics are readable, that is, people can read and understand it well. For any string randomly generated by a system, its semantics are unreadable and cannot be understood by people. If a string cannot be read and understood by people, it can be called semantically unreadable. And when calculating the semantic similarity, the method of word segmentation is not applicable, and machine learning models and algorithms that can achieve the same effect can also be used for calculation.

[0085] Step S14: Detect a target group with suspicious dark links from the several groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspicious dark links in the target group.

[0086] In this embodiment, if the number of semantically readable URLs (uniform resource locators) in a group does not exceed 1, it is considered that the URLs in this group are a list of suspicious dark links, and the suspicious dark links are manually visited and audited and processed. For example Figure 2As shown, it is manually determined that the URL is content promoting the gambling tool "slot machine", and it is determined to be an implanted hidden link. For these orphan links that cannot be crawled from the home page, search engines cannot crawl them under normal circumstances, so they cannot obtain weights. Therefore, attackers will link to these websites through a third-party website, enabling search engines to crawl these web pages, but normal users cannot access them. To further increase the weight, a large number of randomly generated script files will be uploaded to the victim's website when implanting hidden links. Therefore, when introducing links on the third-party website, a large number of orphan links will also be introduced, and search engines will also generate a large number of similar access behaviors. It can be understood that the orphan link is an older way of implanting hidden links, which directly adds keywords to the original web page, but this is easily detected by web crawler-based web crawling detection technology, so the black and gray industries have upgraded their countermeasures. They directly upload or add web pages to the original website, and the newly added web pages cannot be crawled through all the internal links within the website. Such web pages are called orphan links. In this way, through the process shown in Figure 3 it is possible to identify the abnormal drainage behavior of orphan links after detecting the implantation of hidden links by analyzing the data in the access log of the website, so as to identify the web pages where hidden links are implanted in the website.

[0087] As can be seen from the above, when detecting hidden links in this application, first obtain the access log of the website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation; extract the uniform resource locator of the search engine from the access log based on the access source IP to obtain a corresponding list of uniform resource locators; use preset rules to group the list of uniform resource locators to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups; detect the target group with suspicious hidden links from the several groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspicious hidden links in the target group. It can be seen that this application extracts the suspicious hidden link uniform resource locators through the access log of the search engine, analyzes the data in the access log of the website, identifies the abnormal drainage behavior of orphan links after detecting the implantation of hidden links, so as to identify the web pages where hidden links are implanted in the website. In this way, it is possible to comprehensively and quickly detect whether a website has been implanted with hidden links, and it is also particularly effective for detecting orphan links.

[0088] See Figure 4 As shown, an embodiment of the present invention discloses a specific hidden link detection method, including:

[0089] Step S21: Extract the uniform resource locator of the search engine from the access log based on the access source IP to obtain an initial uniform resource locator.

[0090] Step S22: Perform data standardization processing on the initial uniform resource locator by deleting the parameters and values related to the query operation in the path part of the initial uniform resource locator, so as to obtain the standardized uniform resource locator.

[0091] In this embodiment, first extract the path part in the URL and delete the query parameters and values in the path to complete the operation of performing data standardization processing on the initial uniform resource locator.

[0092] Step S23: Perform a duplicate removal operation on the standardized uniform resource locator, and sort the duplicate-removed uniform resource locators in alphabetical order to obtain the sorted uniform resource locator.

[0093] In this embodiment, perform a duplicate removal operation on the standardized uniform resource locator and sort it in alphabetical order to obtain the sorted uniform resource locator, as follows:

[0094] / poczcdc9wosln.shtml

[0095] / xiazai / yy1t7rzxw7.php

[0096] / xiazai0fkvgbays9.shtml

[0097] / xiazai29963 / 56509.phtml

[0098] / xiazai38ssu0zwoo.asp

[0099] / xiazai4tk7a.asp

[0100] / xiazai61fuyzy2f0.asp

[0101] / xiazai65594 / 79380 / / xiazai6smn3f5qz1.asp

[0102] / xiazai73574 / 60474.html

[0103] / xiazai74105cxcp0.phtml

[0104] / xiazaib8ow90vkrf.asp

[0105] / xiazaidni7h.asp

[0106] / xiazaidr62i.asp

[0107] / xiazaihdr6bxo5va.php

[0108] / xiazaiivbuz.asp

[0109] / xiazaij7q921x0vt.asp

[0110] / xiazaimjmbvk2ots.asp

[0111] / xiazaio74umfxncv.asp

[0112] / xiazairxgoo.asp

[0113] / xiazaitn8kzh7r9u.as

[0114] p

[0115] / xiazaitszni.asp

[0116] / xiazaiuxesx.asp

[0117] / xiazaiwhd91hwmhw.as

[0118] p

[0119] / xwzx0h63qh1atm.htm

[0120] / xwzx0phu0fiz87.asp

[0121] / xwzx415ll.asp

[0122] / xwzx50261 / 69755.pht

[0123] ml

[0124] / xwzx6181796433.asp

[0125] / xwzx8ghqon1arx.xls

[0126] / xwzxaxdwm.php

[0127] / xwzxb78fs8rzr6.asp

[0128] / xwzxbuezsl54xa.asp

[0129] / xwzxgksl78khsj.asp

[0130] / xwzxhahfhibmcc.asp

[0131] / xwzxmepz3

[0132] / xwzxvzv2u9vfnw.doc

[0133] / xwzxwhx6viwldy.asp

[0134] / xwzxz0pcu681la.asp

[0135] Step S24: Delete the URLs whose number of characters is less than the first preset number from the sorted URLs to obtain a corresponding URL list.

[0136] In this embodiment, in a specific embodiment, URLs with less than 8 characters can be deleted, and finally a sorted URL list is obtained, that is, the uniform resource locators with less than 8 characters are deleted from the sorted uniform resource locators to obtain the corresponding uniform resource locator list.

[0137] As can be seen from the above, this application can more clearly detect abnormal URLs by performing operations such as data standardization and deduplication on URLs, identify the abnormal traffic-generating behavior of isolated links after dark link implantation, and thus identify the website where the dark link page has been implanted.

[0138] See also Figure 5 As shown, the embodiment of the present invention discloses a specific dark link detection method, including:

[0139] Step S31, obtaining the access log of the website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation.

[0140] Step S32: extract the search engine's uniform resource locator from the access log based on the access source IP to obtain a corresponding uniform resource locator list.

[0141] Step S33: extracting a number of characters in front of each of the uniform resource locators in the uniform resource locator list as a prefix of the uniform resource locator.

[0142] Step S34: group the uniform resource locators with the same prefix into the same group, and discard the group in which the number of uniform resource locators in the group is less than a preset value, so as to obtain a plurality of groups corresponding to different prefixes.

[0143] Step S35: remove the prefixes of the uniform resource locators in the plurality of groups, perform word segmentation on the remaining portion of the uniform resource locator, and mark the uniform resource locator whose number of word segments is not less than a second preset number as semantically readable.

[0144] In this embodiment, after removing the prefix of the URL within the group, the remaining part of the URL is segmented, and the URL with the number of segments >= 2 is marked as semantically readable. That is, the second preset number can be 2, or it can be set by itself according to the actual situation. In a specific embodiment, in the above-mentioned group, within the prefix group of " / xiazai", after removing " / xiazai", no effective segmentation can be performed on the remaining URL, including Chinese pinyin and English phrase segmentation. Therefore, the number of semantically readable URLs in this group is 0. Within the prefix group of " / xwzx", after removing " / xiazai", no effective segmentation can be performed on the remaining URL, including Chinese pinyin and English phrase segmentation. Therefore, the number of semantically readable URLs in this group is 0.

[0145] Step S36: Detect the uniform resource locators marked as semantically readable in the several groups, and determine whether the number of uniform resource locators marked as semantically readable in each group exceeds one.

[0146] Step S37: If not, determine that there are suspicious dark links in the corresponding group.

[0147] In this embodiment, in a specific embodiment, as shown in the above embodiment, the number of semantically readable URLs in the " / xiazai" and " / xwzx" groups after grouping is 0, that is, whether the number of uniform resource locators marked as semantically readable in the group exceeds one. Then all the URLs in the group are determined to be implanted with dark links.

[0148] Among them, the specific process of the above step S28 can refer to the corresponding content disclosed in the foregoing embodiment, and will not be elaborated here.

[0149] As can be seen from the above, this application extracts the suspicious dark link uniform resource locators through the access logs of the search engine, analyzes the data of the access logs of the website, and identifies the abnormal drainage behavior of the orphan link after the dark link is implanted, so as to identify the web page implanted with the dark link. In this way, it is possible to comprehensively and quickly detect whether a website is implanted with a dark link, and it is also particularly effective for the detection of orphan links.

[0150] See Figure 6 As shown, an embodiment of this application discloses a dark link detection device, including:

[0151] A log acquisition module 11, configured to acquire the access log of the website within a preset time; the access log includes the access source IP and the uniform resource locator related to the access operation;

[0152] An extraction module 12, configured to extract the uniform resource locator of the search engine from the access log based on the access source IP to obtain a corresponding list of uniform resource locators;

[0153] A grouping module 13, configured to group the uniform resource locator list according to a preset rule to obtain a plurality of groups, and determine the semantic readability of the uniform resource locators in the plurality of groups;

[0154] A dark link detection module 14, configured to detect a target group with suspicious dark links from the plurality of groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspicious dark links in the target group.

[0155] When performing dark link detection in this application, first obtain the access log of a website within a preset time; the access log includes the access source IP and the uniform resource locators related to the access operation; extract the uniform resource locators of search engines from the access log based on the access source IP to obtain a corresponding uniform resource locator list; group the uniform resource locator list according to a preset rule to obtain a plurality of groups, and determine the semantic readability of the uniform resource locators in the plurality of groups; detect a target group with suspicious dark links from the plurality of groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspicious dark links in the target group. It can be seen that this application extracts the suspicious dark link uniform resource locators through the access log of the search engine, analyzes the data of the access log of the website, identifies the abnormal drainage behavior of the orphan link after the dark link is implanted, so as to identify the web page where the dark link is implanted in the website. In this way, it is possible to comprehensively and quickly detect whether a website has been implanted with a dark link, and it is also particularly effective for the detection of orphan links.

[0156] In some specific embodiments, the extraction module 12 specifically includes:

[0157] A uniform resource locator extraction unit, configured to match threat intelligence by using the access source IP to match a target IP located in the threat intelligence from the access source IP, and extract the uniform resource locators of search engines from the access log by using the target IP to obtain a corresponding uniform resource locator list.

[0158] An initial uniform resource locator acquisition unit, configured to extract the uniform resource locators of search engines from the access log based on the access source IP to obtain an initial uniform resource locator;

[0159] A processing unit, configured to perform standardization processing and deduplication processing on the initial uniform resource locator in sequence to obtain a corresponding uniform resource locator list.

[0160] In some specific embodiments, the extraction module 12 specifically includes:

[0161] A normalization processing unit for performing data normalization processing on the initial uniform resource locator by deleting the parameters and values related to the query operation in the path part of the initial uniform resource locator to obtain a normalized uniform resource locator;

[0162] A sorting unit for performing a duplicate removal operation on the normalized uniform resource locator and sorting the duplicate-removed uniform resource locators in alphabetical order to obtain a sorted uniform resource locator;

[0163] A deletion unit for deleting the uniform resource locators with a character count less than a first preset number from the sorted uniform resource locators to obtain a corresponding list of uniform resource locators.

[0164] In some specific embodiments, the grouping module 13 specifically includes:

[0165] A semantic-readable marking unit for removing the prefix of the uniform resource locator in the several groups, performing word segmentation on the remaining part of the uniform resource locator, and marking the uniform resource locators with a word segmentation count not less than a second preset number as semantically readable.

[0166] In some specific embodiments, the dark link detection module 14 specifically includes:

[0167] A judgment unit for detecting the uniform resource locators marked as semantically readable in the several groups and judging whether the number of uniform resource locators marked as semantically readable in each group exceeds one;

[0168] A dark link determination unit for determining that there is a suspicious dark link in the corresponding group if it does not exceed.

[0169] Furthermore, an electronic device is also disclosed in an embodiment of the present application. Figure 7 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the scope of use of the present application.

[0170] Figure 7 It is a structural schematic diagram of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the dark link detection method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0171] In this embodiment, the power supply 23 is used to provide operating voltages for the various hardware devices on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and specific limitations thereto are not provided herein; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type thereof can be selected according to specific application requirements, and no specific limitations are provided herein.

[0172] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0173] Among them, the operating system 221 is used to manage and control the various hardware devices and the computer program 222 on the electronic device 20, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of implementing the dark link detection method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of performing other specific tasks.

[0174] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed dark link detection method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0175] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0176] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0177] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0178] Finally, it should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0179] The technical solutions provided in this application have been introduced in detail above. Specific examples are used herein to illustrate the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A dark link detection method, characterized in that, Including: Obtain the access logs within the preset time of the website; the access logs include the access source IP and the uniform resource locator related to the access operation; Extract the uniform resource locators of the search engines from the access logs based on the access source IP to obtain a corresponding list of uniform resource locators; Use preset rules to group the list of uniform resource locators to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups; Detect the target group with suspicious hidden links from the several groups based on the semantic readability of the uniform resource locators, and perform corresponding processing on the suspicious hidden links in the target group.

2. The dark link detection method according to claim 1, wherein The extracting the uniform resource locators of the search engines from the access logs based on the access source IP to obtain a corresponding list of uniform resource locators includes: Match the threat intelligence using the access source IP to match the target IP in the threat intelligence from the access source IP, and extract the uniform resource locators of the search engines from the access logs using the target IP to obtain a corresponding list of uniform resource locators.

3. The dark link detection method according to claim 1, wherein, The extracting the uniform resource locators of the search engines from the access logs based on the access source IP to obtain a corresponding list of uniform resource locators includes: Extract the uniform resource locators of the search engines from the access logs based on the access source IP to obtain the initial uniform resource locators; Perform standardization processing and deduplication processing on the initial uniform resource locators in sequence to obtain a corresponding list of uniform resource locators.

4. The dark link detection method according to claim 3, wherein The performing standardization processing and deduplication processing on the initial uniform resource locators in sequence to obtain a corresponding list of uniform resource locators includes: Perform data standardization processing on the initial uniform resource locators by deleting the parameters and values related to the query operation in the path part of the initial uniform resource locators to obtain the standardized uniform resource locators; Perform deduplication operation on the standardized uniform resource locators, and sort the deduplicated uniform resource locators in alphabetical order to obtain the sorted uniform resource locators; Delete the uniform resource locators with the number of characters less than the first preset number from the sorted uniform resource locators to obtain a corresponding list of uniform resource locators.

5. The dark link detection method according to any one of claims 1 to 4, characterized in that, The using preset rules to group the list of uniform resource locators to obtain several groups includes: Extract several characters in front of each uniform resource locator in the list of uniform resource locators as the prefix of the uniform resource locator; Divide the uniform resource locators with the same prefix into the same group, and discard the groups with the number of uniform resource locators within the group less than the preset value to obtain several groups corresponding to different prefixes.

6. The dark link detection method according to claim 5, wherein The determining the semantic readability of the uniform resource locators in the several groups includes: Remove the prefix of the uniform resource locator in the several groups, perform word segmentation on the remaining part of the uniform resource locator, and mark the uniform resource locators with the number of word segments not less than the second preset number as semantically readable.

7. The dark link detection method according to claim 6, characterized in that, Detecting a target group with a suspicious dark link from the several groups based on the semantic readability of the uniform resource locator includes: Detect the uniform resource locators marked as semantically readable in the several groups, and determine whether the number of uniform resource locators marked as semantically readable in each group exceeds one; If not, it is determined that there is a suspicious dark link in the corresponding group.

8. An invisible link detection device, characterized in that, Includes: A log acquisition module, configured to acquire access logs within a preset time of a website; the access logs include the access source IP and the uniform resource locator related to the access operation; An extraction module, configured to extract the uniform resource locators of search engines from the access logs based on the access source IP to obtain a corresponding list of uniform resource locators; A grouping module, configured to group the list of uniform resource locators by using a preset rule to obtain several groups, and determine the semantic readability of the uniform resource locators in the several groups; A dark link detection module, configured to detect a target group with a suspicious dark link from the several groups based on the semantic readability of the uniform resource locator, and perform corresponding processing on the suspicious dark link in the target group.

9. An electronic device, characterized in that, Includes: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the dark link detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program, when executed by a processor, implements the dark link detection method according to any one of claims 1 to 7.

Images