A black link identification method, device, equipment and storage medium

Abstract

The present application discloses a method, device, equipment and storage medium for identifying black links, and relates to the field of computer security technology. The method includes: obtaining the uniform resource locator of a website to be detected, and parsing the uniform resource locator to obtain the web page source code of the website to be detected; extracting content information in a preset attribute tag in the web page source code, and extracting repeated phrases in the content information based on the suffix tree structure of the content information and preset extraction conditions; sorting the repeated phrases according to the number of occurrences of the repeated phrases to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords in the emphasized feature phrases; using the target emphasized feature phrases to judge and identify the website to be detected, so as to identify whether there are black links in the website to be detected. The technical solution of the present application can effectively make up for the problem that black links cannot be discovered in time due to insufficient vocabulary.

Description

Technical Field

[0001] The present invention relates to the field of computer security technology, and in particular to a black link identification method, device, equipment and storage medium. Background Art

[0002] Black links are a common method in SEO (Search Engine Optimization). Their purpose is to improve the ranking of websites in search engines through illegal SEO techniques and achieve economic benefits through black link transactions. Keywords will be implanted in the title and meta of web pages that are most easily indexed by search engines, and the words will appear repeatedly to attract the attention of search engines and users to achieve the purpose of improving search engine rankings and obtaining economic benefits. Often when hackers implant black links in websites, it means that hackers have invaded our website. In addition to hanging black links, they may also hang some information that is not indexed by standard search engines (not limited to websites and names) on our website, and even implant Trojan viruses, so as to use the ranking of the website in the search engine to achieve illegal transactions and other purposes. It is also possible to steal some important files such as data passwords, making the website unsafe. The timely discovery and rectification of black links plays an important role in the entire network environment and website security.

[0003] At present, the identification of black links in certain scenarios is based on keyword feature matching based on thesaurus. Although the identification of black links based on keyword matching based on thesaurus can achieve the effect of black link discovery to a certain extent, due to problems such as incomplete thesaurus and untimely word library updates, some new black links, new keywords, and deformed keywords cannot be identified, resulting in the inability to timely and effectively discover and rectify black links, which has a negative impact on government departments, units, enterprises, schools and even the entire network environment.

[0004] In summary, how to make up for the inability to detect black links in a timely manner due to the lack of vocabulary, reduce the dependence on keyword vocabulary, and achieve adaptive identification of black links are issues that need to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a black link identification method, device, equipment and storage medium, which can make up for the inability to timely discover black links due to insufficient vocabulary, reduce the dependence on keyword vocabulary, and achieve adaptive identification of black links. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a black link identification method, comprising:

[0007] Obtaining the uniform resource locator of the website to be detected, and parsing the uniform resource locator to obtain the webpage source code of the website to be detected;

[0008] Extracting content information from a preset attribute tag in the webpage source code, and extracting repeated phrases in the content information according to a suffix tree structure of the content information and a preset extraction condition;

[0009] Sorting the repeated phrases according to the number of times the repeated phrases appear to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords from the emphasized feature phrases;

[0010] The target emphasis feature phrase is used to judge and identify the website to be detected, so as to identify whether there is a black link in the website to be detected.

[0011] Optionally, parsing the uniform resource locator to obtain the webpage source code of the website to be detected includes:

[0012] The uniform resource locator is parsed using web crawler technology to obtain the web page source code of the website to be detected.

[0013] Optionally, extracting content information from a preset attribute tag in the webpage source code includes:

[0014] The content information in the title tag in the webpage source code and the content information in the Meta tag in the webpage source code are extracted.

[0015] Optionally, extracting repeated phrases in the content information according to the suffix tree structure of the content information and based on preset extraction conditions includes:

[0016] Determining a first threshold for regulating a minimum length of the repeated phrase, and a second threshold for regulating a minimum number of repetitions of the repeated phrase;

[0017] According to the suffix tree structure of the content information, a repeated phrase whose length in the content information reaches the first threshold and whose number of occurrences reaches the second threshold is extracted.

[0018] Optionally, the step of sorting the repeated phrases according to the number of times the repeated phrases appear to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords from the emphasized feature phrases includes:

[0019] sorting the repeated phrases in descending order according to the number of occurrences of the repeated phrases to obtain emphasized characteristic phrases related to the characteristic intensity sorting of the repeated phrases;

[0020] According to the characteristic strength, a target emphasized feature phrase containing a target keyword is screened out from the emphasized feature phrases.

[0021] Optionally, after sorting the repeated phrases according to the number of times the repeated phrases appear, the method further includes:

[0022] Determining whether the repeated phrase contains a target keyword;

[0023] If the repeated phrase contains the target keyword, the repeated phrase is determined to be the target emphasized feature phrase;

[0024] If the target keyword is not included in the repeated phrase, the repeated phrase is judged and identified based on a preset judgment condition to identify whether there is a black link in the website to be detected.

[0025] Optionally, judging and identifying the repeated phrase based on a preset judgment condition to identify whether there is a black link in the website to be detected includes:

[0026] If the number of the emphasized characteristic phrases in the repeated phrases is greater than a third preset threshold, it is determined that there are black links in the website to be detected;

[0027] If the number of the emphasized feature phrases in the repeated phrases is not greater than the third preset threshold, determining whether the maximum number of repetitions of the repeated phrases is greater than a fourth preset threshold;

[0028] If the maximum number of repetitions of the repeated phrase is greater than the fourth preset threshold, it is determined that the black link exists in the website to be detected.

[0029] In a second aspect, the present application discloses a black chain identification device, comprising:

[0030] A web page processing module, used for obtaining the uniform resource locator of the website to be detected, and parsing the uniform resource locator to obtain the web page source code of the website to be detected;

[0031] A repeated phrase extraction module, used to extract content information in a preset attribute tag in the webpage source code, and extract repeated phrases in the content information according to a suffix tree structure of the content information and a preset extraction condition;

[0032] a target emphasis feature phrase determination module, used to sort the repeated phrases according to the number of times the repeated phrases appear to obtain emphasis feature phrases, and then screen out the target emphasis feature phrases containing the target keyword in the emphasis feature phrases;

[0033] The black link identification module is used to use the target emphasis feature phrase to judge and identify the website to be detected, so as to identify whether there is a black link in the website to be detected.

[0034] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the black chain identification method as described above.

[0035] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the black link identification method as described above.

[0036] In the present application, the uniform resource locator of the website to be detected is obtained, and the uniform resource locator is parsed to obtain the web page source code of the website to be detected; then the content information in the preset attribute tag in the web page source code is extracted, and according to the suffix tree structure of the content information, based on the preset extraction conditions, the repeated phrases in the content information are extracted; the repeated phrases are sorted according to the number of times the repeated phrases appear to obtain the emphasized feature phrases, and then the target emphasized feature phrases containing the target keywords in the emphasized feature phrases are screened out; finally, the website to be detected is judged and identified using the target emphasized feature phrases to identify whether there are black links in the website to be detected. It can be seen that the web page source code is obtained by parsing the website to be detected, the content information in the preset attribute tag in the web page source code is extracted, the suffix tree is generated in combination with the suffix tree structure, and the repeated phrases in the suffix tree are extracted by the preset extraction conditions, and then the emphasized feature phrases are combined to effectively make up for the deficiencies of the vocabulary. That is, by guessing the black link keywords through phrase guessing, it is possible to identify black links without relying on the vocabulary, effectively making up for the problem that black links cannot be discovered in time due to insufficient vocabulary, and also reducing the dependence on keyword vocabulary to a certain extent, so as to achieve adaptive identification of black links. In addition, by screening out the target emphasis feature phrases containing the target keywords in the emphasis feature phrases, the method of purifying repeated phrases is implemented to filter out false positive data, so as to achieve the effect of effectively and timely discovering and rectifying black links. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 This is a flow chart of a black link identification method disclosed in this application;

[0039] Figure 2 A schematic diagram of a black chain identification solution disclosed in this application;

[0040] Figure 3 This is a specific flow chart of a black link identification method disclosed in this application;

[0041] Figure 4 A schematic diagram of a black link identification process disclosed in this application;

[0042] Figure 5 This is a schematic diagram of the structure of a black chain identification device disclosed in this application;

[0043] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] At present, the identification of black links in certain scenarios is based on keyword feature matching based on thesaurus. Although the identification of black links based on keyword matching based on thesaurus can achieve the effect of black link discovery to a certain extent, due to problems such as incomplete thesaurus and untimely thesaurus updates, some new black links, new keywords, and deformed keywords cannot be identified, resulting in the inability to timely and effectively discover and rectify black links, which has a negative impact on government departments, units, enterprises, schools and even the entire network environment.

[0046] To this end, the present application provides a black link identification solution, which can make up for the fact that black links cannot be discovered in time due to insufficient vocabulary, reduce the dependence on keyword vocabulary, and achieve adaptive identification of black links.

[0047] The embodiment of the present invention discloses a black link identification method, see Figure 1 As shown, the method includes:

[0048] Step S11: Obtain the uniform resource locator of the website to be detected, and parse the uniform resource locator to obtain the webpage source code of the website to be detected.

[0049] In the embodiment of the present application, the URL (Uniform Resource Locator) of the website to be detected is first input, and the URL can be used to identify where the information resources are on the Internet, and the web page source code of the URL is parsed to extract the content. It should be pointed out that when processing the web page, the uniform resource locator is parsed using web crawler technology to obtain the web page source code of the website to be detected.

[0050] Step S12: extracting content information from preset attribute tags in the webpage source code, and extracting repeated phrases in the content information according to the suffix tree structure of the content information and based on preset extraction conditions.

[0051] In the embodiment of the present application, after the website to be detected is input, the webpage is processed to extract the Title and Meta content in the webpage source code of the website to be detected for phrase extraction and algorithm recognition. Specifically, the content information in the Title tag in the webpage source code and the content information in the Meta tag in the webpage source code are extracted.

[0052] It is understandable that Title is a tag in HTML (Hyper Text Markup Language), which is mainly in the title bar of the web page. The words in the title will be searched by search engines, affecting the ranking of the website in the search engine, and is the focus of hackers' use; Meta is also a tag in HTML. The meta element often does not attract the attention of users, but meta has an impact on the entire web page, and will play a key role in whether the web page can be retrieved by search engines and its ranking in the search. Therefore, it is also a focus of hackers' use. Using the extracted Title and Meta content, further through the suffix tree structure, combined with the preset extraction conditions, the repeated phrases in the content information are extracted. In this way, the dependence on the keyword library is reduced to a certain extent, and the preparation work for black link identification can be realized by further judging the emphasized feature phrases in the repeated phrases based on phrase guesses.

[0053] Step S13: sorting the repeated phrases according to the number of times the repeated phrases appear to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords from the emphasized feature phrases.

[0054] In the embodiment of the present application, it is emphasized that the characteristic phrase is a repeated phrase with certain characteristics that appears repeatedly in a paragraph and the phrase length reaches a specific threshold, which is mainly reflected in the number of occurrences. The repeated phrases extracted in step S12 can be characteristically sorted according to the number of occurrences, and the higher the number of occurrences, the stronger the characteristic.

[0055] In the embodiment of the present application, the number of occurrences of repeated phrases is sorted in reverse order, that is, the repeated phrases are sorted in descending order according to the number of occurrences of the repeated phrases to obtain emphasized characteristic phrases related to the characteristic strength sorting of the repeated phrases.

[0056] It is understandable that there will be a certain degree of uncertainty in the extracted emphasis feature phrases. For example, if the web page content is an article displayed around a theme, and the phrase corresponding to this theme has nothing to do with black links, then the emphasis feature phrases can be purified according to certain characteristics. Specifically, according to the characteristic strength, the target emphasis feature phrases containing target keywords in the emphasis feature phrases are screened out. For example, keywords contain special symbols (with promotional nature, such as black boxes [], email symbols @, etc.); and copywriting that contains some guiding content that is not conducive to dissemination. If the feature is hit, the phrase is considered to be accurate, and the identified black links are more credible. The embodiment of the present application only lists some features, and other features can be supplemented according to needs, and are not specifically limited here.

[0057] Step S14: using the target emphasis feature phrase to identify the website to be detected, so as to identify whether there is a black link in the website to be detected.

[0058] In the embodiment of the present application, the target emphasis feature phrase containing the target keyword is screened out from the emphasis feature phrase, which completes the preparation stage in the black link identification process. The target emphasis feature phrase is used to judge and identify the website to be detected, so as to identify whether there is a black link in the website to be detected. When the target emphasis feature phrase appears in the website to be detected, it is very likely that there is a black link in the website, and the credibility is high.

[0059] like Figure 2 The figure shows the overall flow chart of the black link identification process. It is mainly divided into seven parts: target website input, web page processing, threshold determination, repeated phrase extraction, emphasis on feature phrase screening, emphasis on feature phrase purification, and black link identification. Among them, when determining the threshold, it is generally necessary to determine two thresholds. The first threshold is the minimum length of the repeated phrase minLength, which can be customized. Generally, the threshold should not be less than 2; the second threshold is the minimum number of repetitions of the repeated phrase minRepeatCount, which cannot be less than 2.

[0060] In the present application, the uniform resource locator of the website to be detected is obtained, and the uniform resource locator is parsed to obtain the web page source code of the website to be detected; then the content information in the preset attribute tag in the web page source code is extracted, and according to the suffix tree structure of the content information, based on the preset extraction conditions, the repeated phrases in the content information are extracted; the repeated phrases are sorted according to the number of times the repeated phrases appear to obtain the emphasized feature phrases, and then the target emphasized feature phrases containing the target keywords in the emphasized feature phrases are screened out; finally, the website to be detected is judged and identified using the target emphasized feature phrases to identify whether there are black links in the website to be detected. It can be seen that the web page source code is obtained by parsing the website to be detected, the content information in the preset attribute tag in the web page source code is extracted, the suffix tree is generated in combination with the suffix tree structure, and the repeated phrases in the suffix tree are extracted by the preset extraction conditions, and then the emphasized feature phrases are combined to effectively make up for the deficiencies of the vocabulary. That is, by guessing the black link keywords through phrase guessing, it is possible to identify black links without relying on the vocabulary, effectively making up for the problem that black links cannot be discovered in time due to insufficient vocabulary, and also reducing the dependence on keyword vocabulary to a certain extent, so as to achieve adaptive identification of black links. In addition, by screening out the target emphasis feature phrases containing the target keywords in the emphasis feature phrases, the method of purifying repeated phrases is implemented to filter out false positive data, so as to achieve the effect of effectively and timely discovering and rectifying black links.

[0061] This application embodiment discloses a specific black link identification method, see Figure 3 As shown, the method includes:

[0062] Step S21: Obtain the uniform resource locator of the website to be detected, and parse the uniform resource locator to obtain the webpage source code of the website to be detected.

[0063] For a more specific processing procedure of the above step S21, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0064] Step S22: extracting content information from a preset attribute tag in the webpage source code, and determining a first threshold for regulating the minimum length of the repeated phrase, and a second threshold for regulating the minimum number of repetitions of the repeated phrase.

[0065] In the embodiment of the present application, after the website to be detected is input, the web page is processed to extract the Title and Meta content in the web page source code of the website to be detected for phrase extraction and algorithm recognition. Then two thresholds need to be determined. The first threshold is used to regulate the minimum length of the repeated phrase. The threshold can be customized. Generally, the threshold should not be less than 2; the second threshold is used to regulate the minimum number of repetitions of the repeated phrase. The value cannot be less than 2.

[0066] Step S23: extracting, according to the suffix tree structure of the content information, repeated phrases in the content information whose length reaches the first threshold and whose number of occurrences reaches the second threshold.

[0067] In the embodiment of the present application, after determining the first threshold for regulating the minimum length of a repeated phrase and the second threshold for regulating the minimum number of repetitions of the repeated phrase, the repeated phrase and its number of occurrences are extracted by extracting the repeated phrase through the suffix tree structure. For example, in abcaabcabcbc, extract the repeated phrase with 2≤phrase length≤3 and the number of repetitions>1, then the following meet the conditions: abc appears 3 times, ab appears 3 times, bc appears 4 times, and ca appears 2 times.

[0068] Step S24: sorting the repeated phrases according to the number of times the repeated phrases appear, and determining whether the repeated phrases contain the target keyword.

[0069] In the embodiment of the present application, since a characteristic emphasized feature phrase can be obtained according to the number of times a repeated phrase appears, when the number of times a repeated phrase appears is sorted in reverse order, the higher the number of times, the stronger the characteristic is, and at this time, it is determined whether the repeated phrase contains the target keyword.

[0070] It should be pointed out that the target keyword is a keyword that can improve the credibility of black link identification, such as sensitive phrases, promotional phrases and other self-supplemented keywords. By judging whether the sorted repeated phrases contain the target keyword, the emphasis feature phrase can be purified. If it contains the target keyword, it is considered that the feature is hit, the phrase is accurate, and the credibility of the identified black link is higher.

[0071] Step S25: If the repeated phrase contains the target keyword, the repeated phrase is determined to be the target emphasized feature phrase; if the repeated phrase does not contain the target keyword, the repeated phrase is judged and identified based on preset judgment conditions to identify whether there are black links in the website to be detected.

[0072] In the embodiment of the present application, if the target keyword is contained in the repeated phrase, the repeated phrase is determined as the target emphasized feature phrase, that is, in the black link identification process, if the target emphasized feature phrase exists, it can be considered that the possibility of the existence of the black link is very high. If the repeated phrase does not contain the target keyword, it is necessary to further judge and identify the repeated phrase based on the preset judgment condition to identify whether there is a black link in the website to be detected.

[0073] In an embodiment of the present application, the repeated phrases are judged and identified based on preset judgment conditions to identify whether there are black links in the website to be detected, including: if the number of the emphasized feature phrases in the repeated phrases is greater than a third preset threshold, then it is determined that there are black links in the website to be detected; if the number of the emphasized feature phrases in the repeated phrases is not greater than the third preset threshold, then it is determined whether the maximum number of repetitions of the repeated phrases is greater than a fourth preset threshold; if the maximum number of repetitions of the repeated phrases is greater than the fourth preset threshold, then it is determined that there are black links in the website to be detected.

[0074] It is understandable that if there are many repeated phrases in the title and meta, but no phrase containing the target keyword is identified, it can only be considered as a black link, but the credibility is low, so further judgment is needed. Specifically, if there are many emphasized feature phrases in the repeated phrases, generally more than 5, it can be considered that there is a high possibility of a black link; if the number of emphasized feature phrases does not meet the conditions for being determined as a black link, the maximum number of phrase repetitions will continue to be judged. When the number of occurrences of the same emphasized feature phrase is greater than a certain value, it can be considered that there is a high possibility of a black link, generally more than 5 times.

[0075] like Figure 4 The specific process of black link identification is shown. The extraction of emphasized feature phrases is based on sorting and screening repeated phrases. If there is no emphasized feature phrase, the process ends. If there is, the emphasized feature phrase continues to be sorted and purified. When the target keyword exists, that is, the emphasized feature phrase exists after purification, it is identified as a black link with high credibility; if the emphasized feature phrase does not exist after purification, the number of emphasized feature phrases and the maximum number of phrase repetitions are determined. Since the prior art basically identifies black links based on keyword matching based on thesaurus, the thesaurus is fixed or updated regularly, but the embodiment of the present application can identify black links by guessing black link keywords, effectively making up for the problem that black links cannot be discovered in time due to insufficient thesaurus, and also reducing the dependence on the keyword thesaurus to a certain extent, so as to achieve adaptive identification of black links.

[0076] It is understandable that phrase guessing can also obtain some keywords by frequently analyzing sample data in large quantities by increasing labor costs or machine learning. However, manpower and machine learning can only increase the recognition of black links to a certain extent, but they still rely on the vocabulary. The advantage of phrase guessing is that it can identify black links without the vocabulary.

[0077] Step S26: using the target emphasis feature phrase to identify the website to be detected, so as to identify whether there is a black link in the website to be detected.

[0078] For a more specific processing procedure of the above step S26, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0079] In the present application, the uniform resource locator of the website to be detected is obtained and the uniform resource locator is parsed to obtain the web page source code of the website to be detected; then the content information in the preset attribute tag in the web page source code is extracted, and a first threshold for regulating the minimum length of the repeated phrase and a second threshold for regulating the minimum number of repetitions of the repeated phrase are determined; according to the suffix tree structure of the content information, the repeated phrases in the content information are extracted, the length of the phrases reaches the first threshold, and the number of occurrences of the phrases reaches the second threshold; the repeated phrases are sorted according to the number of occurrences of the repeated phrases, and it is determined whether the repeated phrases contain target keywords; if the repeated phrases contain the target keywords, the repeated phrases are determined as the target emphasis feature phrases; if the repeated phrases do not contain the target keywords, the repeated phrases are judged and identified based on the preset judgment conditions to identify whether there are black links in the website to be detected; finally, the website to be detected is judged and identified using the target emphasis feature phrases to identify whether there are black links in the website to be detected. It can be seen that by parsing the website to be tested to obtain the webpage source code, the content information in the preset attribute tags in the webpage source code is extracted, and the suffix tree structure is combined to generate the suffix tree, and the repeated phrases in the suffix tree are extracted through the preset extraction conditions, and then the emphasis feature phrases are combined to effectively make up for the deficiencies in the vocabulary. That is, by guessing the black link keywords in the way of phrase guessing, it is possible to identify the black links without relying on the vocabulary, effectively making up for the problem that the black links cannot be discovered in time due to the insufficiency of the vocabulary, and also reducing the dependence on the keyword vocabulary to a certain extent, so as to achieve adaptive identification of black links. In addition, by screening out the target emphasis feature phrases containing the target keywords in the emphasis feature phrases, the method of purifying the repeated phrases is implemented to filter out the false positive data, so as to achieve the effect of effectively and timely discovering and rectifying the black links.

[0080] Correspondingly, the present application also discloses a black chain identification device, see Figure 5 As shown, the device comprises:

[0081] The web page processing module 11 is used to obtain the uniform resource locator of the website to be detected, and parse the uniform resource locator to obtain the web page source code of the website to be detected;

[0082] A repeated phrase extraction module 12 is used to extract content information in a preset attribute tag in the webpage source code, and extract repeated phrases in the content information according to a suffix tree structure of the content information and a preset extraction condition;

[0083] A target emphasis feature phrase determination module 13 is used to sort the repeated phrases according to the number of times the repeated phrases appear to obtain emphasis feature phrases, and then filter out the target emphasis feature phrases containing the target keyword in the emphasis feature phrases;

[0084] The black link identification module 14 is used to use the target emphasis feature phrase to judge and identify the website to be detected, so as to identify whether there is a black link in the website to be detected.

[0085] Among them, for more specific working processes of the above-mentioned modules, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0086] It can be seen that through the above scheme of this embodiment, the uniform resource locator of the website to be detected is obtained, and the uniform resource locator is parsed to obtain the web page source code of the website to be detected; then the content information in the preset attribute tag in the web page source code is extracted, and according to the suffix tree structure of the content information, based on the preset extraction conditions, the repeated phrases in the content information are extracted; the repeated phrases are sorted according to the number of times the repeated phrases appear to obtain the emphasized feature phrases, and then the target emphasized feature phrases containing the target keywords in the emphasized feature phrases are screened out; finally, the target emphasized feature phrases are used to judge and identify the website to be detected to identify whether there are black links in the website to be detected. It can be seen that the web page source code is obtained by parsing the website to be detected, the content information in the preset attribute tag in the web page source code is extracted, the suffix tree structure is combined to generate the suffix tree, and the repeated phrases in the suffix tree are extracted according to the preset extraction conditions, and then the emphasized feature phrases are combined to effectively make up for the deficiencies of the vocabulary. That is, by guessing the black link keywords through phrase guessing, it is possible to identify black links without relying on the vocabulary, effectively making up for the problem that black links cannot be discovered in time due to insufficient vocabulary, and also reducing the dependence on keyword vocabulary to a certain extent, so as to achieve adaptive identification of black links. In addition, by screening out the target emphasis feature phrases containing the target keywords in the emphasis feature phrases, the method of purifying repeated phrases is implemented to filter out false positive data, so as to achieve the effect of effectively and timely discovering and rectifying black links.

[0087] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the diagram cannot be considered as any limitation on the scope of use of the present application.

[0088] Figure 6A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the black chain identification method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be a computer.

[0089] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0090] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222 and data 223, etc. The data 223 may include various data. The storage method may be temporary storage or permanent storage.

[0091] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the black link identification method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0092] Furthermore, the embodiments of the present application also disclose a computer-readable storage medium, wherein the computer-readable storage medium mentioned here includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk or an optical disk, or any other form of storage medium known in the technical field. Wherein, when the computer program is executed by the processor, the aforementioned black chain identification method is implemented. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] The steps of the black chain identification or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0095] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0096] The above is a detailed introduction to a black chain identification method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A black link identification method, characterized in that: include: Obtaining the uniform resource locator of the website to be detected, and parsing the uniform resource locator to obtain the webpage source code of the website to be detected; Extracting content information from a preset attribute tag in the webpage source code, and extracting repeated phrases in the content information according to a suffix tree structure of the content information and based on preset extraction conditions; Sorting the repeated phrases according to the number of times the repeated phrases appear to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords from the emphasized feature phrases; Using the target emphasis feature phrase to judge and identify the website to be detected, so as to identify whether there is a black link in the website to be detected; After sorting the repeated phrases according to the number of times the repeated phrases appear, the method further includes: Determining whether the repeated phrase contains a target keyword; If the repeated phrase contains the target keyword, the repeated phrase is determined to be the target emphasized feature phrase; If the repeated phrase does not contain the target keyword, the repeated phrase is judged and identified based on a preset judgment condition to identify whether there is a black link in the website to be detected; The step of judging and identifying the repeated phrase based on a preset judgment condition to identify whether there is a black link in the website to be detected includes: If the number of the emphasized characteristic phrases in the repeated phrases is greater than a third preset threshold, it is determined that there are black links in the website to be detected; If the number of the emphasized feature phrases in the repeated phrases is not greater than the third preset threshold, determining whether the maximum number of repetitions of the repeated phrases is greater than a fourth preset threshold; If the maximum number of repetitions of the repeated phrase is greater than the fourth preset threshold, it is determined that the black link exists in the website to be detected.

2. The black link identification method according to claim 1, characterized in that: The step of parsing the uniform resource locator to obtain the webpage source code of the website to be detected includes: The uniform resource locator is parsed using web crawler technology to obtain the web page source code of the website to be detected.

3. The black link identification method according to claim 1, characterized in that: The extracting content information in the preset attribute tag in the webpage source code includes: Extract content information in the title tag in the webpage source code and content information in the Meta tag in the webpage source code.

4. The black link identification method according to claim 1, characterized in that: The extracting repeated phrases in the content information according to the suffix tree structure of the content information and based on preset extraction conditions includes: Determining a first threshold for regulating a minimum length of the repeated phrase, and a second threshold for regulating a minimum number of repetitions of the repeated phrase; According to the suffix tree structure of the content information, a repeated phrase whose length in the content information reaches the first threshold and whose number of occurrences reaches the second threshold is extracted.

5. The black link identification method according to claim 1, characterized in that: The step of sorting the repeated phrases according to the number of times the repeated phrases appear to obtain emphasized feature phrases, and then screening out target emphasized feature phrases containing target keywords from the emphasized feature phrases includes: sorting the repeated phrases in descending order according to the number of occurrences of the repeated phrases to obtain emphasized characteristic phrases related to the characteristic intensity sorting of the repeated phrases; According to the characteristic strength, a target emphasized feature phrase containing a target keyword is screened out from the emphasized feature phrases.

6. A black chain identification device, characterized in that: include: A web page processing module, used for obtaining the uniform resource locator of the website to be detected, and parsing the uniform resource locator to obtain the web page source code of the website to be detected; A repeated phrase extraction module, used to extract content information in a preset attribute tag in the webpage source code, and extract repeated phrases in the content information according to a suffix tree structure of the content information and a preset extraction condition; a target emphasis feature phrase determination module, used to sort the repeated phrases according to the number of times the repeated phrases appear to obtain emphasis feature phrases, and then screen out the target emphasis feature phrases containing the target keyword in the emphasis feature phrases; A black link identification module, used to use the target emphasis feature phrase to identify the website to be detected, so as to identify whether there is a black link in the website to be detected; The target emphasis feature phrase determination module is further used to: after sorting the repeated phrases according to the number of times the repeated phrases appear, determine whether the repeated phrases contain the target keyword; if the repeated phrases contain the target keyword, determine the repeated phrases as the target emphasis feature phrases; If the repeated phrase does not contain the target keyword, the repeated phrase is judged and identified based on a preset judgment condition to identify whether there is a black link in the website to be detected; The target emphasis feature phrase determination module is specifically used to: If the number of the emphasized characteristic phrases in the repeated phrases is greater than a third preset threshold, it is determined that there are black links in the website to be detected; If the number of the emphasized feature phrases in the repeated phrases is not greater than the third preset threshold, determining whether the maximum number of repetitions of the repeated phrases is greater than a fourth preset threshold; If the maximum number of repetitions of the repeated phrase is greater than the fourth preset threshold, it is determined that the black link exists in the website to be detected.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the black chain identification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein when the computer program is executed by the processor, the black link identification method as described in any one of claims 1 to 5 is implemented.

Images