Website identification method and device, equipment and storage medium

Abstract

The invention discloses a website identification method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a target website and a risk set corresponding to the target website, wherein the risk set comprises a risk category and a risk feature; and extracting webpage feature data from the target website based on the risk category. And determining an identification result of the target website based on the webpage feature data and the risk features. Therefore, by obtaining the risk set corresponding to the target website, polling matching of all the risk sets is avoided, unnecessary calculation is reduced, and execution efficiency is improved. And webpage feature data are extracted from the target website according to the risk category, so that directional extraction is realized, and the feature extraction efficiency is improved. Finally, the webpage feature data are directly compared with the risk features, the recognition result is determined, the probability of misjudgment and missed judgment is effectively reduced, and the accuracy of website risk recognition is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to website identification methods, devices, equipment, and storage media. Background Art

[0002] With the rapid development of Internet technology, corporate business leaks occur frequently, posing a serious threat to corporate business security.

[0003] In related technical fields, network risk matching has become a key tool for effectively preventing the leakage of corporate secrets. This risk matching allows for the early identification of websites and related network behaviors that may pose a risk of leaks, allowing targeted preventative measures to be implemented. Existing technologies primarily rely on pre-set risk rules, such as those containing malicious code, false information keywords, and illegal links. These rules then compare website features with a rule library, directly flagging matching risk types, such as fraudulent and phishing websites.

[0004] However, risk matching methods in related technologies present numerous challenges that need to be addressed. For one thing, risk matching feature algorithms employ a simplistic logic, relying on only a few features for judgment and lacking comprehensive analysis and combinatorial logic operations across multiple web page features. This approach can easily lead to missed matches and overmatches, resulting in the omission of potentially risky websites. Furthermore, risk matching algorithms suffer from inefficient execution. Most existing algorithms rely on polling based on a rule base. During execution, they extract a large number of irrelevant web page features, requiring numerous unrelated rules to participate in the matching operation, consuming significant computing resources. Summary of the Invention

[0005] The purpose of this application is to provide a website identification method, device, equipment and storage medium that can quickly and accurately identify risky websites.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a website identification method, comprising: obtaining a target website and a risk set corresponding to the target website, the risk set comprising a risk category and risk characteristics; extracting webpage feature data from the target website based on the risk category; and determining an identification result for the target website based on the webpage feature data and the risk characteristics.

[0008] The technical solution provided by this application can quickly locate the risk set related to the target website by obtaining the target website and the risk set corresponding to the target website, avoiding polling and matching of all risk sets, thereby reducing unnecessary calculations and improving execution efficiency. Based on the risk category, web page feature data is extracted from the target website, and web page feature data can be extracted in a targeted manner according to the risk category, thereby improving the efficiency of feature extraction. Based on the web page feature data and risk characteristics, the identification result of the target website is determined. By directly comparing the web page feature data with the risk characteristics, the probability of misjudgment and missed judgment can be effectively reduced, and the accuracy of risk identification can be improved.

[0009] One possible implementation method determines the identification result of a target website based on webpage feature data and risk characteristics. Specifically, this can be achieved by calculating the similarity between the webpage feature data and the risk characteristics to obtain a similarity value. Based on the similarity value, a feature matching result set between the webpage feature data and the risk characteristics is determined. Based on the feature matching result set, the identification result of the target website is determined. By using a quantified similarity value, subjective judgment bias is avoided, making the risk assessment of the feature matching result set more objective. The identification result determined based on this feature matching result set can accurately determine whether a website poses a risk, reducing missed or false positives.

[0010] Another possible implementation method is to determine the feature matching result set between the web page feature data and the risk feature based on the similarity value. Specifically, it can be implemented as follows: the similarity value greater than or equal to the similarity threshold is determined as the first feature matching result. The similarity value less than the similarity threshold is determined as the second feature matching result. The first feature matching result and the second feature matching result are aggregated to obtain a feature matching result set. By setting the similarity threshold, the similarity values ​​can be divided into two categories, clearly defining the situations of high matching (first feature matching result) and low matching (second feature matching result). After aggregating and integrating these two types of results, a complete feature matching result set can be formed.

[0011] Another possible implementation method determines the target website's identification result based on the feature matching result set. Specifically, this can be achieved by merging the feature matching results in the feature matching result set based on the logical combination relationship between risk features to obtain the identification result. This logical combination method can avoid the one-sidedness of single feature matching results, making the final identification result more accurate and more consistent with the actual risk situation of the webpage, thereby improving the accuracy of the identification result.

[0012] In another possible implementation, the above-mentioned logical combination relationship includes at least one of logical AND, logical OR, and logical NOT.

[0013] Another possible implementation method involves obtaining the risk set corresponding to the target website: extract the target website and obtain the corresponding website host. Based on the host, the corresponding risk set is searched in a risk set index table. The risk set index table includes the corresponding retrieval index, risk category, and risk characteristics. Constructing a risk set index table with the host as the retrieval index and the risk category and risk characteristics as the values ​​allows for rapid location of the risk set corresponding to the target host, avoiding matching irrelevant risk sets and improving matching efficiency.

[0014] In another possible implementation, the risk feature includes at least one of: a web page title, metadata, body, cascading style sheet (CSS), script (JS), and script tag (Script).

[0015] In a second aspect, a website identification device is provided, which includes: an acquisition module, an extraction module, and a determination module.

[0016] The acquisition module is used to acquire the target website and the risk set corresponding to the target website. The risk set includes risk categories and risk characteristics.

[0017] The extraction module is used to extract web page feature data from the target website based on risk categories.

[0018] The above-mentioned determination module is used to determine the identification result of the target website based on the web page feature data and risk characteristics.

[0019] In one possible implementation, the determination module is further configured to calculate a similarity between the webpage feature data and the risk features to obtain a similarity value. Based on the similarity value, a feature matching result set between the webpage feature data and the risk features is determined. Based on the feature matching result set, an identification result for the target website is determined.

[0020] In another possible implementation, the determination module is further configured to determine similarity values ​​greater than or equal to a similarity threshold as first feature matching results, and similarity values ​​less than the similarity threshold as second feature matching results. The first feature matching results and the second feature matching results are aggregated to obtain a feature matching result set.

[0021] In another possible implementation, the above-mentioned determination module is further configured to merge the feature matching results in the feature matching result set based on the logical combination relationship between the risk features to obtain the identification result.

[0022] In another possible implementation, the above-mentioned logical combination relationship includes at least one of logical AND, logical OR, and logical NOT.

[0023] In another possible implementation, the acquisition module is further configured to extract the target website and obtain the website host corresponding to the target website. Based on the host, the risk set corresponding to the target website is searched in a risk set index table. The risk set index table includes the corresponding retrieval index, risk category, and risk characteristics of the risk set.

[0024] In another possible implementation, the risk feature includes at least one of: a web page title, metadata, body, cascading style sheet (CSS), script (JS), and script tag (Script).

[0025] The technical effects corresponding to any one of the implementation methods in the second aspect can be referred to the technical effects corresponding to any one of the implementation methods in the above-mentioned first aspect, and will not be repeated here.

[0026] In a third aspect, a computer device is provided, comprising: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the website identification method of the above aspect.

[0027] In a fourth aspect, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the website identification method of the above aspect.

[0028] In a fifth aspect, a computer program product is provided. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the website identification method of the above aspect is implemented.

[0029] The solutions provided in the third to fifth aspects are used to implement the method provided in the first aspect, and their specific implementations are not described in detail. The technical effects corresponding to any implementation of the solutions provided in the third to fifth aspects can be referred to the technical effects corresponding to any implementation of the first aspect, and are not described in detail here.

[0030] It should be noted that various possible implementations of any of the above aspects can be combined under the premise that the solutions are not contradictory. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A schematic diagram of the structure of a computer system provided in an embodiment of the present application;

[0033] Figure 2 A flowchart of a website identification method provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram illustrating the principle of a website identification method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of the structure of a website identification device provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "back," "inner," "outer," and the like, indicating directions or positional relationships, are based on the directions or relative positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this application and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific direction, be constructed, or operate in a specific direction. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned directionality descriptions may be flexibly set in actual application, provided that the relative positional relationships shown in the accompanying drawings are met.

[0039] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. Throughout this application, unless otherwise specified, "plurality" means two or more.

[0040] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connected," and "connected" should be understood broadly. For example, they may refer to fixed connections, detachable connections, or integral connections. They may be directly connected, indirectly connected through an intermediary, or internally connected between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0041] In the embodiments of the present application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, article, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of other identical elements in the process, article, or device comprising the element.

[0042] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0043] In the embodiments of the present application, at least one can also be described as one or more, and multiple can be two, three, four or more, which is not limited in this application.

[0044] In the description of this specification, specific features, structures, materials or characteristics may be combined in an appropriate manner in any one or more embodiments or examples.

[0045] To facilitate understanding, the terms involved in the embodiments of this application are first explained.

[0046] Website risk refers to potential threats or adverse factors that a website may face across multiple dimensions, including operations, technology, data, and compliance. These risks can lead to malfunctioning websites, data leaks, a damaged user experience, legal liability, and even financial losses. These risks range from technical vulnerabilities to external attacks, from content compliance to business operations, and require systematic risk assessment and preventative measures to mitigate or avoid their impact. Examples include server failures, network attacks, and domain and hosting risks.

[0047] XML Path Language (XPath): is a query language used to locate and select nodes in eXtensible Markup Language (XML). It describes node locations through path expressions and supports filtering based on element content, attributes, and positional relationships.

[0048] Regular Expression (Regex): A tool for describing string patterns that can be used to match, find, and replace text.

[0049] Host: refers to a device in the network that can receive, process or forward data, has an independent Internet Protocol Address (IP) address, and can run operating systems and applications.

[0050] A web crawler is a program or script that automatically crawls the internet according to specific rules. It simulates human web browsing behavior, starting from the website's initial page and recursively accessing links within it to crawl and store web content (such as text, images, videos, and data). It is a core technology for applications such as search engines, data analysis, and content aggregation.

[0051] It should be noted that the information (including but not limited to device information, network information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the subject or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with relevant laws, regulations, and standards. For example, the network feature data, metadata, etc. involved in this application are all obtained with full authorization.

[0052] The commonly used website identification method in the industry mainly uses preset risk rules to compare website features with the rule library and directly mark the matching risk type. The following is a brief explanation.

[0053] Exemplarily, based on at least one of the web page data of the target web page and the operation data for the target web page, at least one type of characteristic data associated with the target web page is determined; based on the at least one type of characteristic data, first type of indicator data and second type of indicator data are determined; using the first type of indicator data, initial risk data obtained by the second type of indicator data is adjusted to obtain target risk data, wherein the target risk data characterizes the risk situation of the target web page.

[0054] However, the above technical solution still has the following defects:

[0055] 1. The risk matching algorithm has a simple operation logic: it does not perform combinational logic operations on multiple conditions, which may lead to problems such as missed matches and multiple matches, resulting in low accuracy.

[0056] 2. Low execution efficiency of the risk matching algorithm: The polling process based on the rule base causes irrelevant rules to be matched and irrelevant features to be extracted, which reduces the execution efficiency of the algorithm.

[0057] Based on this, the present application proposes a website identification method. By obtaining the target website and the risk set corresponding to the target website, the risk set related to the target website can be quickly located, avoiding polling and matching of all risk sets, thereby reducing unnecessary calculations and improving execution efficiency. Based on the risk category, web page feature data is extracted from the target website. The web page feature data can be extracted in a targeted manner according to the risk category, thereby improving the efficiency of feature extraction. Based on the web page feature data and risk characteristics, the identification result of the target website is determined. By directly comparing the web page feature data with the risk characteristics, the probability of misjudgment and missed judgment is effectively reduced, and the accuracy of risk identification is improved.

[0058] The solution provided in this application can be applied to Figure 1 In the computer system shown, Figure 1 A schematic diagram of the structure of a computer system provided in an embodiment of the present application.

[0059] Exemplarily, the computer system includes a computer device 100. The computer device 100 obtains a target website 101, and determines an identification result 102 of the target website based on a risk set corresponding to the target website 101.

[0060] Alternatively, the computer device 100 may be a device that directly obtains the target website 101. The computer device 100 may also be a device that determines the identification result 102 of the target website based on the risk set corresponding to the target website 101. The term "acquisition" of the computer device 100 includes any term having an acquisition function, such as querying, discovering, and extracting, and this application does not limit this.

[0061] Optionally, the computer device 100 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, embedded hardware for real-time simulation, or a cloud server that provides cloud computing services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data. The embodiments of the present application do not limit the implementation method and application scenarios of the computer device 100.

[0062] Optionally, the computer device 100 may be a terminal device. Terminal devices include, but are not limited to, electronic devices with data processing and information exchange capabilities, such as smart phones, tablet computers, laptop computers, and desktop computers. This embodiment of the application does not specifically limit this.

[0063] Figure 2 A flowchart of a website identification method provided in an embodiment of the present application. The method can be executed by a computer device. The computer device can be Figure 1 The computer device 100 in FIG.

[0064] like Figure 2 As shown, the website identification method provided in the embodiment of the present application may include:

[0065] Step S201: The computer device obtains a target website and a risk set corresponding to the target website.

[0066] The target website is the website that requires risk assessment. A website is a collection of related web pages created on the internet using tools such as Hypertext Markup Language (HTML) according to certain rules to display specific content. For example, the target website is http: / / csdn.cn / 51232.html.

[0067] Optionally, the target website can be various functional websites such as corporate official websites, e-commerce platforms, government websites, etc., or it can be network application carriers such as forums, blogs, and mini-programs, but is not limited to this. The embodiments of this application do not make specific limitations on this.

[0068] Among them, the risk set is a systematic set of various risks and their corresponding characteristics, which is formed by risk categories for a specific assessment object (such as a target website).

[0069] Optionally, the risk set includes risk categories and risk characteristics.

[0070] Risk categories define the classification of risks and reflect the nature and scope of the risk. Risk characteristics refer to the specific manifestations or criteria for determining each type of risk and are key indicators for identifying risks.

[0071] Optionally, the risk categories include network security risks and network information risks, but are not limited thereto, and the embodiments of the present application do not make specific limitations on this.

[0072] In some embodiments, network security risks include at least one of the following, but are not limited thereto, and are not specifically limited in the embodiments of the present application:

[0073] 1. Suspected code leakage risk: This refers to the lack of effective protection measures for website source code, application code, and other content, which poses a risk of unauthorized access. For example, a developer might mistakenly upload code containing core business logic to a public code repository, or improper website server configuration might allow source code to be directly accessed through a specific path. Code leakage could allow attackers to obtain critical information such as system architecture and vulnerability locations, facilitating subsequent targeted attacks (such as injecting malicious code and bypassing security verification), posing a serious threat to system security.

[0074] 2. Suspected password leakage risk: User or system passwords may be leaked during storage or transmission due to insufficient encryption or attacks. For example, a website database may not use salted hashing for user passwords, storing them in plain text or with weak encryption. Once the database is compromised, a large number of user passwords may be directly exposed, or passwords may be intercepted by a middleman during user login. Password leakage may lead to account theft, further triggering chain reactions such as personal information leakage and financial loss. For example, an attacker may use a leaked password to log in to a user's bank account and transfer funds.

[0075] 3. Malicious program risks: risks of computer Trojan viruses, spyware, ransomware, web crawlers and other malicious programs.

[0076] 4. Phishing risk: Attackers disguise themselves as official organizations and send fraudulent emails, text messages, or links. For example, they may send fake bank emails, tricking users into clicking on links to log in to fake websites and entering sensitive information such as account numbers and passwords, thereby stealing their assets.

[0077] 5. Fake Wireless Fidelity (Wi-Fi) Risk: In public places, attackers set up hotspots with names similar to legitimate Wi-Fi networks. Once users connect, their network operation information can be stolen. For example, if a user connects to a fake free Wi-Fi network in a shopping mall to shop, their payment password may be stolen, resulting in financial loss.

[0078] 6. Risk of external device attacks: External devices with malware, such as USB flash drives, mice, keyboards, etc., may automatically install harmful programs when connected to a computer.

[0079] In the embodiments of the present application, the suspected code leakage risk and the suspected password leakage risk in network security risks are taken as examples for detailed description.

[0080] In some embodiments, network information risks include at least one of the following, but are not limited thereto, and are not specifically limited in the embodiments of the present application:

[0081] 1. Risk of user personal information leakage: When engaging in online activities, users are required to provide personal information such as name, ID number, contact information, address, and bank card information when registering accounts, participating in surveys, or using mobile apps. An attack on a website or app's database could result in a large-scale data leak, stealing user information such as names, email addresses, and phone numbers. Furthermore, internal misconduct, such as employees illegally selling user information, can also lead to information leaks.

[0082] 2. Data leakage risk during storage and transmission: If the storage device is not encrypted or the encryption strength is insufficient, data can be easily cracked and accessed. During transmission, if insecure network protocols are used, data may be intercepted by intermediaries. For example, when users conduct online banking operations on public Wi-Fi, their account passwords may be stolen by criminals using network sniffing tools.

[0083] Optionally, the risk features include at least one of: web page title (title), metadata (Metedata, meta), main content (body), Cascading Style Sheets (Cascading Style Sheets, css), script (JavaScript, js), and script tag (script), but are not limited to these. The embodiments of the present application do not make specific limitations on this.

[0084] Title, web page title, usually located in the HTML <title> The tag is one of the core identifiers of the web page. For example,<title> https: / / www.baidu.com / _Baidu Search< / title> Indicates that the title of the website is: https: / / www.baidu.com / _Baidu search.

[0085] meta: Metadata of a web page, including descriptions of the page, keywords, and other information, usually in <meta> tags. For example, <meta name＝"description"content＝"这是一个提供优质商品的购物网站"> . Indicates that the meta of the website is: This is a shopping website that provides high-quality products.

[0086] body: The main content of the webpage, including all visible elements displayed on the webpage, such as text, images, links, etc. For example, body{text-align:center;background:#fff;width:100%} means that the main content of the webpage is: the text within the body element is horizontally aligned to the center, the background color is set to white, and the width of the body element is made to fill the entire viewport width.

[0087] CSS: Cascading Style Sheets, used to define the style and layout of a web page, may exist in an external CSS file or in the HTML <style>标签中。

[0088] 比如，<style>.title{color:#333；font-weight:bold；}.content{line-height:1.5；padding:20px；}< / style>, indicates that the css is: define the element style of the class name title and content, directly acting on the corresponding element in the webpage, affecting its display effect.

[0089] js: JavaScript script, used to realize the interactive function of the webpage, which may be in the external js file or HTML <script>标签内。

[0090] 比如，<script>document.getElementById("btn").addEventListener("click",function(){alert("按钮被点击了！")；})；< / script> , indicates that the js is: realize the interactive function of the pop-up prompt box when clicking the button with id btn, which belongs to the embedded in <script>标签内的js脚本。

[0091] script：网页中的脚本标签，用于引入或编写脚本代码，除了js脚本，还可能包括其他类型的脚本。

[0092] 比如，<script>console.log("这是一段js脚本")；< / script> , indicates that the script tag in the webpage is: output the information of the js script.

[0093] For example, the risk set is "suspected code leakage": "title = "csdn" && (body = ".php" || body = ".java")", indicating that the risk category is suspected code leakage, and the risk features include title and body.

[0094] Specifically, the risk features refer to the features that the title is csdn and the body includes.php or includes.java.

[0095] In some embodiments, the computer device extracts the target website to obtain the website host host corresponding to the target website. Further, the computer device finds the risk set corresponding to the target website in the risk set index table based on the host.

[0096] The risk set index table includes the search index corresponding to the risk set, the risk category and the risk feature.

[0097] The storage form of the mapping relationship between the search index corresponding to the risk set, the risk category and the risk feature is not limited in the embodiments of the present application, which can be stored in the form of text or in the form of table. For example, Table 1 is a storage method in the form of table.

[0098] Table 1

[0099]

[0100] For example, the computer device extracts the target website https: / / csdn.cn / 51232.html to obtain the website host host corresponding to the target website: csdn.cn. Based on csdn.cn, find the corresponding risk set in the risk set index table:

[0101] 1. The risk category is suspected code leakage, and the risk feature is "title = "csdn" && (body = ".php" || body = ".java")".

[0102] 2. The risk category is suspected password leakage, and the risk signature is "(body=\"password\"||body=\"password\")".

[0103] For example, the computer device extracts the target website https: / / baidu.com / and obtains the website host host: baidu.com corresponding to the target website. Based on baidu.com, the risk set corresponding to the target website is searched in the risk set index table:

[0104] 1. The risk category is suspected code leakage, and the risk feature is "title=\"baidu\"&&(body=\".php\"||body=\".java\")".

[0105] 2. The risk category is suspected password leakage, and the risk signature is "(body=\"password\"||body=\"password\")".

[0106] Step S202: The computer device extracts webpage feature data from the target website based on the risk category.

[0107] Among them, web page feature data refers to various types of feature data that are directly related to risk categories (such as suspected code leakage risk, suspected password leakage risk, etc.) and can reflect the potential risk attributes of the web page.

[0108] For example, from the risk category of suspected code leakage and the risk feature of "title=\"csdn\"&&(body=\".php\"||body=\".java\")", the web page feature data that needs to be extracted is the data corresponding to title and boby.

[0109] Optionally, the method of extracting web page feature data from the target website includes at least one of the following: regular expression extraction, XPath extraction, and CSS selector extraction, but is not limited thereto and is not specifically limited in the embodiments of the present application.

[0110] 1. Regular expression extraction: Use XPath expressions to accurately locate specific elements in a web page and extract their text or attributes.

[0111] For example, / / title / text() is used to extract the title of a web page, and / / body / / text() is used to extract the main content of a web page.

[0112] 2. XPath extraction: through <title>(.*？)< / title> Expression to extract web page title. Among them, <title> and< / title>For fixed tag matching, (.*?) represents non-greedy matching of all characters within the tag (including line breaks, multi-line mode needs to be turned on).

[0113] For example, if the title part in HTML is <title> CSDN Blog< / title> , the web page title extracted using this expression is: CSDN blog.

[0114] The main content of the web page is extracted by (.*?).

[0115] For example, for This is a text section <pre> Code snippet< / pre> , first extract This is a text section <pre> Code snippet< / pre> , and then remove the tags to get the pure text this is a paragraph of text code snippet. Therefore, the main content of the extracted web page is: this is a paragraph of text code snippet.

[0116] 3. CSS selector extraction: extract <title> Page title. <b class="d_n">

[0117] For example, if the title in HTML is <title>Java Preliminary Development Guide< / title> by document.querySelector('title'), execute document.querySelector('title'), and extract the web page title as: Java's preliminary development guide.

[0118] The main content of the web page is extracted by the textContent attribute.

[0119] For example, if the body in HTML is technology forum-Python tutorial, executing document.querySelector('title').textContent will extract the main content of the web page as: technology forum-Python tutorial.

[0120] Step S203: The computer device determines the identification result of the target website based on the web feature data and the risk feature.

[0121] Optionally, the identification result of the target website includes suspected code leakage risk, suspected password leakage, and no risk, but not limited to this, and the present application embodiment does not make specific limitation thereto.

[0122] Specifically, the detailed steps of the computer device to determine the identification result of the target website include:

[0123] Step 1: The computer device calculates the similarity between the web feature data and the risk feature to obtain a similarity value.

[0124] The computer device obtains the similarity value by quantifying the matching degree of the web feature data and the risk feature. The similarity value is usually a value of 0-1 or 0-100, and the higher the value, the higher the matching degree.

[0125] In some embodiments, the computer device calculates the similarity value between the web page feature data and the risk feature by combining keyword matching with an edit distance algorithm.

[0126] The following describes in detail the process of calculating similarity values ​​for computer devices, taking the risk characteristics of suspected code leakage risks as an example.

[0127] For example, the computer device extracts the title of the target web page (such as <title> CSDN Blog< / title> , the extracted result title is: CSDN Blog.

[0128] Based on the risk feature title = "csdn", determine whether the title of the target webpage contains the csdn (case-insensitive). If it does, calculate the edit distance between the csdn and the corresponding field in the title.

[0129] For example, if the title includes CSDN blog, the edit distance with csdn is 0 (only the difference in uppercase and lowercase can be eliminated by preprocessing), then the similarity value is 100%.

[0130] The specific calculation process of the edit distance is as follows: compare the string lengths of CSDN blog and csdn, both of which are 4, and the characters in each position are exactly the same (c vs c, s vs s, d vs d, n vs n). The number of required editing operations (insertion, deletion, replacement) is 0, that is, the edit distance is 0.

[0131] Exemplarily, the computer device extracts the body of the target webpage, such as downloading index.phc, and the extraction result body is: download index.phc.

[0132] Based on the risk feature body = "".php", determine whether the body of the target webpage contains ".php". If so, calculate the edit distance between ".php" and the corresponding field in the body.

[0133] If the body download index.phc does not include .php, the similarity value is 0.

[0134] Exemplarily, the computer device extracts the body of the target webpage, such as downloading Demo.java, and the extraction result body is: downloading Demo.java.

[0135] Based on the risk feature body = "".java", determine whether the body of the target webpage contains ".java". If so, calculate the edit distance between ".java" and the corresponding field in the body.

[0136] The similarity value is 100% when the edit distance between.java and.java is 0.

[0137] Step 2: The computer device determines a feature matching result set between the webpage feature data and the risk features according to the similarity values.

[0138] The feature matching result set refers to a result set of "true" or "false" for each risk feature, which is obtained by combining the similarity threshold and the similarity value between the webpage feature data and the risk feature. For example, the feature matching result set of the suspected code leakage risk obtained in step 1 is [true, true, false].

[0139] In a possible implementation, the computer device determines the similarity value greater than or equal to the similarity threshold as a first feature matching result.

[0140] For example, the similarity threshold is 90%. The similarity threshold can be set according to actual needs.

[0141] For example, the similarity value calculated by the computer device is 100%, and the first feature matching result is true.

[0142] In another possible implementation, the computer device determines the similarity value less than the similarity threshold as a second feature matching result.

[0143] For example, the similarity threshold is 90%.

[0144] For example, the similarity value calculated by the computer device is 0, and the second feature matching result is false.

[0145] In some embodiments, the computer device aggregates the first feature matching result or the second feature matching result of the host calculated according to the risk features to obtain a feature matching result set of the host.

[0146] Step 3: The computer device determines the identification result of the target website according to the feature matching result set.

[0147] In some embodiments, the computer device combines the feature matching results in the feature matching result set based on a logical combination relationship between the risk features to determine the identification result of the target website.

[0148] The logical combination relationship includes at least one of logical and, logical or, and logical not.

[0149] Logical and means that all the feature matching results participating in the operation must be "true" at the same time, and the final determination is "true". As long as one is "false", the result is "false".

[0150] Logical OR: refers to the feature matching results involved in the operation. As long as one of them is "true", the final judgment is "true". Only when all the results are "false" will the result be "false".

[0151] Logical NOT: negate the result of a single feature match, that is, "true" becomes "false" and "false" becomes "true".

[0152] For example, the feature matching result set of the target website calculated by the computer device in step 1 is [true, true, false]. The operation formula obtained by the feature splicing logic defined by the suspected code leakage risk in the target website is [true&&(true||false)]. Based on [true, true, false], the computer device calculates the result of the operation formula [true&&(true||false)] as 1 (i.e., true). The identification result of the target website is determined to be the existence of a suspected code leakage risk, that is, the target website has a suspected code leakage risk.

[0153] In summary, the technical solution provided by the present application can quickly locate the risk set related to the target website by obtaining the target website and the risk set corresponding to the target website, avoiding polling and matching of all risk sets, thereby reducing unnecessary calculations and improving execution efficiency. Based on the risk category, web page feature data is extracted from the target website, and web page feature data can be extracted in a targeted manner according to the risk category, thereby improving the efficiency of feature extraction. Based on the web page feature data and risk characteristics, the identification result of the target website is determined, and through direct comparison of the web page feature data with the risk characteristics, the probability of misjudgment and missed judgment is effectively reduced, and the accuracy of risk identification is improved.

[0154] Figure 3 The schematic diagram of a website identification method provided in an embodiment of the present application may include:

[0155] Step S301: The computer device obtains web page data and the website host based on the target website.

[0156] In some embodiments, the host is extracted by parsing the Uniform Resource Locator (URL) of the target website.

[0157] For example, if the URL of the target website is https: / / www.example.com / page1.html, the host after resolution is www.example.com;

[0158] If the URL of the target website is http: / / csdn.cn / 51232.html, the host after resolution is csdn.cn.

[0159] Specifically, when a computer device obtains web page data based on a target website, it usually first establishes a connection with the target website server through a network request, such as HyperText Transfer Protocol (HTTP), and sends a request instruction to obtain the web page data. After receiving the request instruction, the target website server returns the corresponding web page data.

[0160] Among them, the web page data includes at least one of HTML documents, CSS style sheets, JavaScript scripts, pictures, etc., but is not limited to these. The embodiments of the present application do not make specific limitations on this.

[0161] Step S302: The computer device obtains a risk rule set from the risk set index table based on the website host.

[0162] The risk rule set is the risk set described above.

[0163] The specific implementation steps can refer to step S201 and are not described in detail here.

[0164] Step S303: The computer device parses the risk rule set to obtain a risk rule feature set and a risk rule feature value set.

[0165] The risk rule feature set is the set of risk features described above, and the risk rule feature value set is the set of values ​​corresponding to the risk features described above.

[0166] The specific implementation steps can refer to step S201 and are not described in detail here.

[0167] Step S304: The computer device extracts web page feature data based on the web page data using the risk rule feature set.

[0168] The specific implementation steps can refer to step S202 and are not described in detail here.

[0169] Step S305: The computer device calculates based on the webpage feature data and the risk rule feature value set to obtain a risk feature similarity set.

[0170] The risk feature similarity set is the feature matching result set described above.

[0171] The specific implementation steps can refer to step S203 and are not described in detail here.

[0172] Step S306: Calculate based on the risk feature similarity set and finally obtain the risk matching result.

[0173] Among them, the risk matching result is also the identification result described above.

[0174] The specific implementation steps can refer to step S203 and are not described in detail here.

[0175] In summary, the technical solution provided by this application can quickly locate rules related to specific websites by creating a risk set index, avoiding polling and matching of all rules. This targeted matching reduces unnecessary calculations, thereby improving the execution efficiency of the algorithm.

[0176] The above mainly introduces the solution provided by this application. Correspondingly, this application also provides a website identification device, which is used to implement the above method embodiment.

[0177] like Figure 4 The schematic diagram of the website identification device is shown in FIG. 4 , which may include an acquisition module 401, an extraction module 402, and a determination module 403. The acquisition module 401 is used to execute Figure 2 The operation of step S201 in the illustrated method, and Figure 3 The operations of step S301, step S302 and step S303 in FIG; the extraction module 402 is used to perform Figure 2 The operation of step S202, and Figure 3 The operation of step S304 in the determination module 403 is used to perform Figure 5 The operation of step S203 in the determination module 403 is also used to perform Figure 5 The operations of step S305 and step S306.

[0178] In some embodiments, in order to implement the above functions, the website identification device includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0179] The embodiment of the present application can divide the website identification device into functional modules according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into a feature extraction module. The above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. In actual implementation, other division methods can be used.

[0180] As ​ shown, the computer device provided by the embodiments of the present application can include a processor 501, a bus 502, a communication interface 503, and a memory 504. The processor 501, the memory 504, and the communication interface 503 communicate through the bus 502. It should be understood that the number of processors and memories in the network device is not limited by the present application.

[0181] The bus 502 can be a PCI bus or an extended industry standard architecture (EISA) bus, or a UB bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, ​ only one line is used, but it does not mean that there is only one bus or only one type of bus. The bus 502 can include a path for transmitting information between various components (for example, the memory 504, the processor 501, and the communication interface 503) of the network device.

[0182] The processor 501 can include any one or more of a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0183] The memory 504 can include a volatile memory, such as a random access memory (RAM). The processor 501 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard disk drive (HDD), or a solid state drive (SSD).

[0184] The communication interface 503 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to realize the communication between the network device and other devices or communication networks.

[0185] The memory 504 stores executable program codes, and the processor 501 executes the executable program codes to realize the functions of the foregoing method embodiments. That is, the memory 504 stores instructions for executing the website identification method.

[0186] In still another aspect, a computer readable storage medium is provided, and the computer readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the website identification method provided by any of the above method embodiments.

[0187] In still another aspect, a computer program product is provided, and the computer program product comprises a computer program or instructions, which, when executed by a processor, implement the website identification method provided by any of the above method embodiments.

[0188] Those skilled in the art can clearly understand the technical effects of the groove feature extraction module, the computer readable storage medium and the computer program product in the embodiments of the present application by the description of the above embodiments, and for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the module is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, module and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0189] Since the groove feature extraction module, the computer readable storage medium and the computer program product in the embodiments of the present application can be applied to the above method, the technical effects they can obtain can also refer to the above method embodiments, and the embodiments of the present application will not be described here.

[0190] The method steps in the embodiments can be realized by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device. Of course, the processor and the storage medium can also exist as discrete components in the network device.

[0191] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function of the embodiment of the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable module. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0192] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A website identification method, characterized in that: The method comprises: Obtaining a target website and a risk set corresponding to the target website, wherein the risk set includes risk categories and risk characteristics; extracting webpage feature data from the target website based on the risk category; Based on the webpage feature data and the risk feature, an identification result of the target website is determined.

2. The method according to claim 1, characterized in that The determining of the identification result of the target website based on the webpage feature data and the risk feature includes: Calculating the similarity between the webpage feature data and the risk feature to obtain a similarity value; Determining a feature matching result set between the webpage feature data and the risk feature according to the similarity value; The identification result of the target website is determined according to the feature matching result set.

3. The method according to claim 2, characterized in that Determining a feature matching result set between the webpage feature data and the risk feature based on the similarity value includes: Determine the similarity value greater than or equal to the similarity threshold as a first feature matching result; Determine the similarity value that is smaller than the similarity threshold as a second feature matching result; The first feature matching result and the second feature matching result are aggregated to obtain the feature matching result set.

4. The method according to claim 2, characterized in that Determining the identification result of the target website according to the feature matching result set includes: Based on the logical combination relationship between the risk features, the feature matching results in the feature matching result set are merged to obtain the identification result.

5. The method according to claim 4, characterized in that The logical combination relationship includes at least one of logical AND, logical OR, and logical NOT.

6. The method according to any one of claims 1 to 5, characterized in that The obtaining of the risk set corresponding to the target website includes: Extract the target website to obtain the website host corresponding to the target website; Based on the host, the risk set corresponding to the target website is searched in a risk set index table, where the risk set index table includes a search index, a risk category, and a risk feature corresponding to the risk set.

7. The method according to any one of claims 1 to 5, characterized in that The risk features include: at least one of: web page title title, metadata meta, main content body, cascading style sheet css, script js, and script tag script.

8. A website identification device, characterized in that: The device includes: an acquisition module, an extraction module and a determination module; The acquisition module is used to acquire a target website and a risk set corresponding to the target website, wherein the risk set includes risk categories and risk characteristics; The extraction module is used to extract webpage feature data from the target website based on the risk category; The determination module is used to determine the identification result of the target website based on the webpage feature data and the risk feature.

9. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the website identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the website identification method according to any one of claims 1 to 7.

Images