Webpage identification method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a webpage recognition method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining a target page address of a to-be-recognized webpage and feedback information for the to-be-recognized webpage, and extracting the page content of the to-be-recognized webpage based on the target page address; performing risk assessment on the target page address and the page content to obtain a risk assessment value of each to-be-identified webpage; when the risk assessment value exceeds a preset risk threshold value, risk information of the webpage to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification; and when the risk assessment value does not exceed a preset risk threshold value, sending the page content to a risk identification server for identification so as to obtain a target identification result of the to-be-identified webpage. According to the invention, the efficiency and accuracy of illegal webpage identification can be improved. The embodiment of the invention can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a web page recognition method, device, electronic device and readable storage medium. Background Art

[0002] With the development of the Internet and mobile communication networks, the functions and elements provided by Internet pages are becoming more and more abundant. For example, people can use the Internet for social interaction, learning and entertainment. However, the black and gray industrial chain of the Internet has also emerged quietly, and the number of illegal web pages has also increased year by year. Auditors are required to identify and handle illegal web pages to cut off the harm of the black and gray industrial chain on the Internet.

[0003] However, the audit platform receives a large number of web pages to be audited every day. Most audit platforms will reduce the audit pressure by random sampling and merging web pages with the same domain name. However, both of the above two methods will miss some illegal web pages, and the missed illegal web pages will be returned to the audit platform due to re-reporting, so that the existing methods will seriously reduce the audit quality and efficiency of illegal web pages. Summary of the invention

[0004] The embodiments of the present application provide a web page identification method, device, electronic device and storage medium, which can improve the efficiency and accuracy of identifying illegal web pages.

[0005] A first aspect of an embodiment of the present application provides a webpage identification method, the method comprising:

[0006] Acquire a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address;

[0007] Performing risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page;

[0008] When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified;

[0009] When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

[0010] Accordingly, a second aspect of an embodiment of the present application provides a webpage recognition device, the device comprising:

[0011] A page content extraction unit, used to obtain a target page address of a web page to be identified and feedback information for the web page to be identified, and extract the page content of the web page to be identified based on the target page address;

[0012] A risk assessment value determination unit, configured to perform risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value indicates the credibility of the web page to be identified as a risky web page;

[0013] a first target identification result determination unit, configured to determine the risk information of the web page to be identified based on the target page address and feedback information when the risk assessment value exceeds a preset risk threshold, and send the risk information to a risk identification server for identification to obtain a target identification result of the web page to be identified;

[0014] The second target identification result determination unit is used to send the page content to the risk identification server for identification when the risk assessment value does not exceed the preset risk threshold, so as to obtain the target identification result of the web page to be identified.

[0015] Optionally, the target recognition result first determination unit includes:

[0016] a target risk type receiving subunit, configured to send the risk information to a risk server for identification when the risk assessment value exceeds a preset risk threshold, and receive a target risk type corresponding to the risk information returned by the risk identification server;

[0017] A current risk type determination subunit, used to determine the current risk type of the web page to be identified according to the risk assessment value;

[0018] The risk type comparison subunit is used to compare the target risk type with the current risk type to obtain a target recognition result of the web page to be recognized.

[0019] Optionally, the risk type comparison subunit includes:

[0020] a risk webpage determination subunit, configured to determine that the webpage to be identified is a risk webpage when the target risk type is the same as the current risk type;

[0021] The safe webpage determination subunit is used to determine that the webpage to be identified is a safe webpage when the target risk type is different from the current risk type, or that the webpage to be identified is a risky webpage of other risk types.

[0022] Optionally, the device further comprises:

[0023] A risky web page quantity acquisition unit, used to identify the host corresponding to each risky web page, and acquire the number of risky web pages of each host;

[0024] A risk host identification unit, configured to determine that the host is identified as a risk host when the number of risky web pages of the host exceeds a preset number threshold;

[0025] The attribute information acquisition unit is used to acquire the attribute information of the host when the number of risky web pages of the host does not exceed the preset number threshold, and send the attribute information to the risk identification server for identification to obtain the risk identification result of the host.

[0026] Optionally, the attribute information includes a page address and a label corresponding to the host;

[0027] The attribute information acquisition unit is further specifically used for:

[0028] Sending the page address and the tag to the risk identification server so that the risk identification server can perform a risk assessment on the host based on the page address and the tag;

[0029] Receiving a risk assessment result of the host returned by the risk server;

[0030] When the risk assessment result indicates that the host has a risk, the host is determined to be a risky host.

[0031] Optionally, the target recognition result second determination unit includes:

[0032] an identification priority determination subunit, used to determine the identification priority of each web page to be identified based on the page content;

[0033] A page content sorting subunit, used to sort the page content according to the identification priority;

[0034] The target recognition result determination subunit is used to recognize the sorted page content to obtain the target recognition result of the web page to be recognized.

[0035] Optionally, the identification priority determination subunit includes:

[0036] Obtaining priority calculation information of each web page to be identified and a weight value corresponding to the priority calculation information, wherein the priority calculation information includes complaint information, activity information, and page address information;

[0037] For each web page to be identified, the weight value of the complaint information, the weight value of the activity information and the weight value of the page address information are integrated to obtain the priority weight of each web page to be identified;

[0038] The priority of each web page to be identified is determined according to the priority weight.

[0039] Optionally, the target recognition result determination subunit includes:

[0040] A basic risk type determination subunit, configured to determine the basic risk type of the web page to be identified according to the risk assessment value, and identify candidate risk types of the web page to be identified in the page content;

[0041] The risk type comparison second subunit is used to compare the basic risk type with the candidate risk type to obtain a target recognition result of the web page to be recognized.

[0042] Optionally, the risk type comparison second subunit is further specifically used for:

[0043] When the basic risk type is the same as the candidate risk type, determining that the webpage to be identified is a risky webpage;

[0044] When the basic risk type is different from the candidate risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

[0045] Optionally, the device further comprises:

[0046] A target webpage screening unit, used to screen out at least one target webpage corresponding to the same target page address from the webpages to be identified;

[0047] A page content merging unit, used to merge the page contents of the target web pages to obtain the target page content corresponding to each target page address;

[0048] The target recognition result determination subunit is further specifically used to: recognize the target page content, obtain an initial recognition result of each target page address, and use the initial recognition result as the target recognition result of the web page to be recognized corresponding to the target page address.

[0049] Optionally, the risk assessment value determination unit is further specifically configured to:

[0050] Using a web page recognition model to extract features from the target page address and the page content to obtain web page features of each web page to be recognized;

[0051] Based on the web page features, the web page recognition model is used to predict the risk assessment value of the web page to be recognized.

[0052] Optionally, the device further comprises:

[0053] A risk web page feature extraction unit, configured to extract risk web page features of at least one dimension from the web page to be identified when the web page to be identified is a risk web page;

[0054] The web page recognition model updating unit is used to update the web page recognition model based on the risk web page features of the web page to be recognized and the target recognition result, obtain an updated web page recognition model, and use the updated web page recognition model as the web page recognition model.

[0055] Optionally, the risk web page features include link features, image features, text features and web page scene features, and the web page recognition model includes a sparse feature processing network and a dense feature processing network;

[0056] The web page recognition model updating unit is also specifically used for:

[0057] Training the sparse feature processing network according to the link features to obtain an updated sparse feature processing network, and using the updated sparse feature processing network as the sparse feature processing network of the web page recognition model;

[0058] The dense feature processing network is trained according to the image features, the text features and the web page scene features to obtain an updated dense feature processing network, and the updated dense feature processing network is used as the dense feature processing network of the web page recognition model.

[0059] A third aspect of an embodiment of the present application provides an electronic device, including:

[0060] Processors and storage media;

[0061] The processor is used to implement each instruction;

[0062] The storage medium is used to store a plurality of instructions, and the instructions are used to be loaded by a processor and execute the web page recognition method described above.

[0063] A fourth aspect of an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for a processor to load to execute the steps in any one of the web page identification methods provided in the embodiments of the present application.

[0064] A fifth aspect of the embodiments of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements any one of the web page recognition methods provided in the embodiments of the present application.

[0065] It can be seen that by applying the embodiments of the present application, the target page address of the web page to be identified and the feedback information for the web page to be identified can be obtained, and based on the target page address, the page content of the web page to be identified can be extracted, and the target page address and the page content can be used to accurately assess the risk of the web page to be identified, and obtain the risk assessment value of each web page to be identified.

[0066] Since the risk assessment value represents the credibility of the web page to be identified as a risky web page, therefore, for web pages to be identified with higher risk assessment values, their credibility as risky web pages is higher, and only the target page address and feedback information of such web pages to be identified are reviewed to determine whether they are risky web pages; for web pages with medium and low risk assessment values, their credibility as risky web pages is average, and their page content needs to be reviewed.

[0067] Therefore, through the above method, targeted classification and identification operations can be performed on the web pages to be identified based on the risk assessment value, and web pages with greater risk hazards can be quickly and batch reviewed to improve the efficiency of web page identification, and web pages with general risk hazards can be reviewed in detail to avoid omissions and errors in the identification of risky web pages. Therefore, the embodiments of the present application can significantly improve the efficiency and accuracy of web page identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 It is a schematic diagram of an application scenario of the web page identification method provided in an embodiment of the present application;

[0070] Figure 2a It is a flowchart of a web page identification method provided in an embodiment of the present application;

[0071] Figure 2b It is a flowchart of a web page identification method provided in an embodiment of the present application;

[0072] Figure 3a is a schematic diagram of obtaining feedback information of a web page to be identified provided in an embodiment of the present application;

[0073] Figure 3b is a schematic diagram of identifying risk assessment values ​​and risk types of web pages provided by an embodiment of the present application;

[0074] Figure 3c is another flow chart of the web page identification method provided in an embodiment of the present application;

[0075] Figure 3d It is a schematic diagram of the ecosystem of the web page recognition service provided by the embodiment of the present application;

[0076] Figure 4 is a structural diagram of a web page recognition device provided in an embodiment of the present application;

[0077] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0079] In order to realize the automatic subcontracting operation of the application, improve the subcontracting efficiency, reduce the complexity of subcontracting and the human resources required for subcontracting, the embodiments of the present application provide a web page recognition method, device, medium and equipment. The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the drawings, where the same or similar numbers throughout represent the same or similar elements or elements with the same or similar functions.

[0080] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0081] In order to facilitate understanding of the technical solutions described in the embodiments of the present application and the technical effects produced therefrom, the embodiments of the present application explain the relevant professional terms involved:

[0082] Webview: is a control or component that embeds web content in an application. It allows developers to display web content in mobile applications without requiring objects to leave the application and enter a separate browser application. By using webview, developers can embed web content directly into the application's interface, thereby displaying online information, interactive content, or other web elements in the application.

[0083] URL: Uniform Resource Locator, is a string used to identify the location of resources on the Internet. URL provides a concise way to describe the location of network resources and how to obtain these resources.

[0084] Domain: A domain name is a human-readable name used to identify and locate a computer or server on the Internet. A domain name is an easy-to-remember string used to access network resources instead of an IP address. It plays a role similar to that of an address in the real world, making it easier for objects to access websites, send emails, etc.

[0085] Host: A host or host name refers to a computer or device connected to the Internet or other network. A host can be a server, a personal computer, a mobile phone, a router, or any device that can communicate with the network. Each host has a unique identifier, usually an IP address or a domain name. For example, in the URL https: / / abc.def.cn / test, def.cn is the domain name and abc.def.cn is the host.

[0086] The embodiment of the present invention provides a web page recognition method, device and readable storage medium. The web page recognition device can be integrated in an electronic device, which can be a server or a terminal.

[0087] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0088] For example, see Figure 1, taking the web page identification device integrated in the server as an example, the server obtains the target page address of the web page to be identified and the feedback information for the web page to be identified, and extracts the page content of the web page to be identified based on the target page address. A risk assessment can be performed on the target page address and the page content to obtain a risk assessment value for each web page to be identified, and the risk assessment value represents the credibility of the web page to be identified as a risky web page. Then, when the risk assessment value exceeds the preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to the risk identification server for identification to obtain the target identification result of the web page to be identified. Finally, when the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain the target identification result of the web page to be identified.

[0089] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0090] This embodiment will be described from the perspective of a web page recognition device, which can be specifically integrated into an electronic device, which can be a server or a terminal; wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device or other smart devices that can update plug-ins.

[0091] A web page recognition method, comprising:

[0092] Acquire a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address;

[0093] Performing risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page;

[0094] When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified;

[0095] When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

[0096] The following introduces a web page recognition method provided by this application. Figure 2aThis is a flowchart of a web page recognition method provided by an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Please refer to Figure 2a A web page identification method provided in an embodiment of the present application may include the following steps:

[0097] Step 101: Obtain a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address.

[0098] Among them, the web pages to be identified refer to the web pages that may have attack risks and are to be reviewed. In daily Internet scenarios, some black and gray industrial chains will be born, such as online scams, personal information transactions, adult video transactions, etc. In order to cut off the spread and harm of these black and gray industrial chains, network security-related departments need to conduct comprehensive and accurate review and identification of massive web pages.

[0099] The target page address is the link of the web page to be identified, that is, the uniform resource locator URL, which is used to describe the location of the web page to be identified on the Internet. For example, taking the target page address https: / / abc.def.cn / test as an example, def.cn is the domain name domain, and abc.def.cn is the host name host.

[0100] Among them, feedback information refers to the complaints and suggestions from Internet users on the web pages to be identified. In some scenarios, Internet users may find a web page containing malicious or erroneous content, and then they can file a complaint or suggestion on the web page in a specific channel, thereby generating feedback information corresponding to the web page to be identified. The feedback information may include the complaint content, screenshots, and reasons for the complaint against the web page to be identified, thereby providing richer contextual information for the subsequent identification and review of the web page to be identified.

[0101] See also Figure 3a , Figure 3a Schematic diagram of obtaining feedback information of a web page to be identified provided by an embodiment of the present application. Internet and application users can complain about the web page to be identified on a browser page or a webview page of an application to generate feedback information corresponding to the web page to be identified. Figure 3aAs shown, taking the scenario of method one as an example, an Internet user can, as a complainant, directly file a complaint about the current page to be identified in the browser page or web view page. The complaint page displays controls corresponding to the complaint and reporting functions. The complainant can enter the complaint process after clicking the complaint control. After entering the complaint process, the complainant can choose to provide screenshots of the risky and illegal content on the web page to be identified to generate feedback information.

[0102] Taking the scenario of method 2 as an example, it is a scenario of complaining about an illegal account on the web page to be identified. After entering the complaint process, the complainant can provide chat content evidence, as well as QR codes, links, pictures and other evidence to generate richer feedback information as context information of the web page to be identified.

[0103] It is understood that after obtaining the target page address, the actual page content of the web page to be identified can be extracted based on the target page address. For example, the page content can include multiple elements such as text, images, and other links in the page. The above elements can then be processed into features that can be recognized by the machine learning model and input into the relevant model for analysis.

[0104] Step 102: Perform risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified.

[0105] Among them, the risk assessment value represents the credibility of the web page to be identified as a risky web page. In some embodiments, the risk assessment value can be a value not greater than 1, such as 0.9, 90%, etc., indicating that the credibility of the web page to be identified as a risky web page is 90%. Of course, the risk assessment value can also be a value greater than 1, but an upper limit value of the risk assessment value must be set. For example, the upper limit value of the risk assessment value is set to 100. When the risk assessment value of the web page to be identified is 50, it means that the credibility of the page to be identified as a risky page is 50%. Regarding the numerical range of the risk assessment value setting, this embodiment does not limit this.

[0106] In some embodiments, different weights may be assigned to different features when calculating the risk assessment value in order to more accurately assess the risk. It is also possible to determine whether to classify a web page to be identified as a risky web page by setting a threshold. The weights and thresholds may be set based on historical data, experimental results, and professional knowledge.

[0107] It can be understood that by obtaining the risk assessment value of the web page to be identified, a higher risk assessment value indicates that the system believes that the web page to be identified is more likely to contain malicious or bad content, while a lower risk assessment value indicates that the web page to be identified has a lower risk. Therefore, the risk assessment value can be used as a basis for identification decisions. For example, the risk assessment value can be used to decide whether to add the current web page to be identified to the fast review queue or the priority review queue.

[0108] Optionally, step 102 may include:

[0109] Using a web page recognition model to extract features from the target page address and the page content to obtain web page features of each web page to be recognized;

[0110] Based on the web page features, the web page recognition model is used to predict the risk assessment value of the web page to be recognized.

[0111] See also Figure 3b , Figure 3b Schematic diagram of identifying risk assessment values ​​and risk types of web pages provided by the embodiment of the present application. Figure 3b As shown, in this application, since the features of the web page to be identified include multiple types of features such as links, text, images, animations, etc., a trained multimodal machine learning model can be used to calculate the risk assessment value of the web page to be identified, for example Figure 3b After model reasoning, the risk assessment value and risk type of the web page to be identified are output.

[0112] In the model selection stage, two candidate neural network models, the residual network ResNet-18 and the classification tree model Xgboost, are first used for experiments. First, the training data is divided into two groups. The first group of training data only includes image features of one type of risk web pages. For example, the image features of adult risk web pages can be vector features extracted from adult pictures in the web pages. Correspondingly, the second group of training data can include multiple types of features of risk web pages, such as image features, text features, link features, complaint message features, etc.

[0113] Furthermore, after preparing the above two sets of training data, we obtain ResNet-18 and Xgboost, two models with similar performance, input the first set of data into ResNet-18 and the second set of data into Xgboost. The predicted data of model inference can be shown in Table 1 below.

[0114] Accuracy Recall F1 score Support Number ResNet-18 0.63 0.49 0.55 0.77 Xgboost 0.67 0.60 0.63 0.9

[0115] Table 1

[0116] According to the data provided in Table 1, it can be seen that compared with training samples of a single type of feature, adding different types of features as training samples in the training data can significantly improve the performance of the model. Therefore, the present application can fine-tune the structure of ResNet-18, add a multi-layer perceptron for image encoding before the fully connected layer of ResNet-18, and obtain the web page recognition model of the present application.

[0117] Optionally, the method of the present application may further include:

[0118] When the web page to be identified is a risky web page, extracting risky web page features of at least one dimension from the web page to be identified;

[0119] Based on the risk web page features of the web page to be identified and the target identification result, the web page identification model is updated to obtain an updated web page identification model, and the updated web page identification model is used as the web page identification model.

[0120] Among them, risky web page features may include link features, image features, text features, and web page scene features, and the web page recognition model may include a sparse feature processing network and a dense feature processing network. Among them, link features may include the length and number of jumps of the link, and web page scene features may be the contextual information or environmental information of the web page to be identified, such as the location, time, and column type of the illegal content in the web page to be identified.

[0121] It can be understood that after extracting the features of the risky web pages, it is also necessary to obtain the target recognition results of the risky web pages and use the target recognition results as labels for training samples. For example, the target recognition results of a risky web page are "adult risk web pages", "fake red envelope risk web pages", etc.

[0122] Optionally, the step of “updating the web page recognition model” includes:

[0123] Training the sparse feature processing network according to the link features to obtain an updated sparse feature processing network, and using the updated sparse feature processing network as the sparse feature processing network of the web page recognition model;

[0124] The dense feature processing network is trained according to the image features, the text features and the web page scene features to obtain an updated dense feature processing network, and the updated dense feature processing network is used as the dense feature processing network of the web page recognition model.

[0125] It can be understood that the web page recognition model after fine-tuning on the structure of ResNet-18 can solve the defect that the original model cannot process dense features due to its simple structure. In some embodiments, the web page recognition model of the present application adopts the wide & deep model idea in the feature fusion part, so that the model takes into account both memory ability and generalization ability while integrating multiple features. The sparse feature processing network uses a linear sub-model to enhance memory learning to process sparse features such as link features, and the dense feature processing network uses a deep learning sub-model to learn dense features after sparse feature encoding, such as image features, text features, and web page scene features.

[0126] It should be noted, however, that during the training process of the web page recognition model, corresponding training weights can be set for different types of training samples, so that the web page recognition model can pay more attention to training samples with high weights to improve the training efficiency and effect of the model. For example, the training samples of image features can usually be set to correspond to the highest training weights. This embodiment does not limit the details of setting the training weights.

[0127] In some embodiments, the outputs of the dense feature processing network and the coefficient feature processing network can be jointly fitted with the objective function to obtain the training results of the web page recognition model as shown in Table 2 below, where type 0 in Table 2 indicates that the web page to be identified is a normal web page, type 1 indicates that the web page to be identified is a risky web page of the fake red envelope type, and type 2 indicates that the web page to be identified is a risky web page of the adult type.

[0128] Accuracy Recall F1 score Support Number Type 0 0.98 1 0.99 523 Type 1 1 1 1 163 Type 2 0.99 0.97 0.98 235 Accuracy / / 0.99 921 Average Macro 0.99 0.99 0.99 921 Weighted Average 0.99 0.99 0.99 921

[0129] Table 2

[0130] Step 103: When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified.

[0131] It can be understood that when the calculated risk assessment value exceeds the preset risk threshold, it means that the web page to be identified is judged as a high-risk or suspicious risk web page with a high degree of credibility. Combined with the data in Table 2, it can also be seen that the performance of the web page identification model has reached an excellent level of more than 99% successful prediction. Therefore, for web pages to be identified with higher risk assessment values, it is no longer necessary to perform high-intensity audit operations on them. In the fast audit channel, only the target page address and feedback information of such web pages can be audited and identified to determine whether they are ultimately risky web pages, thereby improving the audit and identification efficiency of the web pages to be identified. For example, it can be identified whether the URL of such web pages is too long or whether there are suspicious fields. For example, the feedback information can be a thumbnail of the web page to be identified sent by the complainant. The audit server or relevant auditors only need to identify whether the thumbnail is consistent with the risk web page type predicted by the web page identification model through audit.

[0132] Optionally, step 103 may include:

[0133] When the risk assessment value exceeds a preset risk threshold, the risk information is sent to a risk server for identification, and a target risk type corresponding to the risk information returned by the risk identification server is received;

[0134] Determining the current risk type of the web page to be identified according to the risk assessment value;

[0135] The target risk type is compared with the current risk type to obtain a target recognition result of the web page to be recognized.

[0136] As mentioned above, when receiving such web pages to be identified whose risk assessment values ​​exceed the preset risk threshold, there is no need to conduct high-intensity review of such web pages, and there is no need to click on the target page address to obtain the specific page content. It is only necessary to identify whether the target page address and feedback information of such web pages to be identified are abnormal.

[0137] As an example only, it can be identified that the corresponding risk type is a fake red envelope type based on the thumbnail of the web page to be identified, and the risk assessment value indicates that the current risk type of the web page to be identified is also a fake red envelope type, which means that the result output by the web page recognition model after further review is correct, and the target recognition result obtained is that the web page to be identified is indeed a risky web page of this risk type.

[0138] Optionally, the step of “comparing the target risk type with the current risk type to obtain a target identification result of the web page to be identified” includes:

[0139] When the target risk type is the same as the current risk type, determining that the webpage to be identified is a risky webpage;

[0140] When the target risk type is different from the current risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

[0141] According to the foregoing description, when the target risk type is the same as the current risk type, it means that the recognition result output by the web page recognition model is correct, and it can be clearly identified that the web page to be identified is definitely a risky web page.

[0142] In some scenarios, since the recognition results of the web page recognition model are not necessarily absolutely accurate, when the target risk type is different from the current risk type, it means that the recognition result of the web page recognition model is wrong, and it is necessary to further identify whether the web page to be recognized is a safe web page or a risky web page of other risk types. For example, the recognition result given by the web page recognition model is that the current web page is a fake red envelope risk web page, but after a step of review and recognition, it is found that the target risk type of the web page is no specific type or a safe type. For example, the target risk type can be an adult type or a fraud type. Therefore, after determining that the recognition result of the web page recognition model is wrong, it is necessary to further determine whether the web page to be recognized has risks.

[0143] Optionally, the method of the present application further includes:

[0144] Identify the host corresponding to each risky web page, and obtain the number of risky web pages on each host;

[0145] When the number of risky web pages on the host exceeds a preset number threshold, determining that the host is identified as a risky host;

[0146] When the number of risky web pages of the host does not exceed the preset number threshold, the attribute information of the host is obtained, and the attribute information is sent to the risk identification server for identification to obtain a risk identification result of the host.

[0147] See also Figure 3c , Figure 3c FIG. 2 is another flow chart of the web page identification method provided by the embodiment of the present application. Figure 3c As shown, after the risk assessment of the web pages to be identified is performed and the risk assessment value is obtained, the hosts corresponding to the web pages to be identified can be further identified. In this way, the situation where multiple web pages on the same host may all be identified as risky web pages can be identified. For each host, the system will count the number of risky web pages on it.

[0148] In some embodiments, once the number of risky web pages on each host is identified, the system will analyze the risk of the host. If the number of risky web pages on a host exceeds a preset threshold, then the host will be identified as a risky host, indicating that the malicious activity on the host may be more serious and the host needs to be automatically processed and detected more strictly.

[0149] However, for hosts whose number of risky web pages does not exceed the preset number threshold, the system will consider further analysis. The system will obtain the attribute information of these hosts, which can include the page address and tags corresponding to the host. This attribute information can provide more contextual information about the host and help to better understand the nature of the host. This attribute information will be sent to the risk identification server, which will use advanced risk identification algorithms and models to identify and analyze these hosts to obtain risk identification results for the hosts.

[0150] It can be understood that when the number of risky web pages of a host does not exceed the preset number threshold, since the previous identification results have shown that some risky web pages have been identified on the host, it cannot be directly concluded that the host is not a risky host with network threats. Therefore, the host's attribute information can be sent to the priority review channel for further review and identification to determine whether the host is a risky host.

[0151] Through the above methods, while quickly and batch reviewing the web pages to be identified in the fast review channel, it is also possible to better grasp the risk level of the host and perform corresponding processing and management according to the situation. At the same time, the acquisition and transmission of the host's attribute information further provides the host's context information, which helps to more accurately identify and judge network risks.

[0152] Optionally, the step of “sending the attribute information to the risk identification server for identification” includes:

[0153] Sending the page address and the tag to the risk identification server so that the risk identification server can perform a risk assessment on the host based on the page address and the tag;

[0154] Receiving a risk assessment result of the host returned by the risk server;

[0155] When the risk assessment result indicates that the host has a risk, the host is determined to be a risky host.

[0156] In some embodiments, after receiving the attribute information of a host whose number of risky web pages does not exceed a preset number threshold, the host can be identified based on the attribute information to determine whether the host is a risky host. For example, it can be identified whether the link of the host contains sensitive keywords, such as sensitive keywords that can be common domain names or characters of risky links. For another example, it can be identified whether the label classification of the host is a risky host and a label commonly used for risky web pages, such as a risky host of the fraud type or fake red envelope type may use a financial label, and a risky host of the adult type may use an art label, so as to identify whether the host is a risky host based on the attribute information of the host.

[0157] Step 104: When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

[0158] It can be understood that for web pages to be identified whose risk assessment values ​​do not exceed the preset risk threshold, it means that the credibility of such web pages to be identified as risky web pages is average, but in order to further confirm whether such web pages to be identified are safe web pages, more rigorous identification work is still required. Specifically, it is necessary to open the specific page content through the target page address of the web page to be identified, and use the page content as an important basis for reviewing and identifying whether the web page is a risky web page.

[0159] like Figure 3c As shown, such web pages to be identified can be sent to the risk identification server corresponding to the priority review channel.

[0160] Optionally, step 104 may include:

[0161] Determining the recognition priority of each web page to be recognized based on the page content;

[0162] sorting the page contents according to the identification priority;

[0163] The sorted page contents are identified to obtain a target identification result of the web page to be identified.

[0164] In some embodiments, for each web page to be identified, a corresponding identification priority can be calculated based on the features, attributes, and possible risk factors in the page content. The identification priority can not only determine the priority of audit identification, but also reflect the possibility or importance of the web page to be identified as a risky web page. Calculating the identification priority may involve a variety of factors, such as the source of the web page to be identified, the sensitivity of the content, the suspiciousness of the link, etc.

[0165] It can be understood that by improving the efficiency and effect of web page auditing and identification through the solution of this application, it is possible to increase the number and accuracy of processing risky web pages and risky links, and improve the influence and high profitability of auditing and identification, while keeping the identification cost unchanged. Therefore, when receiving some web pages to be identified with a general or low preset risk threshold, the risky web pages in such web pages to be identified cannot be omitted. It is necessary to concentrate identification resources to give priority to reviewing the parts of such web pages to be identified that are closer to the risky web pages, so that those web pages to be identified with higher identification priorities can be identified first.

[0166] Optionally, the step of “determining the recognition priority of each web page to be recognized based on the page content” includes:

[0167] Obtaining priority calculation information of each web page to be identified and a weight value corresponding to the priority calculation information, wherein the priority calculation information includes complaint information, activity information, and page address information;

[0168] For each web page to be identified, the weight value of the complaint information, the weight value of the activity information and the weight value of the page address information are integrated to obtain the priority weight of each web page to be identified;

[0169] The priority of each web page to be identified is determined according to the priority weight.

[0170] In some embodiments, the corresponding identification priority can be calculated based on the priority calculation information of each web page to be identified. In some embodiments, the complaint information can be the number of complaints and the number of people who made complaints corresponding to the web page to be identified, the activity information can be the number of link clicks on the web page to be identified, and the page address information can include whether the page address has a suspicious domain name, whether it has been blocked and blacklisted, and whether it belongs to an authoritative domain name.

[0171] It can be understood that the method of the present application mentioned above can give priority to the review of web pages to be identified with high complaints and high-impact feedback information. In other words, the higher the number of clicks and the higher the popularity of the web pages to be identified that are pushed to the priority review channel, once they are treated as risky web pages, the greater the benefits of review and identification. Therefore, the process of calculating the identification priority can be likened to a voting mechanism. The complaint information, activity information, and the links to domain names and links that have been intercepted vote in favor of the identification priority, and the links to authoritative domain names vote against the identification priority. The total number of votes is calculated by combining the number of votes with the weight of each information. The higher the number of votes the web page to be identified gets, the higher its corresponding identification priority.

[0172] Optionally, the step of “identifying the sorted page contents to obtain a target identification result of the web page to be identified” includes:

[0173] Determining the basic risk type of the web page to be identified according to the risk assessment value, and identifying candidate risk types of the web page to be identified in the page content;

[0174] The basic risk type is compared with the candidate risk type to obtain a target recognition result of the web page to be recognized.

[0175] It can be understood that, similar to the example given above, as an example only, assuming that the basic risk type identified by the web page recognition model is the fake red envelope type, and the candidate risk type identified based on the page content is also the fake red envelope type, it means that after further review, the result output by the web page recognition model is correct, and the target recognition result obtained is that the web page to be identified is indeed a risky web page of this risk type.

[0176] Optionally, the step of “comparing the basic risk type with the candidate risk type to obtain a target identification result of the web page to be identified” includes:

[0177] When the basic risk type is the same as the candidate risk type, determining that the webpage to be identified is a risky webpage;

[0178] When the basic risk type is different from the candidate risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

[0179] It can be understood that, similar to the examples given above, since the recognition results of the web page recognition model are not necessarily absolutely accurate, when the basic risk type is different from the candidate risk type, it means that the recognition result of the web page recognition model is wrong, and it is necessary to further identify whether the web page to be identified is a safe web page or a risky web page of other risk types. Just as an example, suppose the recognition result given by the web page recognition model is that the current web page is a fake red envelope risk web page, and after a step of review and identification, it is found that the target risk type of the web page is no specific type or a safe type. For example, the target risk type can be an adult type or a fraud type. Therefore, after determining that the recognition result of the web page recognition model is wrong, it is necessary to further determine whether the web page to be identified has risks.

[0180] Optionally, before the step of “identifying the sorted page contents”, the method further includes:

[0181] Filtering out at least one target web page corresponding to the same target page address from the web pages to be identified;

[0182] Merging the page contents of the target web pages to obtain the target page content corresponding to each target page address;

[0183] The identifying of the sorted page contents to obtain the target identification result of the web page to be identified includes: identifying the target page contents to obtain an initial identification result of each target page address, and using the initial identification result as the target identification result of the web page to be identified corresponding to the target page address.

[0184] like Figure 3c As shown, multiple web pages to be identified can be merged. It is understandable that in the scenario of the present application, the same web page to be identified may be complained multiple times by one or more complainants, which will increase unnecessary identification pressure and identification costs. Therefore, the method of the present application can merge target pages from the same target page address, that is, merge multiple web pages from the same link.

[0185] It can be understood that compared with the traditional method of merging web pages to be identified based on domain names, although the frequency and number of merging will be reduced, it can avoid the defect of missing different linked web pages when merging web pages based on domain names, so that the web page merging method of the present application can reduce the pressure of web page identification and improve the efficiency of web page review and identification without affecting the accuracy of web page identification.

[0186] See also Figure 3d , Figure 3d Schematic diagram of the ecosystem of the web page recognition service provided by the embodiment of the present application. Figure 3dAs shown in the figure, in the ecosystem of web page identification business, while reviewing the web pages and links to be identified, it can also provide valuable training samples for the training of web page identification models. By analyzing the web pages and links to be identified reported by Internet users, a large number of training samples of different types of risky web pages can be collected, so that the web page identification model can be trained to identify and prevent these risky web pages, which helps to continuously improve the identification algorithm of the model and enhance the overall security of the system.

[0187] In the process of reviewing and identifying web pages to be identified, the features used can also be used in other aspects of network security environment governance. For example, the features and information extracted from the web pages to be identified can be further extracted and analyzed to construct more accurate features to further optimize the web page identification process.

[0188] In some embodiments, the audit and identification process of reporting web pages to be identified can also become a feedback link in the ecosystem. By analyzing the identification results, some points that can be improved in the identification process can be found. For example, it is possible to identify which type of web pages with potential risks are reported the most, and then adjust the allocation strategy of auditors to improve the audit efficiency.

[0189] In some embodiments, the identification results can be used to verify the recall rate of the entire ecosystem. By counting the proportion of risky web pages among the reported web pages to be identified, the detection capability of the entire ecosystem for risky web pages can be evaluated, thereby helping to adjust the model parameters of the web page identification model and the risky web page identification algorithm to ensure the continued effective operation of the entire ecosystem.

[0190] In some embodiments, reporters can be divided into high-credibility groups and low-credibility groups, and the distribution of these groups can be verified based on the reporting data. In addition, the coverage calculation, that is, the coverage rate of the business ecosystem to be identified, can be verified through the reported web page data to ensure the comprehensiveness of the identifiable scope of the entire ecosystem, so as to make timely and effective rectification of the accounts, group chats and links of the risky web page manufacturing team.

[0191] It can be seen that by applying the embodiments of the present application, the target page address of the web page to be identified and the feedback information for the web page to be identified can be obtained, and based on the target page address, the page content of the web page to be identified can be extracted, and the target page address and the page content can be used to accurately assess the risk of the web page to be identified, and obtain the risk assessment value of each web page to be identified.

[0192] Since the risk assessment value represents the credibility of the web page to be identified as a risky web page, therefore, for web pages to be identified with higher risk assessment values, their credibility as risky web pages is higher, and only the target page address and feedback information of such web pages to be identified are reviewed to determine whether they are risky web pages; for web pages with medium and low risk assessment values, their credibility as risky web pages is average, and their page content needs to be reviewed.

[0193] Therefore, through the above method, targeted classification and identification operations can be performed on the web pages to be identified based on the risk assessment value, and web pages with greater risk hazards can be quickly and batch reviewed to improve the efficiency of web page identification, and web pages with general risk hazards can be reviewed in detail to avoid omissions and errors in the identification of risky web pages. Therefore, the embodiments of the present application can significantly improve the efficiency and accuracy of web page identification.

[0194] Among them, another specific process of the above web page identification method is as follows, refer to Figure 2b , the method further comprises the following steps:

[0195] Step 201: obtaining a target page address and feedback information of a web page to be identified, and extracting the page content of the web page to be identified based on the target page address;

[0196] Step 202: Perform risk assessment on the target page address and page content to obtain a risk assessment value for each web page to be identified;

[0197] Step 203: Send the risk information to a risk server for identification, and receive a target risk type corresponding to the risk information returned by the risk identification server;

[0198] Step 204: Determine the current risk type of the web page to be identified based on the risk assessment value;

[0199] Step 205: Compare the target risk type with the current risk type to obtain a target recognition result of the web page to be recognized;

[0200] Step 206: When the risk assessment value does not exceed the preset risk threshold, determining the recognition priority of each web page to be recognized based on the page content;

[0201] Step 207: sort the page content according to the recognition priority;

[0202] Step 208: Identify the sorted page contents to obtain a target identification result of the web page to be identified.

[0203] The method described in the above embodiment will be further described in detail below.

[0204] like Figure 4 FIG. 1 is a schematic diagram of the structure of a web page recognition device provided in an embodiment of the present application, and the device includes:

[0205] The page content extraction unit 301 is used to obtain the target page address of the web page to be identified and the feedback information for the web page to be identified, and extract the page content of the web page to be identified based on the target page address;

[0206] A risk assessment value determination unit 302 is used to perform risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page;

[0207] The target identification result first determination unit 303 is used to determine the risk information of the web page to be identified based on the target page address and feedback information when the risk assessment value exceeds a preset risk threshold, and send the risk information to the risk identification server for identification to obtain the target identification result of the web page to be identified;

[0208] The target recognition result second determination unit 304 is used to send the page content to the risk recognition server for recognition when the risk assessment value does not exceed the preset risk threshold, so as to obtain the target recognition result of the web page to be recognized.

[0209] Optionally, the target recognition result first determination unit 303 includes:

[0210] a target risk type receiving subunit, configured to send the risk information to a risk server for identification when the risk assessment value exceeds a preset risk threshold, and receive a target risk type corresponding to the risk information returned by the risk identification server;

[0211] A current risk type determination subunit, used to determine the current risk type of the web page to be identified according to the risk assessment value;

[0212] The risk type comparison subunit is used to compare the target risk type with the current risk type to obtain a target recognition result of the web page to be recognized.

[0213] Optionally, the risk type comparison subunit includes:

[0214] a risk webpage determination subunit, configured to determine that the webpage to be identified is a risk webpage when the target risk type is the same as the current risk type;

[0215] The safe webpage determination subunit is used to determine that the webpage to be identified is a safe webpage when the target risk type is different from the current risk type, or that the webpage to be identified is a risky webpage of other risk types.

[0216] Optionally, the device further comprises:

[0217] A risky web page quantity acquisition unit, used to identify the host corresponding to each risky web page, and acquire the number of risky web pages of each host;

[0218] A risk host identification unit, configured to determine that the host is identified as a risk host when the number of risky web pages of the host exceeds a preset number threshold;

[0219] The attribute information acquisition unit is used to acquire the attribute information of the host when the number of risky web pages of the host does not exceed the preset number threshold, and send the attribute information to the risk identification server for identification to obtain the risk identification result of the host.

[0220] Optionally, the attribute information includes a page address and a label corresponding to the host;

[0221] The attribute information acquisition unit is further specifically used for:

[0222] Sending the page address and the tag to the risk identification server so that the risk identification server can perform a risk assessment on the host based on the page address and the tag;

[0223] Receiving a risk assessment result of the host returned by the risk server;

[0224] When the risk assessment result indicates that the host has a risk, the host is determined to be a risky host.

[0225] Optionally, the target recognition result second determination unit 304 includes:

[0226] an identification priority determination subunit, used to determine the identification priority of each web page to be identified based on the page content;

[0227] A page content sorting subunit, used to sort the page content according to the identification priority;

[0228] The target recognition result determination subunit is used to recognize the sorted page content to obtain the target recognition result of the web page to be recognized.

[0229] Optionally, the identification priority determination subunit includes:

[0230] Obtaining priority calculation information of each web page to be identified and a weight value corresponding to the priority calculation information, wherein the priority calculation information includes complaint information, activity information, and page address information;

[0231] For each web page to be identified, the weight value of the complaint information, the weight value of the activity information and the weight value of the page address information are integrated to obtain the priority weight of each web page to be identified;

[0232] The priority of each web page to be identified is determined according to the priority weight.

[0233] Optionally, the target recognition result determination subunit includes:

[0234] A basic risk type determination subunit, configured to determine the basic risk type of the web page to be identified according to the risk assessment value, and identify candidate risk types of the web page to be identified in the page content;

[0235] The risk type comparison second subunit is used to compare the basic risk type with the candidate risk type to obtain a target recognition result of the web page to be recognized.

[0236] Optionally, the risk type comparison second subunit is further specifically used for:

[0237] When the basic risk type is the same as the candidate risk type, determining that the webpage to be identified is a risky webpage;

[0238] When the basic risk type is different from the candidate risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

[0239] Optionally, the device further comprises:

[0240] A target webpage screening unit, used to screen out at least one target webpage corresponding to the same target page address from the webpages to be identified;

[0241] A page content merging unit, used to merge the page contents of the target web pages to obtain the target page content corresponding to each target page address;

[0242] The target recognition result determination subunit is further specifically used to: recognize the target page content, obtain an initial recognition result of each target page address, and use the initial recognition result as the target recognition result of the web page to be recognized corresponding to the target page address.

[0243] Optionally, the risk assessment value determination unit 302 is further specifically configured to:

[0244] Using a web page recognition model to extract features from the target page address and the page content to obtain web page features of each web page to be recognized;

[0245] Based on the web page features, the web page recognition model is used to predict the risk assessment value of the web page to be recognized.

[0246] Optionally, the device further comprises:

[0247] A risk web page feature extraction unit, configured to extract risk web page features of at least one dimension from the web page to be identified when the web page to be identified is a risk web page;

[0248] The web page recognition model updating unit is used to update the web page recognition model based on the risk web page features of the web page to be recognized and the target recognition result, obtain an updated web page recognition model, and use the updated web page recognition model as the web page recognition model.

[0249] Optionally, the risk web page features include link features, image features, text features and web page scene features, and the web page recognition model includes a sparse feature processing network and a dense feature processing network;

[0250] The web page recognition model updating unit is also specifically used for:

[0251] Training the sparse feature processing network according to the link features to obtain an updated sparse feature processing network, and using the updated sparse feature processing network as the sparse feature processing network of the web page recognition model;

[0252] The dense feature processing network is trained according to the image features, the text features and the web page scene features to obtain an updated dense feature processing network, and the updated dense feature processing network is used as the dense feature processing network of the web page recognition model.

[0253] It can be seen that by applying the embodiments of the present application, the target page address of the web page to be identified and the feedback information for the web page to be identified can be obtained, and based on the target page address, the page content of the web page to be identified can be extracted, and the target page address and the page content can be used to accurately assess the risk of the web page to be identified, and obtain the risk assessment value of each web page to be identified.

[0254] Since the risk assessment value represents the credibility of the web page to be identified as a risky web page, therefore, for web pages to be identified with higher risk assessment values, their credibility as risky web pages is higher, and only the target page address and feedback information of such web pages to be identified are reviewed to determine whether they are risky web pages; for web pages with medium and low risk assessment values, their credibility as risky web pages is average, and their page content needs to be reviewed.

[0255] Therefore, through the above method, targeted classification and identification operations can be performed on the web pages to be identified based on the risk assessment value, and web pages with greater risk hazards can be quickly and batch reviewed to improve the efficiency of web page identification, and web pages with general risk hazards can be reviewed in detail to avoid omissions and errors in the identification of risky web pages. Therefore, the embodiments of the present application can significantly improve the efficiency and accuracy of web page identification.

[0256] The present application also provides an electronic device, which may be a terminal, a server, or other device. Figure 5 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0257] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input unit 404, and a communication unit 405. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not limit the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0258] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or units stored in the memory 402, and calling data stored in the memory 402. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, display interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0259] The memory 402 can be used to store software programs and units. The processor 401 executes various functional applications and data processing by running the software programs and units stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0260] The electronic device also includes a power supply 403 for supplying power to various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0261] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to object setting and function control.

[0262] The electronic device may further include a communication unit 405. In some embodiments, the communication unit 405 may include a wireless unit. The electronic device may perform short-range wireless transmission through the wireless unit of the communication unit 405, thereby providing the object with wireless broadband Internet access. For example, the communication unit 405 may be used to help the object send and receive emails, browse web pages, and access streaming media.

[0263] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0264] Acquire a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address;

[0265] Performing risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page;

[0266] When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified;

[0267] When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

[0268] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0269] It can be seen that by applying the embodiments of the present application, the target page address of the web page to be identified and the feedback information for the web page to be identified can be obtained, and based on the target page address, the page content of the web page to be identified can be extracted, and the target page address and the page content can be used to accurately assess the risk of the web page to be identified, and obtain the risk assessment value of each web page to be identified.

[0270] Since the risk assessment value represents the credibility of the web page to be identified as a risky web page, therefore, for web pages to be identified with higher risk assessment values, their credibility as risky web pages is higher, and only the target page address and feedback information of such web pages to be identified are reviewed to determine whether they are risky web pages; for web pages with medium and low risk assessment values, their credibility as risky web pages is average, and their page content needs to be reviewed.

[0271] Therefore, through the above method, targeted classification and identification operations can be performed on the web pages to be identified based on the risk assessment value, and web pages with greater risk hazards can be quickly and batch reviewed to improve the efficiency of web page identification, and web pages with general risk hazards can be reviewed in detail to avoid omissions and errors in the identification of risky web pages. Therefore, the embodiments of the present application can significantly improve the efficiency and accuracy of web page identification.

[0272] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the web page recognition methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0273] Acquire a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address;

[0274] Performing risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page;

[0275] When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified;

[0276] When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

[0277] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0278] Since the instructions stored in the storage medium can execute the steps in any one of the web page identification methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the web page identification methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0279] The web page recognition method, device, electronic device and storage medium provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the present application are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A web page recognition method, characterized in that: The method comprises: Acquire a target page address of a web page to be identified and feedback information for the web page to be identified, and extract page content of the web page to be identified based on the target page address; Performing risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value represents the credibility of the web page to be identified as a risky web page; When the risk assessment value exceeds a preset risk threshold, the risk information of the web page to be identified is determined based on the target page address and the feedback information, and the risk information is sent to a risk identification server for identification to obtain a target identification result of the web page to be identified; When the risk assessment value does not exceed the preset risk threshold, the page content is sent to the risk identification server for identification to obtain a target identification result of the web page to be identified.

2. The method according to claim 1, characterized in that The step of sending the risk information to a risk identification server for identification to obtain a target identification result of the web page to be identified includes: When the risk assessment value exceeds a preset risk threshold, the risk information is sent to a risk server for identification, and a target risk type corresponding to the risk information returned by the risk identification server is received; Determining the current risk type of the web page to be identified according to the risk assessment value; The target risk type is compared with the current risk type to obtain a target recognition result of the web page to be recognized.

3. The method according to claim 2, characterized in that The comparing the target risk type with the current risk type to obtain a target identification result of the web page to be identified includes: When the target risk type is the same as the current risk type, determining that the webpage to be identified is a risky webpage; When the target risk type is different from the current risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

4. The method according to claim 3, characterized in that The method further comprises: Identify the host corresponding to each risky web page, and obtain the number of risky web pages on each host; When the number of risky web pages on the host exceeds a preset number threshold, determining that the host is identified as a risky host; When the number of risky web pages of the host does not exceed the preset number threshold, the attribute information of the host is obtained, and the attribute information is sent to the risk identification server for identification to obtain a risk identification result of the host.

5. The method according to claim 4, characterized in that The attribute information includes a page address and a label corresponding to the host; The sending the attribute information to the risk identification server for identification includes: Sending the page address and the tag to the risk identification server so that the risk identification server can perform a risk assessment on the host based on the page address and the tag; Receiving a risk assessment result of the host returned by the risk server; When the risk assessment result indicates that the host has a risk, the host is determined to be a risky host.

6. The method according to claim 1, characterized in that The sending of the page content to the risk identification server for identification to obtain a target identification result of the web page to be identified includes: Determining the recognition priority of each web page to be recognized based on the page content; sorting the page contents according to the identification priority; The sorted page contents are identified to obtain a target identification result of the web page to be identified.

7. The method according to claim 6, characterized in that The step of determining the recognition priority of each web page to be recognized based on the page content includes: Obtaining priority calculation information of each web page to be identified and a weight value corresponding to the priority calculation information, wherein the priority calculation information includes complaint information, activity information, and page address information; For each web page to be identified, the weight value of the complaint information, the weight value of the activity information and the weight value of the page address information are integrated to obtain the priority weight of each web page to be identified; The priority of each web page to be identified is determined according to the priority weight.

8. The method according to claim 6, characterized in that The identifying the sorted page contents to obtain a target identification result of the web page to be identified includes: Determining the basic risk type of the web page to be identified according to the risk assessment value, and identifying candidate risk types of the web page to be identified in the page content; The basic risk type is compared with the candidate risk type to obtain a target recognition result of the web page to be recognized.

9. The method according to claim 8, characterized in that The comparing the basic risk type with the candidate risk type to obtain a target recognition result of the web page to be recognized includes: When the basic risk type is the same as the candidate risk type, determining that the webpage to be identified is a risky webpage; When the basic risk type is different from the candidate risk type, it is determined that the webpage to be identified is a safe webpage, or that the webpage to be identified is a risky webpage of another risk type.

10. The method according to claim 6, characterized in that Before identifying the sorted page content, the method further includes: Filtering out at least one target web page corresponding to the same target page address from the web pages to be identified; Merging the page contents of the target web pages to obtain the target page content corresponding to each target page address; The identifying of the sorted page contents to obtain the target identification result of the web page to be identified includes: identifying the target page contents to obtain an initial identification result of each target page address, and using the initial identification result as the target identification result of the web page to be identified corresponding to the target page address.

11. The method according to claim 1, characterized in that: The risk assessment of the target page address and the page content to obtain a risk assessment value of each web page to be identified includes: Using a web page recognition model to extract features from the target page address and the page content to obtain web page features of each web page to be recognized; Based on the web page features, the web page recognition model is used to predict the risk assessment value of the web page to be recognized.

12. The method according to claim 11, characterized in that Also includes: When the web page to be identified is a risky web page, extracting risky web page features of at least one dimension from the web page to be identified; Based on the risk web page features of the web page to be identified and the target identification result, the web page identification model is updated to obtain an updated web page identification model, and the updated web page identification model is used as the web page identification model.

13. The method according to claim 12, characterized in that The risk web page features include link features, image features, text features and web page scene features, and the web page recognition model includes a sparse feature processing network and a dense feature processing network; The updating of the web page recognition model includes: Training the sparse feature processing network according to the link features to obtain an updated sparse feature processing network, and using the updated sparse feature processing network as the sparse feature processing network of the web page recognition model; The dense feature processing network is trained according to the image features, the text features and the web page scene features to obtain an updated dense feature processing network, and the updated dense feature processing network is used as the dense feature processing network of the web page recognition model.

14. A web page recognition device, characterized in that: The device comprises: A page content extraction unit, used to obtain a target page address of a web page to be identified and feedback information for the web page to be identified, and extract the page content of the web page to be identified based on the target page address; A risk assessment value determination unit, configured to perform risk assessment on the target page address and the page content to obtain a risk assessment value for each web page to be identified, wherein the risk assessment value indicates the credibility of the web page to be identified as a risky web page; a first target identification result determination unit, configured to determine the risk information of the web page to be identified based on the target page address and feedback information when the risk assessment value exceeds a preset risk threshold, and send the risk information to a risk identification server for identification to obtain a target identification result of the web page to be identified; The second target identification result determination unit is used to send the page content to the risk identification server for identification when the risk assessment value does not exceed the preset risk threshold, so as to obtain the target identification result of the web page to be identified.

15. An electronic device, characterized in that: include: Processors and storage media; The processor is used to implement each instruction; The storage medium is used to store a plurality of instructions, and the instructions are used to be loaded by a processor and execute the web page recognition method according to any one of claims 1 to 13.

16. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the web page recognition method according to any one of claims 1 to 13 is implemented.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the web page recognition method according to any one of claims 1 to 13 is implemented.

Images