Webpage identification processing method and apparatus, computer device, and readable storage medium

Abstract

Embodiments of the present application disclose a webpage identification processing method and device, computer equipment and a readable storage medium, comprising: obtaining a dynamic resource request of a webpage to be identified, the dynamic resource request being used to request a server to obtain real-time updated dynamic webpage resources and render interactive webpage content corresponding to the webpage to be identified; obtaining a static resource request of the webpage to be identified, the static resource request being used to request a server to obtain static webpage resources and render initial webpage content of the webpage to be identified; generating a set of resource requests to be identified based on the dynamic resource request and the static resource request, and obtaining resource request texts based on request fields included in each resource request in the set of resource requests to be identified; mapping the resource request texts to an identification feature sequence of the webpage to be identified, and generating a webpage identification of the webpage to be identified based on the identification feature sequence. By using the present application, the accuracy of obtaining the webpage identification can be improved, and the applicability is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a webpage identification processing method and device, computer equipment and a readable storage medium. BACKGROUND

[0002] The rapid development of Internet technology brings more and more convenience to people's life, and at the same time, the security situation of the Internet is not optimistic. For example, various types of black and gray production websites imitate normal websites to steal the account passwords of the objects, which not only infringes the legitimate rights and interests of the operating objects and other related subjects, but also pollutes the Internet ecology. Therefore, the modern Internet technology has derived a webpage identification detection technology, by obtaining the webpage identification of various websites, suspected malicious websites similar to the webpage identification of malicious websites can be screened out from a large number of websites, so as to output a risk prompt when the object accesses the suspected malicious website.

[0003] Generally speaking, the webpage identification can be constructed by extracting the Document Object Model (DOM) structure of the webpage from the HyperText Mark-up Language (HTML) file of the webpage, and the more rich the DOM structure is, the more reliable the webpage identification obtained according to the DOM structure of the webpage is. However, many malicious websites currently realize the display effect of the final page by dynamically embedding websites, and the accuracy of the webpage identification obtained by the browser according to the DOM structure of the webpage is reduced, the probability of misjudgment is high when comparing different webpages, and the applicability is poor. SUMMARY

[0004] The webpage identification processing method and device, computer equipment and readable storage medium provided by the embodiments of the present application can improve the accuracy of obtaining the webpage identification, and have high applicability.

[0005] In a first aspect, the embodiments of the present application provide a webpage identification processing method, comprising:

[0006] obtaining a dynamic resource request of a webpage to be identified, the dynamic resource request being used to request a server to obtain real-time updated dynamic webpage resources, the dynamic webpage resources being used to render into interactive webpage content corresponding to the webpage to be identified;

[0007] obtaining a static resource request of the webpage to be identified, the static resource request being used to request the server to obtain static webpage resources, the static webpage resources being used to render into initial webpage content of the webpage to be identified;

[0008] The resource request text of the to-be-identified webpage is mapped to an identification feature sequence of the to-be-identified webpage, and the webpage identification of the to-be-identified webpage is generated based on the identification feature sequence.

[0009] The resource request text of the to-be-identified webpage is mapped to an identification feature sequence of the to-be-identified webpage, and the webpage identification of the to-be-identified webpage is generated based on the identification feature sequence.

[0010] In a possible implementation, the obtaining of the dynamic resource request of the to-be-identified webpage includes:

[0011] A plurality of initial dynamic resource requests of the to-be-identified webpage are obtained, and a status code of each initial dynamic resource request is obtained, and an initial dynamic resource request with a status code equal to a preset status code in the plurality of initial dynamic resource requests is determined as the dynamic resource request of the to-be-identified webpage.

[0012] In a possible implementation, the obtaining of the static resource request of the to-be-identified webpage includes:

[0013] A plurality of initial static resource requests of the to-be-identified webpage are obtained, and a status code of each initial static resource request is obtained, and an initial static resource request with a status code equal to a preset status code in the plurality of initial static resource requests is determined as the static resource request of the to-be-identified webpage.

[0014] In a possible implementation, the mapping of the resource request text of the to-be-identified webpage to the identification feature sequence of the to-be-identified webpage includes:

[0015] Feature word extraction is performed on the resource request text of the to-be-identified webpage to obtain a plurality of feature words;

[0016] A weight coefficient of each feature word in the plurality of feature words is obtained, and the plurality of feature words are mapped to a plurality of weighted real number sequences based on the weight coefficient of each feature word;

[0017] The plurality of weighted real number sequences are accumulated to obtain an accumulated real number sequence of the to-be-identified webpage, and the accumulated real number sequence is converted into a binary number to obtain an identification feature sequence corresponding to the resource request text.

[0018] In a possible implementation, the feature word extraction performed on the resource request text of the to-be-identified webpage to obtain a plurality of feature words includes:

[0019] A separator string in the resource request text of the to-be-identified webpage is recognized, and the resource request text is divided into a plurality of words based on the recognized separator string.

[0020] classifying the plurality of words to obtain a plurality of classified words, wherein each classified word comprises a plurality of words of a same category;

[0021] obtaining a frequency of occurrence of each classified word based on a number of words included in the each classified word, and determining K classified words with higher frequencies of occurrence from the plurality of classified words, wherein K is a positive integer;

[0022] determining the K classified words as a plurality of feature words of the resource request text.

[0023] In a possible implementation, the obtaining of a weight coefficient of each feature word in the plurality of feature words and the mapping of the plurality of feature words into a plurality of weighted real number sequences based on the weight coefficient of each feature word include:

[0024] finding an inverse document frequency of each feature word in the plurality of feature words in a corpus;

[0025] obtaining a term frequency of each feature word based on the frequency of occurrence of the each feature word, and obtaining a weight coefficient of the each feature word based on the term frequency and the inverse document frequency of the each feature word;

[0026] mapping any feature word in the plurality of feature words into a binary sequence, and weighting the binary sequence of the any feature word based on the weight coefficient of the any feature word to obtain a weighted real number sequence of the any feature word;

[0027] obtaining a plurality of weighted real number sequences mapped by the plurality of feature words based on the weighted real number sequence of each feature word.

[0028] In a possible implementation, the mapping of any feature word in the plurality of feature words into a binary sequence and the weighting of the binary sequence of the any feature word based on the weight coefficient of the any feature word include:

[0029] mapping the any feature word in the plurality of feature words into a binary sequence of a preset length based on a hash function, the binary sequence of the any feature word comprising a plurality of binary values;

[0030] multiplying the weight coefficient of the any feature word by the plurality of binary values in the binary sequence of the any feature word to weight the binary sequence of the any feature word.

[0031] In a possible implementation, the accumulated real number sequence comprises a plurality of accumulated real numbers, and the binary conversion of the accumulated real number sequence comprises:

[0032] The accumulated real number greater than 0 in the accumulated real number sequence is replaced by 1, and the accumulated real number less than or equal to 0 in the accumulated real number sequence is replaced by 0, and the accumulated real number sequence obtained after the replacement of the accumulated real numbers is determined as the identification feature sequence corresponding to the resource request text to perform the binary conversion on the accumulated real number sequence.

[0033] In a possible implementation, after the generation of the web page identifier of the to-be-identified web page based on the identification feature sequence, the method further comprises:

[0034] finding a target web page with a similarity less than a preset threshold to the to-be-identified web page in a web page identification database based on the web page identifier of the to-be-identified web page, and classifying the to-be-identified web page based on a classification identifier of the target web page; wherein each web page in the web page identification database has a classification identifier, and the classification identifier of any web page is used for classifying the any web page.

[0035] In a second aspect, the embodiments of the present application further provide a web page identifier processing apparatus, comprising:

[0036] a resource request obtaining module, configured to obtain a dynamic resource request of a to-be-identified web page, wherein the dynamic resource request is used to request a server to obtain real-time updated dynamic web page resources, and the dynamic web page resources are used to render interactive web page content corresponding to the to-be-identified web page;

[0037] The resource request obtaining module is further configured to obtain a static resource request of the to-be-identified web page, wherein the static resource request is used to request the server to obtain static web page resources, and the static web page resources are used to render initial web page content of the to-be-identified web page;

[0038] a request text obtaining module, configured to generate a to-be-identified resource request set of the to-be-identified web page based on the dynamic resource request and the static resource request of the to-be-identified web page, and obtain a resource request text of the to-be-identified web page based on a request field included in each resource request in the to-be-identified resource request set;

[0039] a generating module, configured to map the resource request text of the to-be-identified web page to an identification feature sequence of the to-be-identified web page, and generate a web page identifier of the to-be-identified web page based on the identification feature sequence.

[0040] The resource request obtaining module comprises:

[0041] The dynamic resource request obtaining unit is configured to obtain a plurality of initial dynamic resource requests of the to-be-identified webpage, and obtain a status code of each initial dynamic resource request, and determine an initial dynamic resource request with a status code equal to a preset status code in the plurality of initial dynamic resource requests as a dynamic resource request of the to-be-identified webpage.

[0042] The resource request obtaining module further includes:

[0043] The static resource request obtaining unit is configured to obtain a plurality of initial static resource requests of the to-be-identified webpage, and obtain a status code of each initial static resource request, and determine an initial static resource request with a status code equal to a preset status code in the plurality of initial static resource requests as a static resource request of the to-be-identified webpage.

[0044] The generating module includes:

[0045] The feature word extracting unit is configured to perform feature word extraction on the resource request text of the to-be-identified webpage to obtain a plurality of feature words.

[0046] The weighting unit is configured to obtain a weight coefficient of each feature word in the plurality of feature words, and map the plurality of feature words into a plurality of weighted real number sequences based on the weight coefficient of each feature word.

[0047] The identification feature sequence generating unit is configured to accumulate the plurality of weighted real number sequences to obtain an accumulated real number sequence of the to-be-identified webpage, and perform binary conversion on the accumulated real number sequence to obtain an identification feature sequence corresponding to the resource request text.

[0048] The feature word extracting unit includes:

[0049] The word division subunit is configured to identify a separation string in the resource request text of the to-be-identified webpage, and divide the resource request text into a plurality of words based on the identified separation string.

[0050] The classification subunit is configured to classify the plurality of words to obtain a plurality of classified words, wherein one classified word includes a plurality of words of the same category.

[0051] The word frequency obtaining subunit is configured to obtain a frequency of occurrence of each classified word based on a number of words included in the classified word, and determine K classified words with higher frequency of occurrence from the plurality of classified words, wherein K is a positive integer.

[0052] The feature word determining subunit is configured to determine the K classified words as a plurality of feature words of the resource request text.

[0053] The weighting unit includes:

[0054] a searching subunit, configured to search for an inverse document frequency of each feature word in the corpus among the plurality of feature words;

[0055] a weight coefficient obtaining subunit, configured to obtain a term frequency of each feature word based on a frequency of occurrence of the feature word, and obtain a weight coefficient of each feature word based on the term frequency and the inverse document frequency of the feature word;

[0056] a weighted real number obtaining subunit, configured to map any feature word in the plurality of feature words into a binary sequence, and weight the binary sequence of the feature word based on the weight coefficient of the feature word, to obtain a weighted real number sequence of the feature word;

[0057] a weighted real number sequence obtaining subunit, configured to obtain a plurality of weighted real number sequences mapped by the plurality of feature words based on the weighted real number sequence of each feature word.

[0058] The weighted real number obtaining subunit is further configured to:

[0059] map any feature word in the plurality of feature words into a binary sequence of a preset length based on a hash function, the binary sequence of the feature word including a plurality of binary values;

[0060] multiply the weight coefficient of the feature word by the plurality of binary values in the binary sequence of the feature word to weight the binary sequence of the feature word.

[0061] The identification feature sequence generating unit is further configured to:

[0062] replace the accumulated real numbers greater than 0 in the accumulated real number sequence with 1, and replace the accumulated real numbers less than or equal to 0 in the accumulated real number sequence with 0, and determine the accumulated real number sequence obtained after the replacement as an identification feature sequence corresponding to the resource request text to perform binary conversion on the accumulated real number sequence.

[0063] The apparatus further includes:

[0064] a classification module, configured to search for a target webpage with a similarity to the webpage to be identified less than a preset threshold based on a webpage identifier of the webpage to be identified in a webpage identification database, and classify the webpage to be identified based on a classification identifier of the target webpage; each webpage in the webpage identification database has a classification identifier, and the classification identifier of any webpage is used to classify the webpage.

[0065] In a third aspect, the embodiments of the present application further provide a computer device, comprising a processor, a memory and a network interface.

[0066] The processor is connected with the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store program code, and the processor is configured to invoke the program code to execute the method in the embodiments of the present application.

[0067] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program comprises program instructions, and the program instructions are executed by a processor to execute the method in the embodiments of the present application.

[0068] The web page identification processing method provided in the embodiments of the present application can obtain a static resource request and a dynamic resource request of a to-be-identified web page, wherein the static resource request of the to-be-identified web page is used to request a static web page resource from a server, and the dynamic resource request of the to-be-identified web page is used to request a real-time updated dynamic web page resource. The static web page resource can generate initial web page content of the to-be-identified web page under the parsing and rendering of a terminal device, and the dynamic web page resource can generate interactive web page content of the to-be-identified web page under the parsing and rendering of the terminal device. It can be understood that when the web page content of different to-be-identified web pages is the same, the initial web page content and the interactive web page content of different to-be-identified web pages are the same, and because the static resource request or the dynamic resource request is the same, the web page content (the initial web page content and the interactive web page content) obtained by the request is also the same. Therefore, when the web page content of different to-be-identified web pages is the same, the static resource request and the dynamic resource request of different to-be-identified web pages are also the same. Therefore, the embodiments of the present application can obtain the web page identification of the to-be-identified web page through the static resource request and the dynamic resource request of the to-be-identified web page, and the web page identification of different to-be-identified web pages can be used to represent the similarity of different to-be-identified web pages. Further, the to-be-identified resource request set of the to-be-identified web page can be generated based on the dynamic resource request and the static resource request of the to-be-identified web page, the resource request text of the to-be-identified web page can be mapped into an identification feature sequence of the to-be-identified web page, and the web page identification of the to-be-identified web page can be generated based on the identification feature sequence. It can be understood that when the web page content of different to-be-identified web pages is the same, the to-be-identified resource request of different to-be-identified web pages is also the same. Therefore, the identification feature sequence obtained by mapping the resource request text of the to-be-identified web page can be used to represent the web page identification of the to-be-identified web page. The web page identification processing method of the embodiments of the present application can obtain the web page identification of the to-be-identified web page based on the dynamic resource request and the static resource request of the to-be-identified web page, can improve the accuracy of obtaining the web page identification of the to-be-identified web page when the to-be-identified web page is loaded in a website through the way of dynamically embedding the web page, can improve the accuracy of judging the similarity of different to-be-identified web pages, and has high applicability. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0070] Figure 1 is a system architecture diagram provided by the embodiments of the present application;

[0071] Figure 2 is a schematic diagram of an application scenario of the web page identification processing method provided by the embodiments of the present application;

[0072] Figure 3 is a flow diagram of a webpage identification processing method provided by an embodiment of the present application;

[0073] Figure 4 is another flow diagram of a webpage identification processing method provided by an embodiment of the present application;

[0074] Figure 5 is another flow diagram of a webpage identification processing method provided by an embodiment of the present application;

[0075] Figure 6 is another flow diagram of a webpage identification processing method provided by an embodiment of the present application;

[0076] Figure 7 is a calculation principle diagram of a webpage identification processing method provided by an embodiment of the present application;

[0077] Figure 8 is a structure diagram of a webpage identification processing apparatus provided by an embodiment of the present application;

[0078] Figure 9 is a structure diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0080] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence, and produces a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware level technology and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0081] Nature Language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, that is, the language used in daily life, so it has a close relationship with the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0082] The scheme provided by the embodiments of the present application relates to natural language processing technology and deep learning technology of artificial intelligence, which is specifically explained by the following embodiments. Please refer to Figure 1 , Figure 1 is a system architecture schematic diagram provided by the embodiments of the present application. As Figure 1As shown, the system architecture may include a business server 100 and a terminal cluster. The terminal cluster may include terminal devices 200a, 200b, 200c, ..., 200n, etc. The business server 100 may be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It may also be a cloud server providing basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal devices (including terminal devices 200a, 200b, 200c, ..., 200n) may be PDAs, smartphones, laptops, desktop computers, tablets, mobile internet devices (MIDs), wearable devices (such as smartwatches, smart bracelets, etc.), smart computers, smart in-vehicle devices, etc., but are not limited to these. The service server 100 can establish communication connections with each terminal device in the terminal cluster, and the terminal devices in the terminal cluster can also establish communication connections with each other. In other words, the service server 100 can establish communication connections with each terminal device among terminal devices 200a, 200b, 200c, ..., 200n. For example, terminal device 200a can establish a communication connection with the service server 100. Terminal devices 200a and 200b can establish a communication connection, and terminal devices 200a and 200c can also establish a communication connection. The communication connection method is not limited; it can be established directly or indirectly through wired communication or wireless communication, etc., depending on the actual application scenario. This application does not impose any restrictions on this.

[0083] It should be understood that, such as Figure 1 Each terminal device in the terminal cluster shown can have an application client installed. When the application client runs on each terminal device, it can interact with the aforementioned... Figure 1The illustrated business servers 100 interact with each other to enable the business servers 100 to receive service data from each terminal device or push service data (such as web resources) to each terminal device. The application client can be a browser application, a news application, a learning application, a social application, an instant messaging application, a live broadcast application, a short video application, a video application, a music application, a shopping application, a novel application, a payment application, or any other application that can display text, images, and video data. The application client can be a standalone client or an embedded sub-client integrated into a certain client (such as a browser client or a social client). For ease of description, the browser client is taken as an example. In the process of using the browser client through the terminal device, the user can view, click, collect, and share web multimedia resources in the target application through the terminal device. It should be understood that the web multimedia resources can be any multimedia data, including but not limited to audio, pictures, or video, and the like. The business server 100, as a server that provides web multimedia resources to the browser client, can be a collection of multiple servers, including a background server and a data processing server corresponding to the application client. The business server 100 can receive a web resource request from each terminal device and find a web resource matching the web resource request in the web resource database of the business server 100 based on the web resource request of each terminal device. The business server 100 can return the matching web resource to the terminal device requesting the web resource. The browser client can be an application client for displaying web pages to the user. Specifically, the application client can generate a corresponding web resource request based on the click operation of the user on the target URL and send it to the business server 100. Further, the application client can receive the web resource returned by the business server 100 and render the page content of the target URL based on the web resource to display the target web page corresponding to the target URL clicked by the user to the user.

[0084] It can be understood that in the process of the operation object accessing the Internet, the malicious website corresponding to the malicious website can be induced by the operation object to access the malicious website corresponding to the malicious website through the disguised website service content. Once the operation object operates the terminal device to enter these malicious websites, the Trojan horse, virus and other programs planted in the malicious websites will be triggered, causing the terminal device to be infected, facing the risk of losing account or private information. Therefore, the malicious website identifier database can be constructed in the above business server 100, and the webpage identifier of the verified malicious website (or the webpage identifier of the verified safe website or the webpage identifier of the malicious website) is stored in the malicious website identifier database. When the terminal device accesses the business server 100, the business server 100 can detect the website requested by the terminal device based on the malicious website identifier database to determine whether the website requested by the terminal device is a malicious website. Specifically, when the operation object inputs the target website in the address bar of the client application on the terminal device or directly clicks the target website, the terminal device can perform domain name system (DNS) resolution to map the domain name of the target website to the corresponding Internet protocol address (IP address), and send a webpage resource request to the business server 100 based on the IP address of the target website. The webpage resource request can be a hyper text transfer protocol (HTTP) request or a HTTPS (full hypertext transfer protocol secure) request, which can be understood as a request for the business server 100 to request to obtain the webpage resource used for loading and displaying the target website. Further, the business server 100 can download the target webpage resource corresponding to the target website based on the above webpage resource request, and when the target webpage resource fails to be downloaded, it can be considered that the current target website is a non-malicious website. The business server 100 ends the detection of the target website and returns a corresponding safety instruction to the terminal device, and the terminal device can further access the target website based on the safety instruction.

[0085] It can be understood that when the service server 100 downloads the target webpage resource corresponding to the target website based on the webpage resource request, when the target webpage resource is downloaded successfully, the service server 100 can send the downloaded target webpage resource to the webpage identifier processing apparatus to identify and process the webpage identifier of the target webpage resource. Specifically, the webpage identifier processing apparatus can generate a unique target webpage identifier based on the target webpage resource. Since similar websites usually have similar webpage identifiers, the service server 100 can determine whether the target website is a malicious website by searching for a malicious website identifier of the same category as the target webpage identifier of the target website in the malicious website identifier database. When the service server 100 finds a malicious website identifier of the same category as the target webpage identifier of the target website in the malicious website identifier database, the service server 100 determines that the target website is a malicious website and returns a corresponding risk prompt to the terminal device. The terminal device can stop accessing the target website according to the returned risk prompt, thereby avoiding the terminal device being attacked by viruses carried by the malicious website. In addition, when the service server 100 finds a safe website identifier of the same category as the target webpage identifier of the target website in the malicious website identifier database, or does not find a malicious website identifier of the same category as the target webpage identifier of the target website, the service server 100 determines that the target website is a safe website and returns the downloaded target webpage resource to the terminal device. The terminal device can render the website according to the returned target webpage resource to display the webpage content corresponding to the target website.

[0086] It can be understood that in order to effectively intercept malicious websites, a webpage identifier processing method with good distinguishability and robustness can be used to obtain and process the webpage identifier corresponding to the target website accessed by the operation object, so as to ensure the accuracy of obtaining the webpage identifier of the website and determining whether it is a malicious website. The distinguishability can be understood as that the webpage identifier processing method can generate different webpage identifiers for different webpages, and the robustness (i.e. anti-interference) can be understood as that when the content of the webpage changes slightly, the webpage identifier generated by the webpage identifier processing method will not be affected. For example, in some application scenarios, the news content on a news website will be updated frequently, but the basic structure of the news website remains unchanged. At this time, the webpage identifier of the news website generated by the webpage identifier processing method with good robustness should not change.

[0087] In some possible implementation manners, the webpage identifier of the target website can be constructed by extracting the DOM structure of an HTML file of the target website. The HTML file refers to a file that can be read by various application clients (e.g., a web browser) to generate a webpage to deliver various webpage resources (i.e., the webpage multimedia resources described above). Generally, the HTML file is stored on a hard disk distributed in each server, and the application client can remotely obtain the webpage multimedia resources delivered by the HTML file through a transmission protocol. The application client can interpret the HTML file to display the webpage content. For example, when the operation object uses a web browser to browse a webpage on the Internet, the web browser can automatically convert the HTML file into a webpage. Specifically, the HTML file can be converted into the initial webpage content of the webpage as a static webpage resource in the webpage resources. The static webpage resource can be understood as a standard HTML file, and the file extension of the static webpage resource is.HTML or.htm. The static webpage resource can include HTML tags, text, plug-ins, and application client scripts, and does not include any server-side scripts. Each line of HTML code in the static webpage resource is written by a webpage designer and uploaded to the hard disk of each server, and is not changed after being uploaded to the server. As can be seen, when the terminal device sends a webpage resource request to the service server 100 to request to obtain the webpage resource corresponding to the target website, if the webpage resource requested by the target website is a static webpage resource, the service server 100 can return the corresponding HTML file based on the webpage resource request, and the HTML file includes the static webpage resource requested by the target website. The terminal device can interpret the HTML file to obtain the corresponding static webpage. It can be understood that when the webpage content of the static webpage is changed, a new HTML file corresponding to the new static webpage resource can be rewritten and uploaded to each server, so that different terminal devices can call the updated HTML file, thereby updating the webpage content of the target website.

[0088] It can be understood that in addition to the static web page described above, the website can also be displayed by requesting a dynamic web page resource from the server. Specifically, the dynamic web page resource described above can be parsed and rendered by the terminal device into a dynamic page of the website. From the above content, it can be seen that after the HTML code corresponding to the web page is written and the HTML file is uploaded to the server, the content and display effect of the web page will not change, and only when the designer modifies the HTML code of the web page, the page of the web page can be changed. The dynamic web page is different. The page code of the dynamic web page does not change, but the displayed content can change with time, environment or the result of database operation. In addition, the difference between dynamic web pages and static web pages can be generated by whether dynamic website technology is used, and is irrelevant to the content of the web page, that is, it has nothing to do with the visual dynamic effect of various animations, scrolling subtitles and the like on the web page. Dynamic web pages and static web pages can both display text and image information. Specifically, dynamic web pages can interact with the server and the operation object in the background to complete operation object queries, submissions and other operations. Dynamic web pages can automatically update the web page according to the server-side script running on the server, and dynamically respond according to the requirements and choices of the operation object, and have strong interactivity.

[0089] In some possible embodiments, the web browser can parse the HTML file into a tree structure, and define standards for accessing and operating the HTML file on this basis. According to the World Wide Web Consortium (W3C) standard, the tree structure can be referred to as a DOM structure. Specifically, please refer to Figure 2 , Figure 2 is a schematic diagram of an application scenario provided by the web page identification processing method of the embodiments of the present application. As shown in Figure 2 , first, the target website www.xxx.com can be received, and the HTML file corresponding to the target website www.xxx.com can be extracted by the website content extraction program to obtain the HTML file index.HTML of the target website. Then the HTML file index.HTML of the target website can be parsed to obtain the DOM structure of the target website as shown in Figure 2 . Further, the corresponding DOM sequence can be obtained based on the DOM structure of the target website. By processing the DOM sequence, the web page identification of the target website described above can be generated, for example, by processing the DOM sequence "HTML head mata title body divdiv", the web page identification "1110010001001101101100111011110111001011001011011100100001110010" can be obtained.

[0090] It can be understood that the server can determine whether the target website is a malicious website or determine the classification category of the target website by identifying and searching the webpage identifier, so as to realize the webpage recommendation function. However, it can be known from the above that the target website can obtain static page resources, that is, corresponding HTML files to generate static pages, and in addition, the target website can also request the server to obtain corresponding dynamic webpage resources to generate dynamic pages. As can be seen, different webpage resources (dynamic page resources, static webpage resources, and webpage resources combined with dynamic webpage resources and static webpage resources) can be used to display the same webpage page, that is, the target websites of multiple same webpage pages can have different HTML files, and correspondingly, the DOM structures corresponding to each target website are also different. Further, when the webpage identifier of the target website is obtained according to the DOM structure of the HTML file, if the DOM structure of the target website is relatively complex, the webpage identifier obtained based on the DOM structure of the target website is also more accurate. If the DOM structure of the target website is relatively simple, the accuracy of the webpage identifier obtained based on the DOM structure of the target website is low, and when the webpage identifier of the target website is compared with the malicious website identifier in the malicious website identifier database, it is easy to produce a false judgment. Similarly, in the application scenario of website recommendation, the business server 100 can recommend webpages that meet the historical interest preferences of the operation object to the operation object based on the similarity between the webpage identifiers of different websites. However, when the website adopts the dynamic embedded webpage resource mode to obtain the webpage page of the target website, the HTML file corresponding to the target website is only used to request static webpage resources. Therefore, the DOM structure corresponding to the HTML file of the target website cannot be used to represent the dynamic webpage resources of the target website. As can be seen, the accuracy of obtaining the webpage identifier corresponding to the target website through the DOM structure of the HTML file of the target website is low, and it is easy to cause a false judgment when comparing the similarity of webpage identifiers of different websites, thereby affecting the identification of malicious websites and the effect of webpage recommendation.

[0091] It can be understood that in the specific embodiments of the present application, data related to object information is involved, and when the embodiments of the present application are applied to specific products or technologies, the permission or consent of the object is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0092] The webpage identification processing method provided in the embodiments of the present application is applicable to webpage identification processing in an application (such as the target client). It can be understood that the terminal device to which the webpage identification processing method is applicable includes, but is not limited to, a smart phone, a computer, a tablet computer, a personal digital assistant (PDA), a mobile Internet device (MID), a wearable device, and the like. Optionally, the terminal device can also be a server corresponding to the smart phone, the computer, the tablet computer, the PDA, the MID, and the wearable device, and the like, and the specific terminal device can be determined according to an actual application scenario, which is not limited herein. Correspondingly, the webpage identification processing apparatus provided in the embodiments of the present application includes, but is not limited to, a smart phone, a computer, a tablet computer, a PDA, an MID, a wearable device, and the like. For the convenience of description, the webpage identification processing apparatus and / or the terminal device provided in the embodiments of the present application will be described by taking a smart phone (or simply a mobile phone) as an example.

[0093] It can be understood that the webpage identification processing method provided in the embodiments of the present application can be executed by the service server 100 as shown in FIG. 1, can be executed by any one of the terminal devices 200a, 200b, …, 200n as shown in FIG. 2, or can be executed by the terminal device and the service server together, and the specific execution manner can be determined according to an actual application scenario, which is not limited herein. Figure 1 Figure 1 It can be understood that the webpage identification processing method provided in the embodiments of the present application can be executed by the service server 100 as shown in FIG. 1, can be executed by any one of the terminal devices 200a, 200b, …, 200n as shown in FIG. 2, or can be executed by the terminal device and the service server together, and the specific execution manner can be determined according to an actual application scenario, which is not limited herein. Figure 1

[0094] ​​The webpage identification processing method provided by the embodiments of the present application can obtain the static resource request and the dynamic resource request of the to-be-identified webpage, wherein the static resource request of the to-be-identified webpage is used to request the static webpage resource from the server, and the dynamic resource request of the to-be-identified webpage is used to request the real-time updated dynamic webpage resource. The static webpage resource can generate the initial webpage content of the to-be-identified webpage under the parsing and rendering of the terminal device, and the dynamic webpage resource can generate the interactive webpage content of the to-be-identified webpage under the parsing and rendering of the terminal device. It can be understood that when the webpage contents of different to-be-identified webpages are the same, the initial webpage content and the interactive webpage content of different to-be-identified webpages are the same, and because the static resource request or the dynamic resource request is the same, the webpage content (the initial webpage content and the interactive webpage content) obtained by the request is also the same. Therefore, when the webpage contents of different to-be-identified webpages are the same, the static resource request and the dynamic resource request of different to-be-identified webpages are also the same. Therefore, the embodiments of the present application can obtain the webpage identification of the to-be-identified webpage through the static resource request and the dynamic resource request of the to-be-identified webpage, and the webpage identification of different to-be-identified webpages can be used to represent the similarity of different to-be-identified webpages. Further, the to-be-identified resource request set of the to-be-identified webpage can be generated based on the dynamic resource request and the static resource request of the to-be-identified webpage, the resource request text of the to-be-identified webpage is mapped into the identification feature sequence of the to-be-identified webpage, and the webpage identification of the to-be-identified webpage is generated based on the identification feature sequence. It can be understood that when the webpage contents of different to-be-identified webpages are the same, the to-be-identified resource request of different to-be-identified webpages is also the same. Therefore, the identification feature sequence obtained by mapping the resource request text of the to-be-identified webpage can be used to represent the webpage identification of the to-be-identified webpage. The webpage identification processing method of the embodiments of the present application can obtain the webpage identification of the to-be-identified webpage based on the dynamic resource request and the static resource request of the to-be-identified webpage, which can improve the accuracy of obtaining the webpage identification of the to-be-identified webpage, improve the accuracy of judging the similarity of different to-be-identified webpages, and has high applicability.

[0095] The webpage identifier processing method provided in this application is applicable to various types of webpage identifier processing applications. These applications include, but are not limited to, multimedia applications, browser applications, game applications, shopping applications, utility applications, social applications, travel applications, and educational applications—all application clients with multimedia resource processing capabilities. No limitation is imposed here. Furthermore, the same type of application may include multiple applications, which is also not limited here. For example, multimedia applications may include video players, music players, photography applications, photo editing applications, and audio input applications. Shopping applications include, but are not limited to, movie ticketing applications, restaurant reservation applications, and applications for purchasing daily necessities. Utility applications include, but are not limited to, file editing, email, alarm clock, calendar, photo album, settings, and compass applications.

[0096] For further details, please refer to Figure 3 , Figure 3 This is a schematic flowchart illustrating a webpage identifier processing method provided in an embodiment of this application. For ease of understanding, this embodiment uses a terminal device as an example for explanation. Figure 1 Taking the terminal device 200b as an example, the service server can be the one described above. Figure 1 The corresponding embodiment refers to the business server 100. This application's embodiments can be applied to various scenarios, including but not limited to cloud technology and artificial intelligence. Figure 3 In the webpage identifier processing method shown, each step of the webpage identifier processing can be derived from the above. Figure 1 The terminal device 200b in the middle executes, such as Figure 3 As shown, the webpage identification processing method may include at least the following steps S101-S104.

[0097] Step S101: Obtain the dynamic resource request of the webpage to be identified.

[0098] In some possible implementations, the dynamic resource request can be understood as a request for a server to obtain a real-time updated dynamic webpage resource, which can be rendered as the interactive webpage content of the to-be-identified webpage. The interactive webpage content of the to-be-identified webpage can be understood as different webpage content that can be loaded and displayed when different operation objects access the to-be-identified webpage at different times and locations through an application client of a terminal device. The operation objects can update the interactive webpage content of the to-be-identified webpage through interaction with the application client. As can be seen from the foregoing content, the dynamic webpage resource can be parsed and rendered by the terminal device into a dynamic page of the website, and the dynamic page is composed of the interactive webpage content. Specifically, the terminal device can detect the address bar of the application client, and when a target address is input, the terminal device receives the target address and initiates a DNS resolution request to convert the domain name of the target address into an IP address. Further, the terminal device constructs an HTTP request or an HTTPS message and sends the HTTP request or the HTTPS request to the business server 100. The HTTP request or the HTTPS request can be understood as a dynamic resource request of the to-be-identified webpage. The business server 100 receives the dynamic resource request and calls a real-time updated dynamic webpage resource based on the dynamic resource request from the stored database and returns the real-time updated dynamic webpage resource to the terminal device. The terminal device parses and renders the real-time updated dynamic webpage resource to obtain the interactive webpage content of the to-be-identified webpage. In addition, when the dynamic webpage resource is parsed, if the terminal device encounters a new dynamic resource request, the terminal device sends the dynamic resource request to the server again to find the corresponding dynamic webpage resource, and parses the dynamic webpage resource to obtain the dynamic page of the to-be-identified webpage. It can be understood that, since the terminal device can request the server to call the real-time updated dynamic webpage resource in the database based on the dynamic resource request, to obtain the interactive webpage content of the to-be-identified webpage, the operation objects can realize rich interactive functions such as operation object registration, operation object login, online survey, object management, order management, and the like through the interactive webpage content of the to-be-identified webpage.

[0099] For ease of understanding, the embodiments of the present application take the web page to be identified as the web page of a news release website as an example for simple description in the following content. It can be understood that the news release website can be understood as a website for providing real-time updated news information to operation objects, and thus it can be seen that the web page of the news release website includes real-time updated news text content. When different operation objects access the news release website, the news release website can recommend news of interest to the operation objects based on the historical behavior of the operation objects, and when the operation objects access the news release website at different times, the news release website can also update the news text content in the web page in real time to provide hot news to the operation objects. It can be understood that in order to update the news text content in the web page in real time, the news text content on the news release website page can be regarded as interactive web page content of the news release website page, and by sending a dynamic resource request to the server, the server calls real-time updated dynamic web page resources in the database based on the dynamic resource request and returns them to the terminal device, and the terminal device parses and renders the dynamic web page resources to obtain the real-time updated news text content of the news release website, i.e. the above-mentioned interactive web page content. In addition, the operation object can also log in a personal account on the web page of the news release website, and for this purpose, the web page of the news release website can also include login information content. Specifically, the operation object can input login information in the login box of the news release website page, so that the terminal device sends a dynamic resource request to the server based on the login information content input on the news release website page. Further, the server can search in the database based on the dynamic resource request to determine whether the login information input by the operation object is stored in the database. If the login information input by the operation object exists in the database of the server, the server can return an instruction of "login success" to the terminal device, so that the terminal device updates the login information content of the news release website page to the login information of the operation object, i.e. the operation object has logged in the current news release website.

[0100] Step S102, obtaining a static resource request of the web page to be identified.

[0101] In some possible implementations, the static resource request can be understood as a request for a static web page resource to the server, and the static web page resource can be rendered as the initial web page content of the to-be-identified web page. The initial web page content of the to-be-identified web page can be understood as the same web page content that is loaded and displayed by different operation objects through the application client of the terminal device at different times and places when accessing the to-be-identified web page. As can be seen from the above, the static web page resource can be parsed and rendered by the terminal device into a static page of the website, and the static page is composed of the initial web page content. Specifically, the terminal device can detect the address bar of the application client, and when the target address is input, the terminal device receives the target address and initiates a DNS resolution request to convert the domain name of the target address into an IP address. Further, the terminal device constructs a HTTP request or a HTTPS request message and sends the message to the business server 100. The HTTP request or the HTTPS request can be understood as a static resource request of the to-be-identified web page. The business server 100 receives the static resource request sent by the terminal device and parses the static resource request. Further, the business server 100 can return a corresponding HTML file (i.e., a static web page resource) based on the static resource request to the terminal device. The terminal device can parse and render the HTML file to generate the initial web page content (i.e., a static page) of the to-be-identified web page. Since the static page can be directly parsed by the terminal device based on the HTML file without accessing the database of the dynamic server, the pressure on database access is reduced, and therefore the loading speed of the initial web page content in the static page is relatively fast. In addition, since the terminal device can cache the static web page resource of the target address, the terminal device can still query the cached static web page resource to display the initial web page content of the target address in an offline state of the terminal device. It can be understood that when the operation object accesses the target address, in order to improve the access experience of the operation object, the web page content that is first displayed to the operation object and does not need to be frequently updated can be loaded by the static web page resource, that is, the web page content is taken as the initial web page content of the website.

[0102] For ease of understanding, the embodiments of the present application take the webpage to be identified as the webpage of a news release website as an example for simple description in the following content. It can be understood that, from the above content, in order to update the news text content in the webpage in real time, the news text content on the news release website page can be taken as the interactive webpage content of the news release website page, and by sending a dynamic resource request to the server, the server calls the real-time updated dynamic webpage resource in the database based on the dynamic resource request and returns it to the terminal device, and the terminal device parses and renders the dynamic webpage resource to obtain the real-time updated news text content of the news release website. However, since the acquisition of the news text content on the webpage requires a large number of accesses to the database of the server, the efficiency of webpage loading is low, and the access pressure of the server database is increased. In order to improve the webpage loading speed of the news release website and improve the experience of the operation object accessing the news release website, the webpage content on the news release website except the news text content can be taken as the initial webpage content, and the initial webpage content is generated in the form of a static resource request to reduce the access to the database. It can be understood that, the webpage content on the news release website except the news text content can be the trademark, navigation bar, webpage font, webpage background and the like on the to-be-identified page of the news release website. Generally, the trademark, navigation bar, webpage font, webpage background and the like do not change with the change of the access object, time and place, that is, the terminal device loads the above-mentioned webpage content which is the same when different operation objects access the news release website at different times and places. Therefore, the webpage content which does not need to be frequently updated can be taken as the initial webpage content of the to-be-identified page. Specifically, the terminal device can receive the target address and initiate a DNS resolution request when detecting that the target address of the news release website is input, so as to convert the domain name of the target address into an IP address. Further, the terminal device generates a static resource request of the target address and sends it to the service server 100. The service server 100 receives the static resource request sent by the terminal device and parses it. Further, the service server 100 can return the corresponding HTML file (i.e. static webpage resource) to the terminal device based on the static resource request. The terminal device can parse and render the HTML file to generate the initial webpage content such as trademark, navigation bar, webpage font, webpage background and the like of the to-be-identified page. In addition, the terminal device can store the obtained static webpage resource in the local cache, so as to call the corresponding static webpage resource in the local cache for parsing and rendering when the operation object accesses the target address of the news release website again, so as to improve the webpage loading speed.

[0103] In some feasible implementations, as described above, when a website obtains a webpage to be identified using dynamically embedded webpage resources, the corresponding HTML file of that webpage is only used to request static webpage resources. Therefore, the DOM structure corresponding to the HTML file of the webpage to be identified cannot be used to represent the dynamic webpage resources of the webpage to be identified. Consequently, obtaining the webpage identifier of the webpage to be identified through the DOM structure of the HTML file of the webpage to be identified has low accuracy and is prone to misjudgment when comparing the similarity of webpage identifiers of different webpages to be identified, thus affecting the identification of malicious websites and the effectiveness of webpage recommendation. To improve the accuracy of obtaining the webpage identifier of the webpage to be identified, the webpage identifier processing method in this embodiment obtains the dynamic resource requests and static resource requests of the webpage to be identified, and can obtain the webpage identifier of the webpage to be identified based on these requests. It can be understood that the aforementioned dynamic resource requests of the webpage to be identified can be understood as requests for dynamic webpage resources from the server, and the static resource requests can be understood as requests for static webpage resources from the server. When a website obtains a webpage to be identified using dynamically embedded webpage resources, the dynamic resource request corresponding to that webpage can be used to request dynamic webpage resources. It can be understood that the same dynamic resource request can obtain the same dynamic webpage resources; that is, different websites can obtain the same dynamic webpage resources by sending the same dynamic resource request to the server, thus loading and displaying the same interactive webpage content on different websites. Therefore, the dynamic resource request of the webpage to be identified can be used to represent the interactive webpage content of the webpage to be identified. Similarly, the same static resource request can obtain the same static webpage resources; that is, different websites can obtain the same static webpage resources by sending the same static resource request to the server, thus loading and displaying the same initial webpage content on different websites. Therefore, the static resource request of the webpage to be identified can be used to represent the initial webpage content of the webpage to be identified.

[0104] In some feasible implementations, it is understood that if the content of different web pages to be identified is the same—that is, the initial content and interactive content of different web pages to be identified are the same—then, as mentioned above, the static resource requests used to represent the initial content of the web page to be identified, and the dynamic resource requests used to represent the interactive content of the web page to be identified, are also the same for each web page to be identified. Similarly, when the web page content displayed by different websites has a high degree of similarity, the dynamic resource requests and static resource requests loaded by different websites during the process of displaying the page to be identified also have a high degree of similarity. Therefore, it can be seen that the embodiments of this application, by obtaining the web page identifier of the web page to be identified through obtaining the static resource requests and dynamic resource requests, can improve the accuracy of web page identifier processing.

[0105] In some possible implementation manners, it can be understood that the terminal device sends multiple web resource requests to the server when obtaining the web resource of the page to be identified, and in the process of the requests, there can be situations of connection failure, web resource movement, error request, and the like, in which case the terminal device does not obtain the web resource returned by the server. In order to ensure the accuracy of the web identifier processing of the page to be identified, the embodiment of the present application can filter out the web resource requests in which the web resource is failed to be requested, and only keep the web resource requests in which the web resource is successfully requested. Specifically, when the server receives the web resource request (static resource request or dynamic resource request) sent by the terminal device, the server obtains the stored web resource or the generated web resource after dynamic running based on the web resource request, and performs a response action to generate a response message, which can include a status code, a response header, a body content, and the like. When the terminal device completes the loading and display of the page to be identified, an entry can be added in the log file to represent the status code of the web resource request that has been executed, where the status code of the web resource request can be understood as a 3-digit code used to represent the response status of the server to the web resource request. For example, when the status code corresponding to the web resource request is 200, it represents that the server has successfully processed the web resource request and provided the terminal device with the requested web resource; when the status code corresponding to the web resource request is 400, it represents that the server does not understand the syntax of the web resource request and cannot provide the terminal device with the requested web resource; and when the status code corresponding to the web resource request is 500, it represents that the server has an internal error and cannot complete the web resource request. Therefore, the web identifier processing method in the embodiment of the present application can filter the successful web resource requests based on the status of the web resource requests.

[0106] In some possible implementation manners, when the page to be identified is loaded, the web resource request history record in the log file can be queried, and the multiple initial dynamic resource requests of the page to be identified in the web resource request history record can be obtained, and the status code of each initial dynamic resource request can be obtained. It can be understood that the multiple initial dynamic resource requests can include the initial dynamic resource request successfully requesting the dynamic resource and the initial dynamic resource request unsuccessfully requesting the dynamic resource. As can be known from the above, the initial dynamic resource request successfully requesting the dynamic resource in the multiple initial dynamic resource requests can be determined based on the status code of each initial dynamic resource request. Specifically, the initial dynamic resource request with the status code equal to the preset status code in the multiple initial dynamic resource requests is the dynamic resource request of the page to be identified. For example, the initial dynamic resource request with the status code equal to 200 in the multiple initial dynamic resource requests is the dynamic resource request of the page to be identified.

[0107] In some possible embodiments, when the webpage to be identified is loaded, the webpage resource request history record in the log file can be queried, and the plurality of initial static resource requests of the webpage to be identified in the webpage resource request history record can be obtained, and the status code of each initial static resource request can be obtained. It can be understood that the plurality of initial static resource requests can include an initial static resource request successfully requesting a static resource and an initial static resource request unsuccessfully requesting a static resource. As can be known from the above, the initial static resource request successfully requesting a static resource in the plurality of initial static resource requests can be determined based on the status code of each initial static resource request. Specifically, the initial static resource request with the status code equal to the preset status code in the plurality of initial static resource requests is the static resource request of the webpage to be identified. For example, the initial static resource request with the status code equal to 200 in the plurality of initial static resource requests is the static resource request of the webpage to be identified.

[0108] In step S103, the initial resource request set of the webpage to be identified is generated based on the dynamic resource request and the static resource request of the webpage to be identified, and the resource request text of the webpage to be identified is obtained based on the request field included in each resource request in the initial resource request set.

[0109] In some possible embodiments, the dynamic resource request and the static resource request of the webpage to be identified can be obtained after the plurality of webpage resource requests are screened, and further, the dynamic resource request and the static resource request of the webpage to be identified can be combined to generate the initial resource request set of the webpage to be identified, where the combination of the dynamic resource request and the static resource request can be sorting and combination according to the time when each webpage resource request occurs. For ease of understanding, the target website of the webpage to be identified is www.xxx.com, and the dynamic resource request and the static resource request of the webpage to be identified are combined to generate the initial resource request set of the webpage to be identified. Specifically, refer to Table 1, which is a table of the initial resource request set of the webpage to be identified, and is specifically as follows:

[0110]

[0111]

[0112] Table 1

[0113] In some possible implementation manners, when the terminal device receives the target website, the terminal device can send the target website www.xxx.com to a DNS server for querying to obtain a corresponding IP address, and then, the terminal device can send an ssl.captcha.mm.com / TCaptcha.js request to the server to perform a verification operation on the operation object, and further, the terminal device can send requests as shown in numbers 3 to 17 to request static resources and dynamic resources of the to-be-identified webpage, for example, a JavaScript (JS) file, a Cascading Style Sheets (CSS) file, an object identity, and the like. In the process in which the terminal device accesses the target website to load and display the to-be-identified webpage, the terminal device can send a dynamic resource request to the server to obtain interactive webpage content of the to-be-identified webpage, or send a static resource request to the server to obtain initial webpage content of the to-be-identified webpage, as shown in the to-be-identified resource request set.

[0114] In some possible implementation manners, the webpage identification processing method provided by the embodiments of the present application can obtain the resource request text of the webpage to be identified based on the request field included in each resource request in the set of resource requests to be identified. The request field included in each resource request refers to the effective field included in each resource request and having distinguishability. For example, the URL of the webpage to be identified can start with an http field or an https field, where the http field or the https field can be understood as a transmission protocol, and the terminal device can use the transmission protocol of the http field or the https field to communicate with the server. It can be understood that the dynamic resource request and the static resource request in the process of loading all the pages to be identified will include the http field or the https field, that is, the http field or the https field does not have distinguishability for different pages to be identified, and such a field can be referred to as an interference field. Therefore, in order to avoid the influence of the interference field on the accuracy of obtaining the webpage identification, the webpage identification processing method provided by the embodiments of the present application can remove the interference field in each resource request in the set of resource requests to be identified, determine the effective field having distinguishability as the request field, and obtain the resource request text of the webpage to be identified based on the request field included in each resource request in the set of resource requests to be identified. Specifically, taking the set of resource requests to be identified in Table 1 as an example, by removing the interference field in each resource request in the set of resource requests to be identified, that is, removing www.xxx.com in each resource request except the target URL www.xxx.com sent first, the resource request text of the webpage to be identified composed of the request field included in each resource request can be obtained. Specifically, refer to Table 2 for the resource request text of the webpage to be identified.

[0115]

[0116]

[0117] Table 2

[0118] In some possible implementation manners, after screening the request field of each resource request in the set of resource requests to be identified as shown in Table 1, the resource request text as shown in Table 2 can be obtained. It can be understood that the webpage identification processing method provided by the embodiments of the present application can also screen other fields in each resource request as the request field, which can be determined according to actual application scenarios, and the present application does not make any limitation here.

[0119] In step S104, the resource request text of the webpage to be identified is mapped into an identification feature sequence of the webpage to be identified, and the webpage identification of the webpage to be identified is generated based on the identification feature sequence.

[0120] In some possible implementation manners, after obtaining the resource request text of the to-be-identified webpage based on the dynamic resource request and the static resource request, the webpage identification processing method provided by the embodiment of the present application can further map the resource request text of the to-be-identified webpage into an identification feature sequence of the to-be-identified webpage by using a hash algorithm, and generate the webpage identification of the to-be-identified webpage based on the identification feature sequence. It can be understood that the hash algorithm is a secure hash algorithm, and the resource request text of any length can be mapped into a fixed-length binary value by using the hash algorithm. The fixed-length binary value is a hash value, that is, the identification feature sequence of the to-be-identified webpage. It can be understood that the hash value is a unique and extremely compact numerical representation form of data. If the resource request text is changed, the identification feature sequence of the changed resource request text after the hash operation will also change. Therefore, the resource request text can be mapped into the identification feature sequence of the to-be-identified webpage, and the identification feature sequence is determined as the webpage identification of the to-be-identified webpage. As can be known from the above content, the resource request text of the to-be-identified webpage with different webpage contents is also different. Therefore, the resource request text can be mapped into the identification feature sequence (that is, the hash value) of the to-be-identified webpage by using the hash algorithm. The identification feature sequence obtained by mapping the resource request text into the to-be-identified webpage by using the hash algorithm is also different for the to-be-identified webpages with different webpage contents. That is, the identification feature sequence obtained by mapping the resource request text into the to-be-identified webpage by using the hash algorithm can be used to represent the difference in webpage contents of different to-be-identified webpages. Therefore, the identification feature sequence of the to-be-identified webpage can be determined as the webpage identification of the to-be-identified webpage. By comparing the similarity of the webpage identifications of different webpages, the similarity of the webpage contents of different webpages can be determined, so that the effective identification and interception of malicious web addresses can be realized, or the accuracy of webpage recommendation can be improved.

[0121] In some possible implementation manners, the webpage identification processing method provided by the embodiment of the present application can map the resource request text of the to-be-identified webpage into the identification feature sequence of the to-be-identified webpage according to the following steps. Specifically, refer to Figure 4 , Figure 4 is another flow diagram of the webpage identification processing method provided by the embodiment of the present application. As shown in Figure 4 , after obtaining the resource request text of the to-be-identified webpage through steps S201-S203, the webpage identification processing method provided by the embodiment of the present application maps the resource request text of the to-be-identified webpage into the identification feature sequence of the to-be-identified webpage. The specific implementation of steps S201-S203 for obtaining the resource request text of the to-be-identified webpage can refer to the implementation manners provided by steps S101-S103, which will not be described herein. Further, the implementation of mapping the resource request text of the to-be-identified webpage into the identification feature sequence of the to-be-identified webpage can include the following steps:

[0122] In step S204, feature words are extracted from the resource request text of the to-be-identified webpage to obtain a plurality of feature words.

[0123] In some possible implementations, as can be known from the above, if the webpage contents of different to-be-identified webpages are the same, that is, the initial webpage contents and the interactive webpage contents of different to-be-identified webpages are the same, the static resource requests used to represent the initial webpage contents of the to-be-identified webpages and the dynamic resource requests used to represent the interactive webpage contents of the to-be-identified webpages in different to-be-identified webpages are also the same. Specifically, the similarity of the webpage contents of different to-be-identified webpages can be determined by calculating the text similarity of the dynamic resource requests and the static resource requests of different to-be-identified webpages. Specifically, as shown in Table 1 or Table 2, the dynamic resource requests and the static resource requests of the to-be-identified webpages can be composed of a plurality of feature words, for example, the static resource request / static / js / manifest.8bfaa74c.js?1665392849615 shown in Table 2, wherein the static resource request includes the feature words static, js, manifest, 8bfaa74c, js, and 1665392849615. When the feature words of the resource request initiated by any terminal client also include the above plurality of feature words static, js, manifest, 8bfaa74c, js, and 1665392849615, the webpage resource requested by the resource request initiated by the terminal client is the same as the webpage resource requested by the resource request with the number 6 in the resource request text shown in Table 2. It can be understood that if the webpage resource requests of different to-be-identified webpages are the same, the feature words included in the webpage resource requests of the corresponding different to-be-identified webpages are the same. Therefore, the webpage identifier of the to-be-identified webpage can be determined by extracting a plurality of feature words in the resource request text of the to-be-identified webpage, so as to determine the similarity of different to-be-identified webpages according to the webpage identifier. Similarly, the feature words of a plurality of resource requests in the resource request text of Table 2 can be obtained to obtain a plurality of feature words of the to-be-identified webpage.

[0124] In some possible implementation manners, the feature word extraction can be performed on the resource request text of the to-be-identified webpage according to the following content. For the convenience of understanding, the webpage identification processing method provided by the application embodiment is taken as an example to perform feature word extraction on the resource request text of the to-be-identified webpage shown in Table 2. Specifically, for example, the resource request numbered 3 in the resource request text shown in Table 2 is andun1.oss-accelerate.aliiiiis.com / vcaptcha.min.js. The resource request includes the words andun1, oss-accelerate, aliiiiis, com, vcaptcha, min, js, and the separator strings "." and " / ", wherein the separator string can be understood as being used to separate the words in the resource request of the webpage, to represent the root directory or the current directory of the resource request of the webpage, and the like. It can be understood that, in order to extract the words in the resource request text, the webpage identification processing method provided by the application embodiment can identify the separator strings in the resource request text to obtain a plurality of words. For example, the resource request numbered 3 in the resource request text shown in Table 2 is andun1.oss-accelerate.aliiiiis.com / vcaptcha.min.js, by identifying the separator strings in the resource request, the words andun1, oss-accelerate, aliiiiis, min, js can be extracted when the separator string "." is identified, and the words com and vcaptcha can be extracted when the separator string " / " is identified. It can be understood that the resource request includes a plurality of words andun1, oss-accelerate, aliiiiis, com, vcaptcha, min, js. Similarly, the separator strings in the plurality of resource requests in the resource request text of the to-be-identified webpage shown in Table 2 can be identified to obtain a plurality of words of the to-be-identified webpage.

[0125] In some possible implementation manners, after the separator string characters in the resource request text are identified to obtain a plurality of words, the webpage identification processing method provided by the embodiments of the present application can further classify the plurality of words to obtain a plurality of classified words, wherein each classified word includes a plurality of words of the same category. Further, the frequency of occurrence of each classified word is obtained according to the number of words included in each classified word, K classified words with higher frequency of occurrence are determined from the plurality of classified words, and the K classified words are determined as a plurality of feature words of the resource request text. Specifically, as shown in the resource request text in Table 2, different numbered resource requests can include the same words, for example, the words static and js are included in the resource requests numbered 6 to 8 and numbered 10, and the words media and png are included in the resource requests numbered 11 to 13, that is, the words can appear multiple times in the resource request text, and the number of times of appearance of different words in the resource request text can be different. It can be understood that the same words can appear in the resource request texts of different to-be-identified webpages, and when the words with the highest number of times of appearance in the resource request texts of different to-be-identified webpages are the same, it can be understood that the similarity of the resource request texts of different to-be-identified webpages is high, that is, the similarity of the webpage contents of different to-be-identified webpages is high. Therefore, the webpage identification processing method provided by the embodiments of the present application can count the number of times of appearance of a plurality of words in the resource request text of a to-be-identified webpage, and determine a plurality of words with a higher number of times of appearance as feature words of the to-be-identified webpage.

[0126] In some possible implementation manners, the webpage identification processing method provided by the embodiments of the present application can classify a plurality of words in the resource request text of the webpage to be identified to obtain a plurality of classified words, and obtain the frequency (i.e., the number of occurrences) of each classified word. For example, as shown in Table 2, the words static in the resource request text of the resource request numbered 5 to 8 are classified as a classified word static, and the frequency of the classified word static in the resource request numbered 5 to 8 is 4, i.e., the classified word static occurs 4 times in the resource request numbered 5 to 8. In addition, the words js in the resource request numbered 6 to 8 are classified as a classified word js, and the frequency of the classified word js in the resource request numbered 6 to 8 is 6, i.e., the classified word static occurs 6 times in the resource request numbered 6 to 8. Similarly, the plurality of words in the resource request text shown in Table 2 can be classified, and the frequency of each classified word can be obtained. Further, since the resource request texts of different webpages to be identified that are the same or have high similarity may be different. Therefore, if the different webpages to be identified are determined to be the same or have high similarity only when each classified word of the different webpages to be identified is the same, the robustness of the webpage identification processing is easily reduced. Therefore, the webpage identification processing method provided by the embodiments of the present application can determine K classified words with high frequencies in all classified words of the webpage to be identified as K feature words used to obtain the webpage identification of the webpage to be identified, so as to improve the robustness of the webpage identification processing. Wherein, K is a positive integer, and the specific value of K can be determined according to the actual application scenario, which is not limited herein. As can be seen, when the plurality of feature words with high frequencies in the resource request texts of different webpages to be identified are the same, it can be understood that the similarity of the resource request texts of the different webpages to be identified is high, i.e., the similarity of the webpage contents of the different webpages to be identified is high. For ease of understanding, the webpage identification processing method provided by the embodiments of the present application can take the resource request text shown in Table 2 as an example, classify the plurality of words in Table 2 to obtain a plurality of classified words and the frequency of each classified word, and determine the plurality of feature words of the webpage to be identified and the frequency of each feature word according to the frequency of each classified word. For details, please refer to Table 3, which is a table of the frequency of the feature words of the webpage to be identified:

[0127]

[0128]

[0129] Table 3

[0130] In step S205, the weight coefficients of each feature word in the plurality of feature words are obtained, and the plurality of feature words are mapped into a plurality of weighted real sequences based on the weight coefficients of each feature word.

[0131] In some possible implementation manners, it can be understood that, according to the above description, the plurality of feature words obtained based on the plurality of words with high occurrence frequencies in the resource request text of the to-be-identified webpage can represent the webpage identifier of the to-be-identified webpage to a certain extent. However, because the information amount contained in different feature words is different, that is, the importance of each feature word with respect to the to-be-identified webpage is different, in order to improve the accuracy of processing the webpage identifier of the to-be-identified webpage, after obtaining the plurality of feature words of the to-be-identified webpage, the weight coefficients of each feature word in the plurality of feature words can be further obtained, and the corresponding feature word can be weighted based on the weight coefficients of each feature word. The weight coefficient of each feature word can be understood as a value used to represent the importance of each feature word with respect to the webpage identifier of the to-be-identified webpage. The higher the weight coefficient of any feature word is, the more important the feature word is compared with other feature words for obtaining the webpage identifier of the to-be-identified webpage. For ease of understanding, the embodiment of the present application takes the plurality of feature words in Table 3 as an example for simple description. Specifically, it is assumed that, in the four feature words js, static, com, and 1665392849615 of the to-be-identified webpage, the feature word com appears more in the webpage resource requests of different websites for loading the to-be-identified webpage, that is, the feature word com has poorer distinguishability compared with the distinguishability of other feature words js, static, and com. Therefore, the weight coefficient of the feature word com in the four feature words js, static, com, and 1665392849615 of the to-be-identified webpage is the lowest. Similarly, the feature word 1665392849615 appears less in the webpage resource requests of different websites for loading the to-be-identified webpage, and therefore, the feature word 1665392849615 has better distinguishability compared with the distinguishability of other feature words js, static, and com. Therefore, the weight coefficient of the feature word 1665392849615 in the four feature words js, static, com, and 1665392849615 of the to-be-identified webpage is the highest. Specifically, the method for determining the weight coefficient of each feature word can be determined according to actual application scenarios, which is not limited in the present application. Further, the weight coefficients of all feature words in Table 3 can be obtained to obtain the weight coefficient distribution of each feature word. Specifically, refer to Table 4 for the weight coefficients of the feature words of the to-be-identified webpage:

[0132]

[0133]

[0134] Table 4

[0135] It can be understood that, from the weight coefficients of the various characteristic words of the to-be-identified webpage in Table 4, it can be known that the characteristic words with higher weight coefficients such as js, static, com, 1665392849615, and member have higher importance degrees for obtaining the webpage identifier of the to-be-identified webpage, and the characteristic words with lower weight coefficients have lower importance degrees for obtaining the webpage identifier of the to-be-identified webpage. By assigning the corresponding weight coefficients to the various characteristic words of the to-be-identified webpage, the accuracy of the webpage identifier processing of the to-be-identified webpage can be improved.

[0136] In some possible implementation manners, the webpage identification processing method provided by the embodiments of the present application can obtain the weight coefficient of each feature word in the plurality of feature words according to the following content. Specifically, first, the inverse document frequency of each feature word in the plurality of feature words in the corpus can be found, where the inverse document frequency of any feature word in the corpus can be understood as a measure for representing the general importance of the feature word. The inverse document frequency of any feature word in the corpus can be calculated by the ratio of the number of files containing the feature word to the total number of files in the corpus. It can be understood that if any feature word appears multiple times in the resource request text of the webpage to be identified (i.e., a file in the corpus) and rarely appears in the resource request texts of other webpages, the inverse document frequency of the feature word in the corpus is low, and the feature word has good class distinguishing ability, which is suitable for distinguishing the resource request texts of different webpages to be identified, i.e., the importance of obtaining the webpage identification of the webpage to be identified is high, and accordingly, the weight coefficient of the feature word is higher. Therefore, the webpage identification processing method provided by the embodiments of the present application can obtain the weight coefficient of each feature word by the inverse document frequency of each feature word in the corpus. Further, the weight coefficient of each feature word is inversely proportional to the inverse document frequency of the feature word in the corpus, and is proportional to the occurrence frequency of the feature word in the resource request text of the webpage to be identified. Therefore, the webpage identification processing method provided by the embodiments of the present application can also obtain the term frequency of each feature word based on the occurrence frequency of each feature word, and obtain the weight coefficient of each feature word based on the term frequency and the inverse document frequency of each feature word. Wherein, the term frequency of each feature word can be understood as the number of occurrences of the feature word in the resource request text. If the term frequency of any feature word is higher, the number of occurrences of the feature word in the resource request text is more, and the importance of the feature word is higher. It can be understood that the term frequency of any feature word is proportional to the weight coefficient of the feature word. Further, the product of the term frequency of any feature word and the inverse document frequency of the feature word can be calculated to obtain the weight coefficient of the feature word. Similarly, the webpage identification processing method provided by the embodiments of the present application can calculate the weight coefficients of all feature words in the resource request text based on the above content.

[0137] In some possible embodiments, as can be known from the above, after obtaining the plurality of feature words of the webpage to be identified, the plurality of feature words can be mapped into a plurality of weighted real number sequences based on the weight coefficients of the feature words. To this end, the webpage identification processing method provided in the embodiments of the present application maps the plurality of feature words of the webpage to be identified into a plurality of binary sequences respectively, and weights the binary sequence of any feature word based on the weight coefficient of the feature word, to obtain the weighted real number sequence of the feature word. Specifically, the webpage identification processing method provided in the embodiments of the present application can map the plurality of feature words in Table 4 through a hash function, to obtain the binary sequences of the plurality of feature words as shown in Table 5. Specifically, refer to Table 5, which is a table of binary sequences of the plurality of feature words of the webpage to be identified:

[0138]

[0139]

[0140]

[0141] Table 5

[0142] It can be understood that the above plurality of characteristic words can obtain a plurality of binary sequences with fixed length after being calculated by the hash function, wherein each binary sequence includes a plurality of binary values, and the number of binary values in any binary sequence is the length of the binary sequence. It can be understood that the arrangement order of the plurality of binary values in the binary sequence of each characteristic word is different, and the binary sequence of any characteristic word is the unique identifier of the characteristic word. Further, after obtaining the weight coefficients and binary sequences of the plurality of characteristic words as shown in Table 5, it can be known from the above that the importance of each characteristic word of the to-be-identified webpage is different. Therefore, in order to improve the accuracy of webpage identification processing, the webpage identification method provided by the embodiment of the application can weight the binary sequence of any characteristic word based on the weight coefficient of the characteristic word. Specifically, the weight coefficient of the characteristic word js as shown in Table 5 is 0.130435, and the binary sequence of the characteristic word js is 0000000000110000001101000001100100100110111011101100110011001111. If the binary sequence of the characteristic word js is weighted, the binary values in the binary sequence of the characteristic word js are weighted based on the weight coefficient 0.130435 of the characteristic word js, respectively. Specifically, when the binary value of the binary sequence is 0, the above weighting of the binary value based on the weight coefficient is that the weight coefficient 0.130435 is multiplied by -1 to obtain -0.130435, and when the binary value of the binary sequence is 1, the above weighting of the binary value based on the weight coefficient is that the weight coefficient 0.130435 is multiplied by 1 to obtain 0.130435. Similarly, the weight coefficient of the characteristic word js is used to weight all binary values in the binary sequence, and the weighted real sequence of the characteristic word js is obtained. It can be understood that according to the above, the webpage identification processing method provided by the embodiment of the application can weight the binary sequence of the plurality of characteristic words based on the weight coefficients of the characteristic words as shown in Table 5 to obtain a plurality of weighted real sequences corresponding to the plurality of characteristic words as shown in Table 6. Specifically, please refer to Table 6, which is a table of a plurality of weighted real sequences corresponding to a plurality of characteristic words of a to-be-identified webpage:

[0143]

[0144]

[0145]

[0146]

[0147] Table 6

[0148] In step S206, the weighted real number sequences are accumulated to obtain an accumulated real number sequence of the webpage to be identified, and the accumulated real number sequence is converted into binary to obtain an identification feature sequence corresponding to the resource request text.

[0149] In some possible embodiments, the webpage identification processing method provided by the embodiments of the present application can further accumulate the weighted real number sequences of the webpage to be identified to obtain an accumulated real number sequence of the webpage to be identified after obtaining the weighted real number sequences of the webpage to be identified based on the above content. Specifically, as shown in Table 6, when the weighted real number sequences of the feature word js and the feature word static of the webpage to be identified are accumulated, since each weighted real number sequence of the feature word includes weighted real numbers of a fixed length, in the accumulation process, the first weighted real number -0.1304 in the weighted real number sequence of the feature word js and the first weighted real number 0.0869 in the weighted real number sequence of the feature word static can be added to obtain the first accumulated real number obtained by accumulating the weighted real number sequences of the feature word js and the feature word static. The second weighted real number -0.1304 in the weighted real number sequence of the feature word js and the second weighted real number -0.0869 in the weighted real number sequence of the feature word static can be added to obtain the second accumulated real number obtained by accumulating the weighted real number sequences of the feature word js and the feature word static. Similarly, the Nth weighted real number in the weighted real number sequence of the feature word js and the Nth weighted real number in the weighted real number sequence of the feature word static can be added in the same way until each weighted real number in the weighted real number sequences of the feature word js and the feature word static is accumulated. It can be understood that the embodiments of the present application can accumulate the weighted real number sequences of the webpage to be identified as shown in Table 6 according to the above content to obtain the accumulated real number sequence SUM as shown in Table 6.

[0150] In some possible implementation manners, in order to simplify the processing of the webpage identification, the above accumulated real number sequence can be converted into binary. Specifically, the accumulated real number greater than 0 in the above accumulated real number sequence can be replaced by 1, and the accumulated real number less than or equal to 0 in the above accumulated real number sequence can be replaced by 0, so as to realize the binary conversion of the accumulated real number sequence of the webpage to be identified. For example, the embodiment of the present application takes the binary conversion of the accumulated real number sequence SUM of the webpage to be identified shown in Table 6 as an example for description. Specifically, the accumulated real number sequence SUM of the webpage to be identified shown in Table 6 includes a plurality of accumulated real numbers -1.304e-01, -1.304e-01, 0.000e+0, -1.387e-17, …, by replacing the accumulated real number less than or equal to 0 in the plurality of accumulated real numbers by 0, the plurality of accumulated real numbers -1.304e-01, -1.304e-01, 0.000e+0, -1.387e-17, … can be replaced by 0000…, so as to obtain the identification feature sequence of the webpage to be identified. Specifically, please refer to Table 7, which is a table of the identification feature sequence of the webpage to be identified:

[0151] Number Cumulative real Permutation result 1 -1.304e-01 0 2 -1.304e-01 0 3 0.000e+0 0 4 -1.387e-17 0 5 -4.130e-01 0 6 -2.826e-01 0 7 -2.391e-01 0 8 -3.478e-01 0 9 1.304e-01 1 10 -1.086e-01 0 11 0.000e+0 0 12 2.826e-01 1 13 1.739e-01 1 14 -3.260e-01 0 15 -3.913e-01 0 16 -6.521e-02 0 … … … 60 6.521e-02 1 61 6.521e-02 1 62 4.347e-02 1 63 4.347e-02 1 64 -6.521e-02 0

[0152] It can be understood that the webpage identification processing method provided by the embodiment of the present application can obtain the identification feature sequence of the webpage to be identified corresponding to the target website www.xxx.com as 0000000010011000000101000000100100000110110001101110110010011110 according to the above content. For the convenience of understanding, the embodiment of the present application takes the flow shown in Table 8 as an example to briefly describe the process of obtaining the identification feature sequence of the webpage to be identified corresponding to the target website www.xxx.com. Specifically, please refer to Table 9, which is a table of the process of obtaining the identification feature sequence of the webpage to be identified corresponding to the target website www.xxx.com: Figure 5 Figure 5 Figure 5 is another flow diagram of the webpage identification processing method provided by the embodiment of the present application. As shown in Table 10, the webpage identification processing method provided by the embodiment of the present application includes the following steps: Figure 5 ​​As shown, the webpage identification processing method provided by the embodiments of the present application can extract the words in the resource request text according to the above-mentioned separation string after obtaining the resource request text of the webpage to be identified based on the dynamic resource request and the static resource request of the webpage to be identified corresponding to the target URL, so as to realize the word segmentation of the resource request text. After the above-mentioned word segmentation processing obtains a plurality of words, the plurality of words can be classified to obtain classified words, and the word frequency of each classified word is calculated, so that the weight coefficient of each classified word is calculated according to the word frequency of each classified word. Subsequently, the K classified words with the highest importance can be obtained as the K feature words (i.e. TOP-K feature words) of the webpage to be identified according to the weight coefficient of each classified word. After obtaining the plurality of feature words of the webpage to be identified, the above-mentioned plurality of feature words can be mapped into binary sequences with a fixed length respectively, and the binary sequence of any feature word is weighted based on the weight coefficient of the feature word, so as to obtain the weighted real sequence of any feature word, thereby obtaining the URL real vector (i.e. the above-mentioned accumulated real sequence) corresponding to the target URL www.xxx.com. Further, by performing binary conversion on the URL real vector of the target URL, the identification feature sequence of the target URL can be obtained, that is, the webpage identification of the webpage to be identified corresponding to the target URL.

[0153] It can be understood that the identification feature sequence of the webpage to be identified can be understood as the webpage content for representing the webpage to be identified, that is, the higher the similarity of the webpage content of different webpages to be identified, the higher the similarity of the identification feature sequences of different webpages to be identified. Therefore, the identification feature sequence of the above-mentioned webpage to be identified can be directly determined as the webpage identification of the webpage to be identified, that is, the webpage identification of the webpage to be identified corresponding to the target URL www.xxx.com is 0000000010011000000101000000100100000110110001101110110010011110. In addition, the identification feature sequence of the webpage to be identified can be further processed to obtain the webpage identification of the webpage to be identified according to the needs of actual application scenarios, which is not limited herein.

[0154] In some possible implementation manners, after obtaining the webpage identifier of the webpage to be identified, the webpage identifier processing provided by the embodiments of the present application can further classify the webpage to be identified based on the webpage identifier of the webpage to be identified. It can be understood that the embodiments of the present application can search the webpage identification database based on the webpage identifier of the webpage to be identified to find a target webpage having a similarity less than a preset threshold with the webpage to be identified. If the webpage identification database has a webpage having a similarity less than the preset threshold with the webpage identifier of the webpage to be identified, the webpage having the similarity less than the preset threshold with the webpage identifier of the webpage to be identified is determined as the target webpage, and the webpage to be identified is classified based on the classification identifier of each webpage in the webpage identification database, that is, the webpage to be identified is classified into the classification of the target webpage. The classification identifier of any webpage is used to classify the any webpage. If the webpage identification database does not have a webpage having a similarity less than the preset threshold with the webpage identifier of the webpage to be identified, the webpage identifier of the webpage to be identified is stored into the webpage identification database to facilitate searching again. It can be understood that if the webpage identification database has a webpage having a similarity less than the preset threshold with the webpage identifier of the webpage to be identified, it indicates that the webpage content of the webpage has a similarity less than the preset threshold with the webpage content of the webpage to be identified, that is, the webpage and the webpage to be identified can be similar webpages, and therefore, the webpage to be identified can be classified into the same category as the webpage.

[0155] In some possible implementation manners, the webpage identifier processing method provided by the embodiments of the present application can determine the similarity between the webpage to be identified and any other webpage by calculating the Hamming distance between the webpage identifier of the webpage to be identified and the webpage identifier of any other webpage. Specifically, refer to Figure 6 , Figure 6 is another flowchart of the webpage identifier processing method provided by the embodiments of the present application. As shown in Figure 6 , the webpage identifier of the webpage to be identified of the first target website www.xxx.com is the first webpage identifier, and the webpage identifier of any other webpage, for example, the webpage identifier of the second target website www.yyy.com is the second webpage identifier. Specifically, refer to Figure 6As shown, the webpage identification processing method provided by the embodiment of the present application can extract the words in the resource request text according to the above-mentioned separation string after obtaining the resource request text of the webpage to be identified based on the dynamic resource request and the static resource request of the webpage to be identified corresponding to the first target website www.xxx.com, so as to realize the word segmentation of the resource request text. After obtaining the plurality of words through the above-mentioned word segmentation processing, the plurality of words can be classified to obtain classified words, and the word frequency of each classified word is calculated, so as to calculate the weight coefficient of each classified word according to the word frequency of each classified word. Subsequently, the K classified words with the highest importance can be obtained as the K feature words (i.e. TOP-K feature words) of the webpage to be identified according to the weight coefficient of each classified word. After obtaining the plurality of feature words of the webpage to be identified, the above-mentioned plurality of feature words can be respectively mapped into binary sequences with a fixed length, and the binary sequence of any feature word is weighted based on the weight coefficient of the feature word, so as to obtain the weighted real number sequence of any feature word, thereby obtaining the website real number vector corresponding to the above-mentioned target website www.xxx.com (i.e. the above-mentioned accumulated real number sequence). Further, the identification feature sequence of the first target website can be obtained through the binary conversion of the website real number vector of the target website, that is, the first webpage identification of the webpage to be identified corresponding to the first target website. Similarly, as shown, Figure 6 As shown, the webpage identification processing method provided by the embodiment of the present application can extract the words in the resource request text according to the above-mentioned separation string after obtaining the resource request text of the webpage to be identified based on the dynamic resource request and the static resource request of the webpage to be identified corresponding to the first target website www.xxx.com, so as to realize the word segmentation of the resource request text. After obtaining the plurality of words through the above-mentioned word segmentation processing, the plurality of words can be classified to obtain classified words, and the word frequency of each classified word is calculated, so as to calculate the weight coefficient of each classified word according to the word frequency of each classified word. Subsequently, the K classified words with the highest importance can be obtained as the K feature words (i.e. TOP-K feature words) of the webpage to be identified according to the weight coefficient of each classified word. After obtaining the plurality of feature words of the webpage to be identified, the above-mentioned plurality of feature words can be respectively mapped into binary sequences with a fixed length, and the binary sequence of any feature word is weighted based on the weight coefficient of the feature word, so as to obtain the weighted real number sequence of any feature word, thereby obtaining the website real number vector corresponding to the above-mentioned target website www.xxx.com (i.e. the above-mentioned accumulated real number sequence). Further, the identification feature sequence of the first target website can be obtained through the binary conversion of the website real number vector of the target website, that is, the first webpage identification of the webpage to be identified corresponding to the first target website. Similarly, as shown,

[0156] It can be understood that, based on the above steps, the first webpage identifier of the first target website and the second webpage identifier of the second target website are obtained, and then, as shown in Figure 6 , the similarity between the to-be-identified webpage of the first target website and the to-be-identified webpage of the second target website can be obtained by calculating the Hamming distance of the first webpage identifier and the second webpage identifier. If the Hamming distance of the first webpage identifier and the second webpage identifier is greater than 3 (i.e., a preset threshold), the similarity between the to-be-identified webpage corresponding to the first target website and the to-be-identified webpage corresponding to the second target website is high, and the to-be-identified webpage corresponding to the second target website can be determined as the target webpage. If the Hamming distance of the first webpage identifier and the second webpage identifier is less than or equal to 3, the similarity between the to-be-identified webpage and the webpage is low, and the to-be-identified webpage corresponding to the second target website cannot be determined as the target webpage. For ease of understanding, the first webpage identifier is 0000000010011000000101000000100100000110110001101110110010011110 and the second webpage identifier is 0001000010011000000101000000100100000110110001101110110010011110 in the above embodiment, which is taken as an example for description. For details, please refer to Figure 7 , Figure 7 is a schematic diagram of a calculation principle of the webpage identifier processing method provided in the embodiment of the present application. As shown in Figure 7 , the first webpage identifier and the second webpage identifier can be calculated by XOR to obtain the Hamming distance, wherein the Hamming distance is equal to the number of "1" after the first webpage identifier and the second webpage identifier are calculated by XOR. As can be seen, the Hamming distance of the first webpage identifier and the second webpage identifier is equal to 1, that is, the similarity between the to-be-identified webpage and the webpage is high, and the webpage can be determined as the target webpage.

[0157] The webpage identification processing method provided in the embodiments of the present application can obtain a static resource request and a dynamic resource request of a webpage to be identified, wherein the static resource request of the webpage to be identified is used to request a static webpage resource from a server, and the dynamic resource request of the webpage to be identified is used to request a real-time updated dynamic webpage resource. The static webpage resource can generate initial webpage content of the webpage to be identified under parsing and rendering of a terminal device, and the dynamic webpage resource can generate interactive webpage content of the webpage to be identified under parsing and rendering of the terminal device. It can be understood that when webpage content of different webpages to be identified is the same, the initial webpage content and the interactive webpage content of the different webpages to be identified are the same, and because the static resource request or the dynamic resource request is the same, the webpage content (the initial webpage content and the interactive webpage content) obtained by the request is also the same. Therefore, when the webpage content of different webpages to be identified is the same, the static resource request and the dynamic resource request of the different webpages to be identified are also the same. Therefore, the embodiments of the present application can obtain the webpage identification of the webpage to be identified through the static resource request and the dynamic resource request of the webpage to be identified, and the webpage identification of different webpages to be identified can be used to represent the similarity of the different webpages to be identified. Further, the dynamic resource request and the static resource request of the webpage to be identified can be used to generate a resource request set of the webpage to be identified, the resource request text of the webpage to be identified can be mapped to an identification feature sequence of the webpage to be identified, and the webpage identification of the webpage to be identified can be generated based on the identification feature sequence. It can be understood that when the webpage content of different webpages to be identified is the same, the resource request of the different webpages to be identified is also the same. Therefore, the identification feature sequence obtained by mapping the resource request text of the webpage to be identified can be used to represent the webpage identification of the webpage to be identified. The webpage identification processing method of the embodiments of the present application can obtain the webpage identification of the webpage to be identified based on the dynamic resource request and the static resource request of the webpage to be identified, and when the webpage to be identified is loaded by a website in a manner of dynamically embedding a webpage, the webpage identification processing method can improve the accuracy of obtaining the webpage identification of the webpage to be identified, improve the accuracy of judging the similarity of different webpages to be identified, and has high applicability.

[0158] Based on the description of the webpage identification processing method embodiments, the embodiments of the present application further disclose a webpage identification processing apparatus. The webpage identification processing apparatus can be applied to Figures 1 to 7 The webpage identification processing method of the embodiments shown in the foregoing method embodiments can be executed by the webpage identification processing apparatus. Here, the webpage identification processing apparatus can be the service server or the terminal device in the foregoing method embodiments, that is, the webpage identification processing apparatus can be the execution subject of the webpage identification processing method in the foregoing method embodiments. Please refer to Figures 1 to 7 The webpage identification processing apparatus can be the service server or the terminal device in the foregoing method embodiments, that is, the webpage identification processing apparatus can be the execution subject of the webpage identification processing method in the foregoing method embodiments. Please refer to Figures 1 to 7 The webpage identification processing apparatus can be the service server or the terminal device in the foregoing method embodiments, that is, the webpage identification processing apparatus can be the execution subject of the webpage identification processing method in the foregoing method embodiments. Please refer to Figure 8 , Figure 8This is a schematic diagram of a webpage identifier processing device provided in an embodiment of this application. In this embodiment, the device can operate the following modules:

[0159] Resource request acquisition module 1 is used to acquire dynamic resource requests of the webpage to be identified. The dynamic resource requests are used to request real-time updated dynamic webpage resources from the server. The dynamic webpage resources are used to render the interactive webpage content corresponding to the webpage to be identified.

[0160] The resource request acquisition module 1 is also used to acquire the static resource request of the webpage to be identified. The static resource request is used to request static webpage resources from the server. The static webpage resources are used to render the initial webpage content of the webpage to be identified.

[0161] The request text acquisition module 2 is used to generate a set of resource requests to be identified for the webpage to be identified based on the dynamic resource requests and static resource requests of the webpage to be identified, and to obtain the resource request text of the webpage to be identified based on the request fields included in each resource request in the set of resource requests to be identified.

[0162] The generation module 3 is used to map the resource request text of the webpage to be identified to the identifier feature sequence of the webpage to be identified, and generate the webpage identifier of the webpage to be identified based on the identifier feature sequence.

[0163] According to the above Figure 3 The corresponding implementation, Figure 3 The implementation method described in steps S101 to S104 of the webpage identifier processing method shown can be derived from... Figure 8 The various modules of the illustrated device are executed. For example, the above... Figure 3 The implementation method described in step S101 of the webpage identifier processing method shown can be derived from... Figure 7 The resource request acquisition module 1 in the device shown executes the request, and the implementation method described in step S102 can be achieved by... Figure 7 The resource request acquisition module 1 in the illustrated device performs the execution, the implementation method described in step S103 can be performed by the request text acquisition module 2, and the implementation method described in step S104 can be performed by the generation module 3. The implementation methods performed by the resource request acquisition module 1, the request text acquisition module 2, and the generation module 3 are as described above. Figure 3 The implementation methods provided for each step in the corresponding embodiments will not be repeated here.

[0164] The resource request acquisition module 1 mentioned above includes:

[0165] The dynamic resource request obtaining unit 11 is configured to obtain a plurality of initial dynamic resource requests of a webpage to be identified, and obtain a status code of each initial dynamic resource request, and determine an initial dynamic resource request with a status code equal to a preset status code in the plurality of initial dynamic resource requests as a dynamic resource request of the webpage to be identified.

[0166] The resource request obtaining module 1 further includes:

[0167] The static resource request obtaining unit 12 is configured to obtain a plurality of initial static resource requests of the webpage to be identified, and obtain a status code of each initial static resource request, and determine an initial static resource request with a status code equal to a preset status code in the plurality of initial static resource requests as a static resource request of the webpage to be identified.

[0168] The generating module 3 includes:

[0169] The feature word extracting unit 31 is configured to perform feature word extraction on a resource request text of the webpage to be identified to obtain a plurality of feature words;

[0170] The weighting unit 32 is configured to obtain a weight coefficient of each feature word in the plurality of feature words, and map the plurality of feature words into a plurality of weighted real number sequences based on the weight coefficient of each feature word;

[0171] The identification feature sequence generating unit 33 is configured to accumulate the plurality of weighted real number sequences to obtain an accumulated real number sequence of the webpage to be identified, and perform binary conversion on the accumulated real number sequence to obtain an identification feature sequence corresponding to the resource request text.

[0172] The feature word extracting unit 31 includes:

[0173] The word division sub-unit 311 is configured to identify a separation string in the resource request text of the webpage to be identified, and divide the resource request text into a plurality of words based on the identified separation string;

[0174] The classification sub-unit 312 is configured to classify the plurality of words to obtain a plurality of classified words, wherein one classified word includes a plurality of words of the same category;

[0175] The word frequency obtaining sub-unit 313 is configured to obtain an occurrence frequency of each classified word based on a number of words included in the classified word, and determine K classified words with higher occurrence frequencies from the plurality of classified words, wherein K is a positive integer;

[0176] The feature word determining sub-unit 314 is configured to determine the K classified words as a plurality of feature words of the resource request text.

[0177] The weighting unit 32 comprises:

[0178] The lookup sub-unit 321 is configured to look up the inverse document frequency of each feature word in the corpus among the plurality of feature words.

[0179] The weight coefficient acquisition sub-unit 322 is configured to acquire the term frequency of each feature word based on the occurrence frequency of the feature word, and acquire the weight coefficient of each feature word based on the term frequency and the inverse document frequency of the feature word.

[0180] The weighted real number acquisition sub-unit 323 is configured to map any feature word in the plurality of feature words into a binary sequence, and weight the binary sequence of the feature word based on the weight coefficient of the feature word, to obtain a weighted real number sequence of the feature word.

[0181] The weighted real number sequence acquisition sub-unit 324 is configured to obtain a plurality of weighted real number sequences mapped by the plurality of feature words based on the weighted real number sequence of each feature word.

[0182] The weighted real number acquisition sub-unit 323 is further configured to:

[0183] map any feature word in the plurality of feature words into a binary sequence of a preset length based on a hash function, the binary sequence of the feature word comprising a plurality of binary values.

[0184] multiply the weight coefficient of the feature word by the plurality of binary values in the binary sequence of the feature word to weight the binary sequence of the feature word.

[0185] The identification feature sequence generation unit 33 is further configured to:

[0186] replace the accumulated real numbers greater than 0 in the accumulated real number sequence with 1, and replace the accumulated real numbers less than or equal to 0 in the accumulated real number sequence with 0, and determine the accumulated real number sequence after the replacement as the identification feature sequence corresponding to the resource request text to perform binary conversion on the accumulated real number sequence.

[0187] The apparatus further comprises:

[0188] The classification module 4 is configured to search for a target webpage with a similarity less than a preset threshold to the webpage to be identified in a webpage identification database based on the webpage identification of the webpage to be identified, and classify the webpage to be identified based on the classification identification of the target webpage. Each webpage in the webpage identification database has a classification identification, and the classification identification of any webpage is used to classify the webpage.

[0189] The webpage identification processing method provided by the embodiment of the present application can obtain the static resource request and the dynamic resource request of the webpage to be identified. The static resource request of the webpage to be identified is used to request the static webpage resource from the server, and the dynamic resource request of the webpage to be identified is used to request the real-time updated dynamic webpage resource. The static webpage resource can generate the initial webpage content of the webpage to be identified under the parsing and rendering of the terminal device, and the dynamic webpage resource can generate the interactive webpage content of the webpage to be identified under the parsing and rendering of the terminal device. It can be understood that when the webpage contents of different webpages to be identified are the same, the initial webpage contents and the interactive webpage contents of different webpages to be identified are the same, and because the static resource request or the dynamic resource request is the same, the webpage contents (the initial webpage content and the interactive webpage content) obtained by the request are also the same. Therefore, when the webpage contents of different webpages to be identified are the same, the static resource request and the dynamic resource request of different webpages to be identified are also the same. Therefore, the webpage identification of the webpage to be identified can be obtained by the static resource request and the dynamic resource request of the webpage to be identified. The webpage identification of different webpages to be identified can be used to represent the similarity of different webpages to be identified. Further, the dynamic resource request and the static resource request of the webpage to be identified can be used to generate the resource request set of the webpage to be identified. The resource request text of the webpage to be identified is mapped to the identification feature sequence of the webpage to be identified, and the webpage identification of the webpage to be identified is generated based on the identification feature sequence. It can be understood that when the webpage contents of different webpages to be identified are the same, the resource request of different webpages to be identified is also the same. Therefore, the identification feature sequence obtained by mapping the resource request text of the webpage to be identified can be used to represent the webpage identification of the webpage to be identified. The webpage identification processing method provided by the embodiment of the present application can obtain the webpage identification of the webpage to be identified based on the dynamic resource request and the static resource request of the webpage to be identified. When the webpage to be identified is loaded by the website in the form of dynamically embedded webpage, the accuracy of obtaining the webpage identification of the webpage to be identified can be improved, the accuracy of judging the similarity of different webpages to be identified can be improved, and the applicability is high.

[0190] In the embodiments of the present application, each module in the device shown in the above figure can be combined into one or several other modules respectively or entirely to constitute, or some module(s) therein can be further split into a plurality of modules of smaller functions to constitute, which can realize the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions, and in actual application, the function of one module can also be realized by a plurality of modules, or the functions of a plurality of modules are realized by one module. In other feasible implementation manners of the present application, the above device can also include other modules, and in actual application, these functions can also be realized by other modules, and can be realized by cooperation of a plurality of modules, which is not limited herein.

[0191] Please refer to Figure 9 , Figure 9 is a structural schematic diagram of a computer device provided by the embodiments of the present application. As shown in Figure 9 , the computer device 1000 can be a terminal device in the embodiments corresponding to the above. Figures 1-7 The computer device 1000 can include a processor 1001, a network interface 1004 and a memory 1005, in addition, the computer device 1000 can also include a user interface 1003 and at least one communication bus 1002. Wherein, the communication bus 1002 is used to realize the connection communication between these components. Wherein, the user interface 1003 can include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a non-volatile memory, for example, at least one disk memory. The memory 1005 can optionally be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 9 , the memory 1005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module and a device control application program.

[0192] Wherein, the network interface 1004 in the computer device 1000 can also be connected with the terminal 200b in the embodiments corresponding to the above Figure 1 , and the optional user interface 1003 can also include a display screen (Display) and a keyboard (Keyboard). In Figure 9In the computer device 1000 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for developers to input; and the processor 1001 can be used to invoke a device control application stored in the memory 1005 to implement the foregoing Figure 3 The embodiment corresponds to a webpage identification processing method.

[0193] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the foregoing Figure 3 The embodiment corresponds to a webpage identification processing method, and details are not described herein. In addition, the beneficial effects of using the same method are not described herein.

[0194] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, the computer program including program instructions, the program instructions being executed by a processor to implement Figure 3 The webpage identification processing method provided by each step, and details can be referred to the foregoing Figure 3 The implementation manner provided by each step, and details are not described herein.

[0195] The computer readable storage medium described above can be the webpage identification processing apparatus provided by any of the foregoing embodiments or the internal storage unit of the computer device, for example, the hard disk or the memory of the computer device. The computer readable storage medium can also be the external storage device of the computer device, for example, the plug-in hard disk, the smart media card (SMC), the secure digital (SD) card, the flash card, and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0196] The terms "first", "second", and the like in the specification and claims of the embodiments of the present application and the drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, device, product, or equipment.

[0197] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0198] The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 1 The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow

[0199] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the present application. Therefore, equivalent changes made in accordance with the claims of the present application are still within the scope of the present application.

Claims

1. A method of processing a web page identification, characterized by, The method comprises the following steps: acquiring a dynamic resource request of a to-be-identified webpage, the dynamic resource request being used to request a server to acquire a real-time updated dynamic webpage resource, the dynamic webpage resource being used to render an interactive webpage content corresponding to the to-be-identified webpage; acquiring a static resource request of the to-be-identified webpage, the static resource request being used to request the server to acquire a static webpage resource, the static webpage resource being used to render an initial webpage content of the to-be-identified webpage; generating a to-be-identified resource request set of the to-be-identified webpage based on the dynamic resource request and the static resource request of the to-be-identified webpage, and acquiring a resource request text of the to-be-identified webpage based on a request field included in each resource request in the to-be-identified resource request set; extracting feature words from the resource request text of the to-be-identified webpage to obtain a plurality of feature words; acquiring a weight coefficient of each feature word in the plurality of feature words, and mapping the plurality of feature words into a plurality of weighted real number sequences based on the weight coefficient of each feature word; accumulating the plurality of weighted real number sequences to obtain an accumulated real number sequence of the to-be-identified webpage, and performing binary conversion on the accumulated real number sequence to obtain an identification feature sequence corresponding to the resource request text, and generating a webpage identification of the to-be-identified webpage based on the identification feature sequence.

2. The method of claim 1, wherein, The acquiring of the dynamic resource request of the to-be-identified webpage comprises: acquiring a plurality of initial dynamic resource requests of the to-be-identified webpage, and acquiring a status code of each initial dynamic resource request, and determining an initial dynamic resource request with a status code equal to a preset status code in the plurality of initial dynamic resource requests as the dynamic resource request of the to-be-identified webpage.

3. The method of claim 1, wherein, The acquiring of the static resource request of the to-be-identified webpage comprises: acquiring a plurality of initial static resource requests of the to-be-identified webpage, and acquiring a status code of each initial static resource request, and determining an initial static resource request with a status code equal to a preset status code in the plurality of initial static resource requests as the static resource request of the to-be-identified webpage.

4. The method of claim 1, wherein, The extracting of the feature words from the resource request text of the to-be-identified webpage comprises: identifying a separation string in the resource request text of the to-be-identified webpage, and dividing the resource request text into a plurality of words based on the identified separation string; classifying the plurality of words to obtain a plurality of classified words, wherein each classified word comprises a plurality of words of the same category; acquiring a frequency of occurrence of each classified word based on the number of words included in the each classified word, and determining K classified words with higher frequency of occurrence from the plurality of classified words, wherein K is a positive integer; determining the K classified words as the plurality of feature words of the resource request text.

5. The method of claim 4, wherein, The acquiring of the weight coefficient of each feature word in the plurality of feature words, and the mapping of the plurality of feature words into the plurality of weighted real number sequences based on the weight coefficient of each feature word, comprises: looking up an inverse document frequency of each feature word in a corpus. obtaining term frequencies of the feature words based on the frequencies of the feature words, and obtaining weight coefficients of the feature words based on the term frequencies and inverse document frequencies; mapping any feature word in the feature words into a binary sequence, and weighting the binary sequence of the feature word based on the weight coefficient of the feature word, to obtain a weighted real sequence of the feature word; obtaining a plurality of weighted real sequences mapped by the feature words based on the weighted real sequences of the feature words.

6. The method of claim 5, wherein, The mapping any feature word in the feature words into a binary sequence, and weighting the binary sequence of the feature word based on the weight coefficient of the feature word, includes: mapping any feature word in the feature words into a binary sequence of a preset length based on a hash function, the binary sequence of the feature word including a plurality of binary values; multiplying the weight coefficient of the feature word by the plurality of binary values in the binary sequence of the feature word to weight the binary sequence of the feature word.

7. The method of claim 1, wherein, The accumulated real sequence includes a plurality of accumulated real numbers, and the binary conversion of the accumulated real sequence includes: substituting the accumulated real numbers greater than 0 in the accumulated real sequence into 1, and substituting the accumulated real numbers less than or equal to 0 in the accumulated real sequence into 0, and determining the accumulated real sequence obtained after the substitution as an identification feature sequence corresponding to the resource request text to perform the binary conversion on the accumulated real sequence.

8. The method according to any one of claims 1 to 7, characterized in that, After the generation of the web page identifier of the to-be-identified web page based on the identification feature sequence, the method further includes: finding a target web page with a similarity less than a preset threshold to the to-be-identified web page in a web page identification database based on the web page identifier of the to-be-identified web page, and classifying the to-be-identified web page based on a classification identifier of the target web page; each web page in the web page identification database has a classification identifier, and the classification identifier of any web page is used for classifying the any web page.

9. A web page identification processing apparatus characterized by comprising: includes: a resource request obtaining module, configured to obtain a dynamic resource request of a to-be-identified web page, the dynamic resource request being used to request a server to obtain real-time updated dynamic web page resources, and the dynamic web page resources being used to render into interactive web page content corresponding to the to-be-identified web page; the resource request obtaining module is further configured to obtain a static resource request of the to-be-identified web page, the static resource request being used to request the server to obtain static web page resources, and the static web page resources being used to render into initial web page content of the to-be-identified web page; a request text obtaining module, configured to generate a to-be-identified resource request set of the to-be-identified web page based on the dynamic resource request and the static resource request of the to-be-identified web page, and obtain a resource request text of the to-be-identified web page based on request fields included in each resource request in the to-be-identified resource request set; The determining module is configured to perform feature word extraction on the resource request text of the webpage to be identified to obtain a plurality of feature words, acquire a weight coefficient of each feature word in the plurality of feature words, map the plurality of feature words into a plurality of weighted real number sequences based on the weight coefficient of each feature word, accumulate the plurality of weighted real number sequences to obtain an accumulated real number sequence of the webpage to be identified, perform binary conversion on the accumulated real number sequence to obtain an identification feature sequence corresponding to the resource request text, and generate a webpage identification of the webpage to be identified based on the identification feature sequence.

10. A computer device, comprising: Comprise: A processor, a memory and a network interface; The processor is connected with the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store program code, and the processor is configured to invoke the program code to execute the steps of the method in any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program comprises program instructions. When the program instructions are executed by a processor, the steps of the method in any one of claims 1 to 8 are executed.

Images