Inference of information about a web page based on a uniform resource locator of the web page

Abstract

This paper describes a technique related to inferring information about a webpage based on the semantics of its Uniform Resource Location (URL). The URL is lexicalized to create a sequence of lexical terms. Based on the sequence of lexical terms, an embedding for the URL is generated, where the embedding represents the semantics of the URL. Based on the embedding for the URL, information about the webpage pointed to by the URL is inferred, the webpage is retrieved, and information is extracted from the webpage based on the inferred information about the webpage.

Description

Background Technology

[0001] A computer-implemented search engine is configured to receive queries from client computing devices and return search results representing documents (e.g., web pages, images, videos, etc.) to the client computing devices, wherein the documents corresponding to the search results have been identified by the search engine as relevant to the query. More specifically, when the search engine receives a query, it searches a computer-readable search engine index based on the query, wherein the search engine index includes entries corresponding to web pages that have previously been crawled by the search engine's web crawler. The search engine identifies the web pages indexed in the search engine index based on the query, and then ranks the identified web pages based on several features of the web pages and / or the query, such as a static score assigned to the web page, the number of inbound and outbound links in the web page, whether the words in the query match the words in the web page title, and many other features. The search engine then returns a search engine results page (SERP) to the client computing device, wherein, for example, the SERP includes ten search results representing the ten web pages ranked highest by the search engine.

[0002] A general-purpose search engine's index comprises billions of entries, representing hundreds of millions to billions of web pages available on the World Wide Web. However, an even larger number of web pages exist; therefore, it can be determined that a significant number of web pages are not represented in the search engine index. These web pages are not represented in the search engine index because: 1) the search engine is unaware of the web page's existence; or 2) the search engine deems the web page unworthy of being represented in its index (e.g., the web page contains content irrelevant to the search engine's users, or the web page contains malicious content, etc.).

[0003] Traditionally, when a search engine knows about a webpage but chooses not to index it, it still has to retrieve the page and analyze its content before deciding not to index it, consuming the search engine's limited resources. Alternatively, the search engine might determine not to index a page based on some known arbitrary characteristic (such as the identity of the website to which the page belongs, the number of known inbound links to the page, etc.). However, it can be determined that the number of inbound links to a page does not indicate the page's content. Therefore, without retrieving and analyzing the page's content, there is currently no robust method to identify which pages are "important" enough to be included in the search engine's index. Summary of the Invention

[0004] The following is a brief summary of the invention, describing the subject matter in more detail herein. This summary is not intended to limit the scope of the claims.

[0005] This article describes various techniques related to inferring information about a webpage based on alphanumeric characters in the Uniform Resource Locator (URL) pointing to the webpage. The inferred information may include, but is not limited to: 1) the likelihood that any user of the search engine would choose to represent the webpage in search results if the webpage were presented to them on a search engine results page (SERP); 2) the likelihood that the content of the webpage is related to a specific topic; 3) the likelihood that the webpage would lead to the installation of malware on a client computing device when retrieved; 4) the likelihood that the content of the webpage has been updated within a certain threshold time; 5) the likelihood that the content of the webpage is written in a specific language; 6) the likelihood that the webpage has a threshold number of outbound links; 7) the likelihood that the webpage is accessible to users (e.g., the webpage is a permission-based page and / or is region-locked); and / or 8) the likelihood that a crawler will be able to successfully crawl the webpage.

[0006] Combining inferred information about web pages, search engines identify the URLs of web pages not represented in their search engine indexes. The search engine extracts semantic information from the URLs and uses this information to infer information about the web pages. More specifically, the URLs are fed to a tokenizer, which tokenizes the URLs to form a sequence of tokens. In one example, the tokenizer is learned based on alphanumeric patterns identified in a vast number of URLs from a known vast number of web pages (e.g., billions to trillions of web pages). This allows the learned tokenizer to search for patterns in URLs and tokenize the URLs based on these patterns.

[0007] Once a URL is tokenized, the tokens can be mapped to corresponding identifiers. For example, for a URL... http: / / www.example.com The lexer can tokenize such a URL into the tokens "http: / / ", "www.", "example", and ".com", and map these tokens to four corresponding token identifiers. Optionally, the lexer forms an n-gram based on the tokens and maps the n-gram to the corresponding identifier (where the n-gram identifier does not overlap with the token identifier), where n can be 2, 3, 4, 5, etc. Therefore, continuing the example where the n-gram is a bigram, the lexer forms the bigram "http: / / www.", "www.example", "example.com", and ".comull" and maps the bigram to four corresponding bigram identifiers. The lexer then outputs an identifier representing the URL, where the identifier consists of a series of token identifiers and a series of n-gram identifiers.

[0008] Word embeddings are performed on the sequence of identifiers, such that each identifier (and therefore each token and each n-gram output by the lexer) is represented as an s-dimensional vector. Thus, a URL is represented by a sequence of s-dimensional vectors, each s-dimensional vector encoding semantic information about the tokens or n-grams represented by the s-dimensional vectors.

[0009] URL embeddings are generated based on an s-dimensional vector sequence, where any suitable method can be used to generate the URL embeddings. For example, the values ​​in the s-dimensional vector are averaged to form the URL embedding. In another example, a recurrent neural network (RNN) is provided with an s-dimensional vector and configured to output the URL embedding. Other embodiments can also be considered. The vector of URLs is provided to a classifier, where, in one example, the classifier is configured to output a score indicating the probability that the search results for a webpage will be clicked by any user if the search results for that webpage are included in the SERP returned to any user. When such a score is above a threshold (or alternatively, within a certain maximum threshold number of scores output by the classifier), the webpage pointed to by the URL is retrieved, information is extracted from that webpage, and an entry for that webpage in the search engine index is created based on the information extracted from that webpage. In this way, computational resources are saved by not retrieving and extracting webpages that do not have sufficient scores.

[0010] The classifier is trained using a relatively large number of training items identified from search engine logs and / or from a list of URLs of web pages that are known to exist but not represented in the search engine index. Each training item is a pair that includes a URL and a value indicating whether the search result corresponding to that URL was clicked by one or more search engine users. Therefore, for example, for...<URL1,1> It is a training project where the search result corresponding to URL1 is selected by one or more users of the search engine, while for...<URL2,0> The negative training items are those where the search result corresponding to URL2 was not selected by one or more users of the search engine. In the example training set, the training items consist of approximately equal numbers of positive and negative training items.

[0011] The lexer outputs identifiers for URLs in the training list and generates URL embeddings for the URLs (as described above). The classifier is trained based on these URL embeddings representing the URLs and corresponding values ​​from the training pairs; any suitable training technique can be used to train the classifier. Therefore, when the classifier receives previously unseen embeddings representing URLs, it outputs a score indicating the probability that the search result corresponding to the URL will be selected by any user when included in the SERP returned to that user. The classifier outputs this probability based on the URL and does not need to retrieve the webpage pointed to by the URL to calculate the score.

[0012] As previously stated, the techniques described herein are not limited to calculating a score indicating the probability that a search result will be selected based on the URL corresponding to the search result. In other examples, a classifier is trained to output a score indicating the probability that a webpage pointed to by a URL contains content related to a specific topic (or multiple topics); a classifier is trained to output a score indicating the probability that a webpage will lead to malware being installed on a client computing device retrieving the webpage; a classifier is trained to output a score indicating the probability that a webpage pointed to by a URL contains a certain threshold number of outbound links, and so on.

[0013] To provide a basic understanding of certain aspects of the systems and / or methods discussed herein, the above summary provides a brief overview of the invention. This summary is not a broad overview of the systems and / or methods discussed herein. It is not intended to identify key / important elements, nor to define the scope of such systems and / or methods. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that follows. Attached Figure Description

[0014] Figure 1 This is a functional block diagram of an exemplary computing system configured to update a search engine index to include entries for web pages based on a score calculated for each web page, representing the "importance" of the web page.

[0015] Figure 2 It describes the conversion from Uniform Resource Locator (URL) to a sequence of identifiers.

[0016] Figure 3 Showing the target Figure 2 The word embeddings are generated from the identifiers in the identifier sequence shown.

[0017] Figure 4 This is a functional block diagram of a computational system configured to learn and adapt to a lexer module that performs lexerization on URLs.

[0018] Figure 5 This is a functional block diagram of a computational system configured to learn a classifier module adapted to calculate a score for a webpage based on its URL.

[0019] Figure 6 A method for training a classifier is shown, wherein when training the classifier, it is configured to output the score of a webpage based on the URL of the webpage.

[0020] Figure 7 A method for updating a search engine index based on a score calculated for a webpage, where the score is based on the webpage's URL, is shown.

[0021] Figure 8 This is an exemplary computing system. Detailed Implementation

[0022] The following description, with reference to the accompanying drawings, describes various techniques for inferring information about web pages using web page-based Uniform Resource Locators (URLs), wherein the same reference numerals are used throughout the text to denote the same elements. In the following description, numerous specific details are presented for illustrative purposes and to provide a thorough understanding of one or more aspects. However, it will be apparent that such aspects(s) can be implemented without these specific details. In other instances, well-known structures and devices are shown in block diagram form for ease of description of one or more aspects. Furthermore, it should be understood that functions described as being performed by certain system components can be performed by multiple components. Similarly, for example, components can be configured to perform functions described as being performed by multiple components.

[0023] This document describes various techniques related to inferring information about a webpage from its URL. Information about a webpage that can be inferred includes, but is not limited to: 1) the likelihood that a search result representing the webpage will be selected by any user of the search engine when the search engine returns the search results representing the webpage as part of a search engine results page (SERP) to any user; 2) the likelihood that the webpage contains content related to one or more specific topics; 3) the likelihood that the webpage contains a threshold number of outbound links pointing to other documents; 4) the likelihood that the webpage will lead to the installation of malware on a client's computing device when the webpage is retrieved; 5) the likelihood that the webpage content has changed within a certain threshold time period (e.g., within the last two weeks); 6) the likelihood that the webpage content is written in a specific language; 7) the likelihood that a crawler will be able to successfully crawl the webpage; and / or 8) the likelihood that the webpage is accessible to end users (e.g., the webpage requires permission, the webpage is only available to certain geographic regions, etc.). As will be described in detail below, information about a webpage is inferred based on a semantic understanding of its URL, where the URL is tokenized to generate tokens, and embeddings of the URL are generated based on these tokens, where the embeddings represent the semantics of the webpage's URL.

[0024] The techniques described in this article offer numerous advantages. These advantages include reduced computational resources compared to traditional techniques, as search engines can save computational resources by determining not to crawl and / or index “spam” pages based on information inferred about such pages from their URLs. These techniques are also associated with improved search engine indexing, as semantic information extracted from the URLs of web pages is used to determine whether to include an entry for a page in the search engine index. Furthermore, since the search engine index does not include entries for “spam” pages, the computer-readable storage space required to store the index is reduced, and search results can be returned faster because the search engine does not need to consider entries for “spam” pages. Other advantages will also be highlighted.

[0025] Now for reference Figure 1 The diagram presents a functional block diagram of system 100, wherein system 100 is configured to update a search engine index to include entries for web pages accessible via the Internet based on inferred information about web pages, and further wherein the information is inferred based on the URLs of the web pages. System 100 includes multiple web servers 102-104 and a computing system 106 communicating with web servers 102-104 via a network 107. Web servers 102-104 host corresponding websites, wherein each website hosted by web servers 102-104 includes at least one web page. Figure 1 As shown, a first web server 102 hosts a first website 108, which includes N web pages 110-112. A Qth web server 104 hosts a Qth website 114, which includes M web pages 116-118. Although each of web servers 102-104 is shown as hosting a single website, it is understood that one or more of web servers 102-104 may host multiple websites. Furthermore, although websites 108 and 114 are shown as including several web pages, one or more of websites 108 and 114 may include a single web page. Web pages 110-112 and 116-118 have URLs assigned to them, pointing to the locations where web pages 110-112 and 116-118 can be retrieved.

[0026] Computing system 106 includes a computing system implementing a search engine. For this purpose, computing system 106 includes a computer-readable data repository 120 that maintains a URL list 122, where each URL in URL list 122 points to a webpage that can be retrieved via the Internet. The data repository also includes a search engine index 124, which includes entries for the individual webpages that have been crawled by the search engine's crawler. In one example, the URLs in URL list 122 correspond to webpages that the search engine knows about but have no entries in search engine index 124. Websites 108 and 114 may be relatively new websites, where sitemaps for websites 108 and 114 have been submitted to computing system 106, but the search engine has not yet crawled webpages 110-112 and 116-118 of websites 108 and 114, respectively. The sitemaps include URLs for webpages 110-112 and 116-118, therefore URL list 122 includes URLs for webpages 110-112 and 116-118.

[0027] The technique described herein involves inferring information about web pages 110-112 and 116-118 based on their corresponding URLs, without needing to retrieve web pages 110-112 and 116-118 from web servers 102-104 to analyze their content. As will be described in more detail below, information about the web pages can be inferred based on semantic information in the URLs extracted by computing system 106.

[0028] The computing system 106 includes a processor 126 and a memory 128, the memory 128 including data retrieved by the processor 126 and instructions executed by the processor 126. The memory 128 has a P-th URL 130 loaded therein, wherein the P-th URL 130 is retrieved from a URL list 122 in a data repository 120. In one example, the P-th URL 130 is the URL of a first webpage 110 of a first website 108 hosted by a first web server 102. The memory 128 also includes a lexicon module 132 configured to lexiconize URLs such that the lexicon module 132 decomposes a received URL into a sequence of lexicons. The memory 128 also includes an embedder module 134 configured to generate embeddings of URLs based on the lexicons generated by the lexicon module 132. The embeddings represent the semantics of the URL lexiconized by the lexicon module 132.

[0029] The memory 128 also includes a classifier module 136 configured to receive the embedding of a URL output by the embedder module 134 and calculate a score based on the embedding. In one embodiment, the score calculated by the embedder module 134 indicates the likelihood that any user of the search engine would select the search result corresponding to that URL if the search result is included in a SERP returned by the search engine. Finally, the memory 128 includes a crawler module 138 configured to crawl the webpage when the score output by the classifier module 136 for the webpage's URL exceeds a threshold (or when the score is within the highest threshold number of scores).

[0030] The operation of the lexicon module 132, the embedder module 134, the classifier module 136, and the crawler module 138 is now described with reference to the Pth URL 130. The lexicon module 132 receives the Pth URL 130, parses it, and decomposes it into a sequence of lexicons. In one example, the lexicon module 132 is trained to identify a predefined number of distinct lexicons (e.g., 250,000 potential lexicons) and parses the Pth URL 130 to search for such lexicons. After lexiconizing the Pth URL, the lexicon module 132 maps the lexicons to predefined (numerical) lexicon identifiers. Thus, the lexicon module 132 converts the sequence of lexicons extracted from the Pth URL 130 into a sequence of lexicon identifiers. Optionally, the lexer module 132 forms a sequence of n-grams (e.g., bigrams, trigrams, quadrams, etc.) and maps each n-gram to one of a predefined number of n-gram identifiers. In a particular example, the lexer module 132 forms a sequence of bigrams from the lexer sequence and hashes each bigram to one of 150 million bigram identifiers. Thus, the lexer module 132 converts the P-th URL 130 into multiple identifiers, where the identifiers include a sequence of lexer identifiers and a sequence of n-gram identifiers.

[0031] Embedder module 134 performs word embedding on each identifier in the identifier sequence output by lexer module 132. Specifically, for each identifier output by lexer module 132, embedder module 134 generates an s-dimensional vector, where the distance between two s-dimensional vectors represents the semantic similarity between the lexical and / or n-gram representations by the s-dimensional vectors. According to one example, s is two, three, four, or five. Thus, embedder module 134 generates a sequence of s-dimensional vectors, each s-dimensional vector representing the semantic information of a lexical or n-gram in the P-th URL 130.

[0032] Embedder module 134 constructs an embedding for the P-th URL based on an s-dimensional vector sequence (e.g., embedder module 134 "combines" the sequence of s-dimensional vectors into an embedding for the P-th URL 130). Embedder module 134 can utilize any suitable method to generate the embedding for the P-th URL, including calculating the average of entries in the s-dimensional vector, feeding the s-dimensional vector to a recurrent neural network (RNN) trained to generate URL embeddings based on a sequence of s-dimensional vectors, feeding the s-dimensional vector to a bidirectional RNN with an attention mechanism, wherein the Bi-RNN is trained to generate URL embeddings based on a sequence of s-dimensional vectors, or feeding the s-dimensional vector sequence to a transformer configured to output the URL embedding, the embedding of the P-th URL output by embedder module 134 representing the semantics of the P-th URL.

[0033] The classifier module 136 receives the embedding of the P-th URL output by the embedder module 134 and calculates a score based on the embedding, where the score indicates the probability that the search results for the first webpage 110 will be selected by any user of the search engine if the search results include SERPs returned to any user. In other words, when receiving the output of the embedder module 134, the classifier module 136 determines whether the semantics of the P-th URL are closer to: 1) the semantics of a URL whose webpage has been selected by a user of the search engine; or 2) the semantics of a URL whose webpage has not been selected by a user of the search engine.

[0034] As will be described in more detail below, classifier module 136 is trained based on a number of positive and negative training items, wherein positive training items include URLs of web pages that have been selected by the user (as noted in the search engine logs of the search engine), and negative training items include URLs of web pages that have not yet been selected by the user by the search engine. When classifier module 136 is trained, lexer module 132 receives URLs from the positive and negative training items and outputs identifiers based on lexical units extracted from the URLs. Embedder module receives the identifiers as described above and outputs embeddings of the URLs, and classifier module 136 is trained based on vectors of such values.

[0035] The classifier module 136 outputs a score, and the crawler module 138 crawls the first webpage 110 when the score exceeds a predetermined threshold. Therefore, when the score indicates that the probability of any user of the search engine selecting the first webpage 110 is relatively high if it is represented in the ranking list of search results output by the search engine, the crawler module 138 retrieves the first webpage 110 from the first web server 102, extracts information from it, and updates the search engine index 124 to include an entry for the first webpage 110, where the entry includes the information extracted from the first webpage 110. In another example, instead of crawling the first webpage 110 when the score output by the classifier module 136 is higher than the threshold, the crawler module 138 crawls the first webpage 110 when the score output by the classifier module 136 for the P-th URL is within a certain maximum threshold number of the scores output by the classifier module 136 for URLs in the URL list 122. After the search engine index 124 is updated to include the entry for the first webpage 110, the search engine can receive a query from the user-operated client computing device, and the search engine can return search results based on the updated index 124 (e.g., the search engine returns search results representing the first webpage 110).

[0036] Although classifier module 136 has been described as being configured to calculate a score indicating the likelihood that any user would select the first webpage 110 when included in search results, it should be understood that classifier module 136 may be adapted to output a score indicating other possibilities for the first webpage 110. In another embodiment, based on the embedding of the Pth URL 130 output by embedder module 134, classifier module 136 is configured to output a score indicating the likelihood that the first webpage 110 includes content related to a specific topic. In such an embodiment, classifier module 136 is trained based on training items including the URL of the webpage and the topics(s) assigned to the webpage. In yet another embodiment, based on the embedding of the Pth URL output by embedder module 134, classifier module 136 is configured to output a score indicating the likelihood that the first webpage 110 will cause the client computing device to install malware on it when the client computing device retrieves the first webpage 110. In such an embodiment, classifier module 136 is trained based on training items including the URL of the webpage and indications about whether the webpage is associated with malware.

[0037] In another embodiment, based on the embedding of the Pth URL output by the embedder module 134, the classifier module 136 is configured to output a score indicating the likelihood that the first webpage 110 includes a predefined number of outbound links (and thus would be a good "seed" webpage for initiating searches for other webpages). In such an embodiment, the classifier module 136 is trained based on training items including the URL of the webpage and the number of outbound links in the webpage. In another embodiment, based on the embedding of the Pth URL output by the embedder module 134, the classifier module 136 is configured to output a score indicating the likelihood that the content of the first webpage 110 has changed over a period of time (e.g., the likelihood that the content of the first webpage 110 has changed within the last five days). In this embodiment, the classifier module 136 is trained based on training items including the URL of the webpage and the amount of time since the webpage was changed. In each of these embodiments, the training items are readily available in the search logs of the search engine, where a relatively large number of training items (e.g., approximately millions or billions of training items) can be used to train the classifier module 136.

[0038] The above-described technique offers various advantages compared to conventional methods used to determine whether to include an entry in a search engine index of a newly discovered webpage. Because the computing system 106 is configured to consider the semantics of the URL, it can determine whether a search engine user is likely to select the webpage represented by the URL based on such semantics. Therefore, the computing system 106 does not need to use arbitrary metrics (such as the number of identified inbound links, whether the webpage belongs to a specific website, etc.) to decide whether to index the webpage. For example, when the computing system 106 determines that the Pth URL 130 includes semantic information indicating that the first webpage 110 represented by the Pth URL 130 belongs to a product, and that historically users have selected webpages containing information about similar products in search results, the computing system 106 chooses to index the first webpage 110 because a search engine user is also likely to select the (newly discovered) first webpage 110.

[0039] Now for reference Figure 2 A diagram illustrating the operation of the word morphist module 132 is provided. In one example, the Pth URL 130 is... http: / / www.example.com / prdcttype / newitm / product.htmlContinuing this example, the lexer module 132 decomposes the URL into a sequence of 11 separate lexical units: “http: / / www.”, “example”, “.com / ”, “prdct”, “type”, “ / ”, “new”, “itm”, “ / ”, “product”, and “.html”. As mentioned earlier, each of these 11 lexical units can be one of a predefined number (e.g., 150,000, 250,000, 500,000, etc.) of lexical units, and the lexer module 132 is configured to identify these lexical units. The lexer module 132 maps the lexical units extracted from the Pth URL 130 to the corresponding lexical unit identifiers. Figure 2 In the example shown, the lexer module 132 maps 11 lexes to the identifier sequence “5”, “25”, “676”, “2567”, “10876”, “356”, “12567”, “165789”, “356”, “25674” and “213584”.

[0040] In addition, Figure 2 In the example shown, lexer module 132 forms a bigram based on lexical units (or lexical identifiers), where the bigram is "http: / / www.example", "example.com / ", ".com / prdct", "prdcttype", "type / ", " / new", "newitm", "itm / ", " / product", "product.html", and ".html-null". Lexer module 132 then converts the bigram into corresponding bigram identifiers. In one example, lexer module 132 hashes each bigram into one of a predefined number (e.g., 150 million) of libraries, where each library corresponds to a bigram identifier. Figure 2 In this process, the bigram "http: / / www.example" is converted to bigram identifier 250623, the bigram "example.com" is converted to bigram identifier 1345832, and so on. Therefore, the output of the lexer module 132 is a series of identifiers including a sequence of lexer identifiers and a sequence of bigram identifiers. Thus, the Pth URL is represented by multiple identifiers.

[0041] Now for reference Figure 3The operation of the embedder module 134 is illustrated. The embedder module 134 receives multiple identifiers output by the lexer module 132, wherein the multiple identifiers include lexical identifiers and bigram identifiers. In one example, the embedder module 134 receives a sequence of lexical identifiers separately from the sequence of bigram identifiers. In another example, the embedder module 134 includes a first word embedding algorithm and a second word embedding algorithm, wherein the first word embedding algorithm receives the sequence of lexical identifiers, and the second word embedding algorithm receives the sequence of bigram identifiers. Other arrangements are also considered.

[0042] Embedder module 134 performs word embedding on the received identifier, and Figure 3 In the example shown, a two-dimensional vector is output for each identifier in the identifier sequence. In one embodiment, the embedder module 134 forms a two-dimensional vector for the identifier (lexical identifier or bigram identifier) ​​based solely on the identifier. In another example, the embedder module 134 generates a two-dimensional vector for the identifier based on the identifier and at least one other identifier in the identifier sequence preceding and / or following the identifier. Furthermore, in one example, the embedder module 134 employs an embedding table to generate word embeddings for the identifier. The values ​​of the two-dimensional vectors output by the embedder module 134 for the identifier represent semantic information about the identifier, and therefore semantic information about the lexical and / or bigram representations of the identifier. Thus, when the distance between two two-dimensional vectors is relatively small, the semantics of the two lexical or bigram representations of the two vectors are similar.

[0043] Embedder module 134 generates the embedding of the Pth URL 130 based on a two-dimensional vector sequence, as described above. In this example, the embedding of the Pth URL is also a two-dimensional vector, although it can be understood that the embedding of the Pth URL is a v-dimensional vector, where v is any suitable value. The embedding of the Pth URL is provided as input to classifier module 136, and classifier module 136 outputs a score for the Pth URL 130 (and therefore a score for the first webpage 110), where the score indicates the probability that any user of the search engine would select the first webpage 110 when the first webpage is represented in the search results shown on the SERP returned by the search engine.

[0044] Now for reference Figure 4The diagram illustrates a functional block diagram of a computing system 400 configured to learn a lexicalizer module 132. The computing system 400 has a computer-readable data repository 402 including a URL list 404. The URL list may include URLs of web pages known to a search engine. Therefore, URLs may include URLs with corresponding entries in a search engine index 124, as well as URLs of web pages known to the search engine but not with corresponding entries in the search engine index 124. The computing system 400 also includes a processor 406 and a memory 408 storing data accessed by the processor 406 and instructions executed by the processor 406. The memory 408 has a sampler module 410 loaded therein, which selects a number of URLs from the URL list 404 in the computer-readable data repository 402. For example, the sampler module 410 randomly samples URLs from the URL list 404, so the URLs selected by the sampler module 410 may include URLs with corresponding entries in the search engine index 124, URLs of web pages already selected by one or more users of the search engine, URLs without corresponding entries in the search engine index 124, etc. In another example, sampler module 410 samples from a list of URLs.

[0045] In the 404 error, a weighted method is used during sampling, making sampler module 410 more efficient than the sampler module in the search engine index.

[0046] The URL of a webpage that does not have a corresponding entry in 124 is more likely to be selected by the user of the search engine.

[0047] The memory 408 also includes a learner module 412 that learns the identities of lexical units in URLs sampled from the URL list 404 by the sampler module 410, thereby learning the lexer module 132. The learner module 412 can learn the lexer module 132 based on statistical analysis of alphanumeric sequences in the sampled URLs, thereby identifying frequently occurring alphanumeric sequences in the sampled URLs as lexical units. The learner module 412 is configured to identify a predefined number of lexical units in the URLs. As previously described, in one example, the learner module 412 is configured to identify 250,000 lexical units, such that the vocabulary of the lexer module 132 is 250,000.

[0048] Now for reference Figure 5The diagram shows a functional block diagram of a computing system 500 configured to train classifier module 136. Although system 500 is shown separately from system 400, it is understood that system 400 can be configured to train classifier module 136. Computing system 500 includes a computer-readable data repository 502, which stores a database 506 containing URLs of web pages known to search engines and the number of times these web pages have been selected by a user if and when they are presented as search results for SVPs generated by the search engine. Since many URLs have no corresponding entries in search engine index 124, the number of selections for the web pages corresponding to these URLs is zero. Furthermore, in database 506, the number of selections for web pages corresponding to URLs can be within a predefined time window, such as the last two months.

[0049] The computing system 500 includes a processor 508 and a memory 510, which includes data accessed by the processor 508 and instructions executed by the processor 508. The memory 510 includes a sampler module 410 that samples URLs from a database 506. In one example, the sampler module 410 initially samples from URLs representing web pages with choices from the database 506, thereby obtaining a set of positive training items. The sampler module 410 then samples from URLs representing web pages without any choices from the database 506, thereby obtaining a set of negative training items. In one embodiment, when obtaining positive training items, the sampler module 410 considers multiple choices when sampling from the database 506, making it more likely that the sampler module 410 will sample URLs associated with a higher number of choices than those associated with a lower number of choices. In another example, the sampler module 410 samples uniformly from URLs with corresponding choices, regardless of the number of choices associated with those URLs. According to one example, the sampler module 410 samples hundreds of millions to billions of training items (e.g., URLs and indications about whether the webpage corresponding to the URL has been clicked) from the database 506.

[0050] The memory 510 also includes a trainer module 512, which trains the classifier module 136 based on training items sampled from the database 506 by the sampler module 410. More specifically, each URL sampled from the database 506 by the sampler module 410 is tokenized by the tokenizer module 132, and the embedder module 134 outputs the URL embedding based on the tokens generated by the tokenizer module 132. Thus, each sampled URL from the database 506 is converted into a sequence of identifiers by the tokenizer module 132, and the embedder module 134 generates the URL embedding based on the sequence of identifiers. The trainer module 512 receives for each sampled URL the URL embedding representing the semantics of the URL and a tag indicating whether the webpage pointed to by the URL has been selected by one or more users of the search engine. Optionally, for each sampled URL, the trainer module 512 receives an indication of how many times the webpage pointed to by that URL was selected by one or more users of the search engine, such that when the trainer module 512 trains the classifier module 136, the semantics of URLs associated with a large number of selections are weighted more heavily than the semantics of URLs associated with a smaller number of selections.

[0051] The trainer module 512, in conjunction with the training classifier module 136, uses any suitable supervised learning method; in this example, the classifier module 136 acts as a computer-implemented binary classifier. For example, the trainer module 512 uses backpropagation when training the classifier module 136. However, it should be understood that the classifier module 136 can be a multi-class classifier when trained on different features (e.g., topics). In one example, when the classifier module 136 is trained to identify topics in webpage content, it can be trained to generate outputs indicating the probability that the webpage includes content related to any one of ten different topics.

[0052] While database 506 shows an option to include URLs and corresponding numbers, it may also include URLs and one or more topics assigned to the web pages pointed to by those URLs, and may include indicators of whether the web pages are associated with malware, may include indicators of the number of inbound and / or outbound links in the web pages, may include indicators of how often the web pages are included in the top X search results by search engines, may include values ​​indicating when the web pages were last modified, and so on. This information is typically found in the search engine's search logs and can be used to train classifier module 136 based on the information expected to be inferred about the web pages.

[0053] According to one example, classifier module 136 is trained periodically (e.g., every three days) and based on relatively recent (e.g., within the last two months) data from database 506. This allows classifier module 136 to adapt to changes in human behavior. For example, when a newsworthy event such as an unexpected earthquake occurs, search engine users may select web pages containing content about the earthquake; however, two weeks before the earthquake, most users may not be interested in earthquake-related content. By periodically retraining classifier module 136 based on recent data, the score output by classifier module 136 reflects the recent behavior of search engine users.

[0054] Figure 6 and Figure 7 Methods relating to inferring information about a webpage from its URL are illustrated. While these methods are shown and described as a series of actions performed sequentially, it should be understood and recognized that these methods are not restricted by order. For example, some actions may occur in a different order than described herein. Furthermore, one action may occur concurrently with another. Moreover, in some cases, not all actions may be required to implement the methods described herein.

[0055] Furthermore, the actions described herein can be computer-executable instructions that can be implemented by one or more processors and / or stored on one or more computer-readable media. Computer-executable instructions can include routines, subroutines, programs, threads of execution, etc. Moreover, the results of the method's actions can be stored in a computer-readable medium, displayed on a display device, etc.

[0056] For reference only. Figure 6 The diagram illustrates a flowchart of the method for training the classifier module 136. Method 600 begins at 602, and at 604, a lexer designed to lexically represent URLs is learned based on URLs sampled from the search logs of a search engine. At 606, URLs are sampled from the search logs to create two sets of training items: a first set of training items comprising the URLs of web pages clicked when included in the SERP presented to users of the search engine; and a second set of training items comprising the URLs of web pages not clicked by users of the search engine.

[0057] At 608, the lexer learned at 604 is used to lexerize URLs in the first training item set and URLs in the second training item set to form the first identifier set and the second identifier set.

[0058] At 610, a first set of URL embeddings is created based on a first set of identifiers, and a second set of URL embeddings is created based on a second set of identifiers. As previously mentioned, these URL embeddings represent the semantics of the URLs. At 612, classifier module 136 is trained based on the first and second sets of URL embeddings. Method 600 is completed at 614.

[0059] Now for reference Figure 7 The diagram illustrates a flowchart of method 700 for updating a search engine index. Method 700 begins at 702 and receives the URL of a webpage at 704. In one example, there is no entry for this webpage in the search engine's index. At 706, the URL is tokenized to generate a token sequence. At 708, an embedding of the URL is generated based on the token sequence. At 710, a score for the URL is calculated based on the embedding generated at 708. At 712, it is determined whether the score is greater than a threshold. If the score is greater than the threshold, at 714, the search engine index is updated to include an entry for the webpage. For example, a crawler retrieves the webpage, extracts information from the webpage, and updates the search engine index based on the information extracted from the webpage. Method 700 ends at 716 when it is determined at 712 that the score is not greater than the threshold or after updating the search engine index at 714.

[0060] Now for reference Figure 8 This illustration shows a high-level diagram of an example computing device 800 that can be used according to the systems and methods disclosed herein. For example, computing device 800 can be used in a system to infer information about a webpage based on the semantics of its URL. As another example, computing device 800 can be used in a system that trains a classifier module to output a score indicating whether a webpage has a specific feature. Computing device 800 includes at least one processor 802 that executes instructions stored in memory 804. For example, the instructions may be instructions for implementing functions described as being performed by one or more of the aforementioned components, or instructions for implementing one or more of the aforementioned methods. Processor 802 can access memory 804 via system bus 806. In addition to storing executable instructions, memory 804 may also store URLs, the number of URL selections, URL terms, etc.

[0061] The computing device 800 also includes a data repository 808 accessible by the processor 802 via the system bus 806. The data repository 808 may include executable instructions, search engine logs, lexical terms, etc. The computing device 800 also includes an input interface 810 that allows external devices to communicate with it. For example, the input interface 810 can be used to receive instructions from external computer devices, users, etc. The computing device 800 also includes an output interface 812 that interfaces the computing device 800 with one or more external devices. For example, the computing device 800 can display text, images, etc., through the output interface 812.

[0062] It is conceivable that external devices communicating with computing device 800 via input interface 810 and output interface 812 can be included in an environment that provides a user interface of virtually any type to which the user can interact. Examples of user interface types include graphical user interfaces, natural user interfaces, and so on. For example, a graphical user interface can accept input from a user using multiple input devices such as a keyboard, mouse, remote control, etc., and provide output on an output device such as a display. Furthermore, a natural user interface allows the user to interact with computing device 800 in a manner unrestricted by input devices such as keyboards, mice, remote controls, etc. Instead, a natural user interface can rely on voice identifiers, touch and stylus identifiers, on-screen and near-screen gesture identifiers, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, machine intelligence, and so on.

[0063] Furthermore, although shown as a single system, it is understood that computing device 800 can be a distributed system. Therefore, for example, several devices can communicate via a network connection and collaboratively perform tasks described as being performed by computing device 800.

[0064] The various functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, these functions can be stored or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media. A computer-readable storage medium can be any available storage medium accessible to a computer. By way of example and not limitation, such a computer-readable storage medium can be or includes RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Disks and optical discs as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs (BDs), where discs typically reproduce data magnetically, while optical discs typically reproduce data optically using lasers. Furthermore, transmitted signals are not included within the scope of computer-readable storage media. Computer-readable media also include communication media, which includes any medium that facilitates the transfer of a computer program from one place to another. For example, a connection can be a communication medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of communication media. Combinations of the above should also be included within the scope of computer-readable media.

[0065] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0066] Based at least on the examples provided below, the features described in this paper involve inferring information about a webpage based on semantic information extracted from the webpage's URL.

[0067] (A1) In one aspect, some embodiments include a method performed by at least one processor (e.g., 126) of a computing system (e.g., 106), wherein the method includes actions including: retrieving the URL (e.g., 130) of a webpage (e.g., 110) from a list of URLs of webpages (e.g., 122), wherein the webpage is included in the World Wide Web; creating a value vector based on the URL representing semantics present in alphanumeric characters of the URL; inferring information about the webpage based on the vector; retrieving the webpage from a computing device hosting the webpage based on the inferred information about the webpage; and after retrieving the webpage, extracting content from the webpage and storing the extracted content in a computer-readable repository.

[0068] (A2) In some embodiments of the method of A1, the information is the probability that a webpage will be selected by any user of the search engine when it is included in a SERP provided by the search engine to any user.

[0069] (A3) In some embodiments of the method in A2, the content extracted from the webpage is included in the search engine index of the search engine.

[0070] (A4) In some embodiments of any of the methods in A1, the information is the possibility that the webpage includes content belonging to a predefined topic.

[0071] (A5) In some embodiments of the method in A1, the information is the probability that the content of the webpage has been updated within a threshold time period.

[0072] (A6) In some embodiments of the method in A1, the information is the possibility that the web page content is written in a specific language.

[0073] (A7) In some embodiments of the method in A1, the information is the possibility that the webpage is associated with malware.

[0074] (A8) In some embodiments of any of the methods in A1-A7, creating a vector representing the semantics present in the alphanumeric characters of the URL includes: lexicalizing the URL to extract lexicals from the URL; mapping the extracted lexicals to corresponding identifiers; generating an n-gram from the extracted lexicals, wherein each n-gram includes several lexicals; and using word embeddings and generating an s-dimensional vector of the n-gram based on the n-gram, wherein the s-dimensional vector represents the semantics of the n-gram.

[0075] (A9) In some embodiments of the method in A8, the s-dimensional vector is a two-dimensional vector.

[0076] (B1) In another aspect, some embodiments include a method performed by a computing system (e.g., 106) including a processor (e.g., 126) and a memory (e.g., 128). The method includes lexicalizing a URL (e.g., 130) of a webpage (e.g., 110) to generate lexicals of the URL. The method also includes generating embeddings of the URL based on the generated lexicals, wherein the URL embeddings represent the semantics of the URL. The method further includes calculating a score for the URL based on the URL embeddings, wherein the score indicates the likelihood that a user would select a search result when the search result representing the webpage is included on a SERP generated by a search engine, wherein the search engine index of the search engine is updated based on a determination that the score is greater than a threshold.

[0077] (B2) In some embodiments of the method in B1, the score of the URL is output by a computer-implemented binary classifier.

[0078] (B3) In some embodiments of the method in B2, a computer-implemented binary classifier is trained based on the content of the search engine's search logs, wherein the training data used to train the binary classifier includes URLs in the search engine's search logs and indications about whether the search results corresponding to the URLs were selected by the search engine's users.

[0079] (B4) In some embodiments of any of the methods in B1-B3, the method further includes not including the webpage entry in the search engine's search engine index when the score is less than or equal to a threshold.

[0080] (B5) In some embodiments of any of the methods in B1-B4, the method further includes mapping the generated lexical units to corresponding identifiers, wherein the embedding of the URL is generated based on the identifiers mapped to the generated lexical units.

[0081] (B6) In some embodiments of any of the methods in B1-B5, generating the URL embedding includes generating word embeddings based on lexical units, wherein each word embedding in the word embeddings is a two-dimensional vector.

[0082] (B7) In some embodiments of any of the methods in B1-B6, the method further includes generating an n-gram based on the lexicalization of the URL, wherein the embedding of the URL is generated based on the generated n-gram.

[0083] (B8) In some embodiments of any of the methods in B1-B7, the lexicalization of URLs is performed by a lexicalizer trained on a set of known URLs.

[0084] (B9) In some embodiments of any of the methods in B1-B8, the method further includes: calculating a second score for the URL based on the URL embedding, wherein the second score indicates the likelihood that the webpage includes a threshold number of outbound links.

[0085] (B10) In some embodiments of any of the methods in B1-B9, the method further includes: calculating a second score for the URL based on the URL embedding, wherein the second score indicates the likelihood that the webpage includes content closely related to the topic.

[0086] (C1) In another aspect, some embodiments include a method performed by a computing system (e.g., 106) including a processor (e.g., 126) and a memory (e.g., 128). The method includes: lexicalizing a URL (e.g., 130) of a webpage (e.g., 110) to generate lexicals of the URL; generating an embedding of the URL based on the generated lexicals, wherein the embedding of the URL represents the semantics of the URL; calculating a score for the URL based on the embedding of the URL, wherein the score indicates the likelihood that a user would select the search result when the search result representing the webpage is included on a SERP generated by a search engine; and updating the search engine index of the search engine to include entries for the webpage if the score is greater than a threshold.

[0087] (D1) In another aspect, some embodiments include a computing system (e.g., 106) that includes a processor (e.g., 126) and a memory (e.g., 128), wherein the memory includes instructions that, when executed by the processor, cause the processor to perform any of the methods described herein (e.g., any of A1-A9, B1-B10 and / or C1).

[0088] (E1) In yet another aspect, the computer-readable storage medium includes instructions that, when executed by a processor (e.g., 126) of a computing system (e.g., 106), cause the processor to perform any of the methods described herein (e.g., any one of A1-A9, B1-B10 and / or C1).

[0089] The term “or” signifies an inclusive “or”, not an exclusive “or.” That is, unless otherwise stated or clearly apparent from the context, the phrase “X adopts A or B” means any natural inclusive permutation. Specifically, the phrase “X adopts A or B” is satisfied by any of the following instances: X adopts A; X adopts B; or X adopts both A and B. Furthermore, the articles “a” and “an” used in this application and the appended claims should generally be interpreted as meaning “one or more”, unless otherwise stated or clearly apparent from the context to the singular form.

[0090] Furthermore, as used herein, the terms “component,” “module,” and “system” are intended to include computer-readable data storage configured with computer-executable instructions that, when executed by a processor, cause certain functions to be performed. Computer-executable instructions may include routines, functions, etc. It should also be understood that a component or system may reside on a single device or be distributed across multiple devices.

[0091] The foregoing includes examples of one or more embodiments. Of course, in describing the foregoing aspects, it is impossible to describe every conceivable modification and alteration of the described apparatus or method; however, those skilled in the art will recognize that many further modifications and arrangements of various aspects are possible. Therefore, the described aspects are intended to encompass all such changes, modifications, and variations falling within the spirit and scope of the appended claims. Furthermore, with regard to the extent to which the term "comprising" is used in the detailed specification or claims, such a term is intended to be inclusive, in a manner similar to the term "comprising," as it is interpreted when "comprising" is interpreted as a transitional word in the claims.

Claims

1. A computing system, comprising: processor; as well as A memory storing instructions that, when executed by the processor, cause the processor to perform actions, including: Lexicalization is performed on the Uniform Resource Locator (URL) of the webpage to generate the lexicals of the URL; Based on the generated lexical units, an embedding for the URL is generated, wherein the embedding for the URL represents the semantics of the URL, and generating the embedding for the URL includes: Map the lexical to the corresponding lexical identifier for the lexical; Based on the lexical units, an n-gram is formed; Map the n-gram to an n-gram identifier; and Generate a first embedding for the lexical identifier and a second embedding for the n-gram identifier, wherein the embedding for the URL is based on the first embedding and the second embedding; and Based on the embedding for the URL, a score is calculated for the URL, wherein the score indicates the likelihood that a user will select the search result when the search result representing the webpage is included on a search engine results page (SERP) generated by the search engine, wherein the search engine index for the search engine is updated based on determining that the score is greater than a threshold.

2. The computing system of claim 1, wherein the score for the URL is output by a computer-implemented binary classifier.

3. The computing system of claim 2, wherein the computer-implemented binary classifier is trained based on the content of the search logs of a search engine, wherein the training data for training the binary classifier includes URLs in the search logs of the search engine and indications regarding whether search results corresponding to the URLs were selected by the user of the search engine.

4. The computing system according to claim 1, further comprising: When the score is less than or equal to the threshold, the entry for the webpage will not be included in the search engine index for the search engine.

5. In the computing system of claim 1, the lexicalization of the URL is performed by a lexicalizer trained based on a known set of URLs.

6. The computing system according to claim 1, further comprising: Based on the embedding for the URL, a second score is calculated for the URL, wherein the second score indicates the likelihood that the webpage includes a threshold number of outbound links.

7. The computing system according to claim 1, further comprising: Based on the embedding for the URL, a second score is calculated for the URL, wherein the second score indicates the likelihood that the webpage includes content closely related to the topic.

8. The calculation system of claim 1, wherein the score for the webpage is calculated without retrieving the webpage.

9. A method executed by at least one processor of a computing system, the method comprising: Retrieve the URLs of the web pages from a list of Uniform Resource Locator URLs for multiple web pages, where the web pages are included in the World Wide Web; The URL of the webpage is lexicalized to generate the lexical units of the URL; Based on the URL, a vector representing the semantics present in the alphanumeric characters of the URL is created, wherein the vector for creating the values ​​includes: Map the lexical to the corresponding lexical identifier for the lexical; Based on the lexical units, an n-gram is formed; Map the n-gram to an n-gram identifier; as well as Generate a first embedding for the lexical identifier and a second embedding for the n-gram identifier, wherein the vector of values ​​is based on the first embedding and the second embedding; and Information about the webpage is inferred based on the vector; Based on the information inferred about the webpage, the webpage is retrieved from the computing device hosting the webpage; and After retrieving the webpage, extract the content from the webpage and store the extracted content in a computer-readable repository.

10. The method of claim 9, wherein the extracted content is included in the search engine index of the search engine.

11. The method of claim 9, wherein the information includes the possibility that the webpage includes content belonging to a predefined topic.

12. The method of claim 9, wherein the information includes the probability that the content of the webpage will be updated within a threshold time period.

13. The method of claim 9, wherein the information includes the possibility that the content of the webpage is written in a particular language.

14. The method of claim 9, wherein the information includes the possibility that the webpage is associated with malware.

15. The method of claim 9, wherein the information includes the probability that the webpage will be selected by the arbitrary user of the search engine when the webpage is included in a search engine results page (SERP) provided by the search engine to any user.

16. A computer-readable storage medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform an action, the action comprising: Lexicalization is performed on the Uniform Resource Locator (URL) of the webpage to generate the lexicals of the URL; Based on the generated lexical units, an embedding for the URL is generated, wherein the embedding for the URL represents the semantics of the URL, and generating the embedding for the URL includes: Map the lexical to the corresponding lexical identifier for the lexical; Based on the lexical units, an n-gram is formed; Map the n-gram to an n-gram identifier; as well as Generate a first embedding for the lexical identifier and a second embedding for the n-gram identifier, wherein the embedding for the URL is based on the first embedding and the second embedding; and Based on the embedding for the URL, a score is calculated for the URL, wherein the score indicates the likelihood that a user will select the search result when the search result representing the webpage is included on a search engine results page (SERP) generated by the search engine. Based on the score being greater than the threshold, the search engine index for the search engine is updated to include entries for the webpage.

17. The computer-readable storage medium of claim 16, further comprising: Based on the embedding for the URL, a second score is calculated for the URL, wherein the second score indicates the likelihood that the webpage includes content belonging to a predefined topic, and wherein the search engine index is updated based on the second score for the URL.

18. The computer-readable storage medium of claim 16, further comprising: Based on the embedding for the URL, a second score is calculated for the URL, wherein the second score indicates the probability that the content of the webpage will be updated within a threshold time period, and wherein the search engine index is updated based on the second score for the URL.

19. The computer-readable storage medium of claim 16, further comprising: Based on the embedding for the URL, a second score is calculated for the URL, wherein the second score indicates the likelihood that the content of the webpage is written in a specific language, and wherein the search engine index is updated based on the second score for the URL.

20. The computer-readable storage medium of claim 16, wherein the score for the webpage is calculated without retrieving the webpage.

Images