System and method for segmenting client sessions of a website using web scripts

Abstract

The present invention relates to a system and method for segmenting client sessions on a website using web scripts. The system may include a visitor management module for identifying client sessions established on the website. The visitor management module may determine node pairs including a destination node and a reference node. The system may include a graph construction module for constructing a graph including node pairs and edges between the node pairs representing click-through rates. The system may include a graph clustering module for assigning information resources to a first group based on one of i) the number of edges between the nodes and conversion nodes or ii) the click-through rates of the node pairs. The system may include a remarketing list generation module for assigning the first group of information resources to a remarketing list. The remarketing list generation module may assign identifiers of clients that accessed the first group of information resources to the remarketing list.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201680034574.8, filed on October 6, 2016.

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application claims the benefit of U.S. Patent Application No. 14 / 984,341, filed on December 30, 2015, and entitled “Systems and Methods for Automatically Generating Remarketing Lists,” the contents of which are incorporated herein by reference in their entirety. Background Art

[0005] In a computer networking environment such as the Internet, third-party content providers provide third-party content items for display on end-user computing devices. These third-party content items, such as advertisements, may be displayed on web pages associated with corresponding publishers. These third-party content items may include content identifying the third-party content provider providing the content item. The third-party content item may direct users to the third-party content provider's website. Third-party content providers may be interested in understanding the navigation behavior of users who visit their websites. Summary of the Invention

[0006] At least one aspect relates to a system for segmenting client sessions of a website using web scripts. The system may include a visitor management module, executing on a data processing system and configured to identify a plurality of client sessions established at the website, each client session corresponding to a client device accessing one or more information resources of the website. The visitor management module may also be configured to determine, for each of the plurality of client sessions, one or more node pairs, each of the one or more node pairs including a destination node representing a destination information resource and a reference node representing a reference information resource corresponding to the information resource from which the client device accessed the destination information resource. The system may also include a graph construction module, executing on the data processing system and configured to construct a graph using the plurality of client sessions, the graph including the determined node pairs and a plurality of edges between the destination nodes of the node pairs and corresponding reference nodes, each of the plurality of edges between the destination nodes and the reference nodes of the corresponding node pairs representing a ratio of a first number of client devices accessing the destination information resource from the reference information resource to a second number of client devices accessing the reference information resource. The system may also include a graph clustering module, executing on the data processing system and configured to assign one or more of the plurality of information resources to a first set of information resources based on one of: i) a number of edges between nodes corresponding to the first set of information resources and a conversion node representing an information resource presented in response to a conversion event, or ii) a click-through rate for node pairs including the same reference node and different destination nodes. The system may also include a remarketing list generation module, executing on the data processing system and configured to assign the first set of information resources to a remarketing list, the remarketing list including client identifiers of client devices that accessed information resources assigned to the first set of information resources. The remarketing list generation module may also be configured to assign the client identifiers that accessed the first set of information resources to the remarketing list. The remarketing list generation module may also be configured to use the remarketing list to select content items for display in response to receiving a content request from one of the client identifiers that accessed the first set of information resources.

[0007] In some embodiments, the visitor management module may receive one or more of an identification of a destination information resource, an identification of a referencing information resource, or a number of times a client device has accessed the destination resource and the referencing information resource.

[0008] In some implementations, the graph construction module may omit from the graph at least one edge of the plurality of edges and a node pair associated with the at least one edge that has a ratio below a ratio threshold.

[0009] In some embodiments, the visitor management module may identify the transition node based on the number of reference information resource identifiers linked to the transition node. In some embodiments, the transition node corresponds to a node among a plurality of nodes in the graph, the node being linked to the most reference nodes relative to the number of reference nodes linked to each of the other nodes in the plurality of nodes.

[0010] In some implementations, the graph clustering module is executable on the data processing system to assign each of the nodes corresponding to the first group of information resources having the same number of edges between each of the nodes and the transition node to the first group.

[0011] In some implementations, the graph clustering module may determine a click-through rate threshold, and the graph clustering module may assign one or more information resources associated with edges that link to the same referencing node as an edge having a click-through rate greater than the click-through rate threshold to the first group.

[0012] In some implementations, the graph clustering module may cluster the one or more information resources based on proximity of one or more nodes corresponding to the one or more information resources in the graph and based on weights of click-through rates between the one or more nodes.

[0013] In some implementations, assigning one or more of the plurality of information resources to the first group is based on a predetermined hierarchical model of pages of a website based on click-through rates of node pairs including the same referencing node and different destination nodes.

[0014] In some embodiments, assigning one or more of the plurality of information resources to the first group based on click-through rates of node pairs including the same reference node and different destination nodes is based on a predetermined hierarchical model of pages of a website. In some embodiments, constructing the graph is based on a predetermined hierarchical model of pages of a website.

[0015] At least one aspect relates to a system for segmenting client sessions of a website using web scripts. The method includes identifying, by a data processing system including one or more processors, a plurality of client sessions established at the website, each client session corresponding to a client device accessing one or more information resources of the website. The method includes determining, by the data processing system, one or more node pairs for each of the plurality of client sessions, each of the one or more node pairs including a destination node representing a destination information resource and a reference node representing a reference information resource corresponding to an information resource from which the client device accessed the destination information resource. The method includes constructing, by the data processing system, a graph using the plurality of client sessions, the graph including the determined node pairs and a plurality of edges between the destination nodes of the node pairs and the corresponding reference nodes, each of the plurality of edges between the destination nodes and the reference nodes of the corresponding node pairs representing a ratio of a first number of client devices accessing the destination information resource from the reference information resource to a second number of client devices accessing the reference information resource. The method includes assigning one or more of the plurality of information resources to a first set of information resources based on one of: i) a number of edges between nodes corresponding to the first set of information resources and a transition node representing an information resource presented in response to a transition event or ii) a click-through rate for node pairs including the same reference node and different destination nodes. The method includes assigning the first set of information resources to a remarketing list, the remarketing list including client identifiers of client devices that accessed information resources assigned to the first set of information resources. The method includes assigning the client identifiers that accessed the first set of information resources to the remarketing list. The method includes selecting a content item for display using the remarketing list in response to receiving a content request from one of the client identifiers that accessed the first set of information resources.

[0016] In some embodiments, the method further includes receiving, by the data processing system, one or more of an identification of a destination information resource, an identification of a reference information resource, or a number of times a client device has accessed the destination resource and the reference information resource.

[0017] In some implementations, the method further includes omitting from the graph at least one edge of the plurality of edges and node pairs associated with the at least one edge that have a ratio below a ratio threshold.

[0018] In some embodiments, the method further includes identifying, by the data processing system, the transition node based on a number of reference resource identifiers linked to the transition node. In some embodiments, the transition node corresponds to a node among a plurality of nodes in the graph that has the greatest number of reference nodes linked to it relative to a number of reference nodes linked to each of the other nodes in the plurality of nodes.

[0019] In some embodiments, the method further includes assigning, by the data processing system, each of the nodes corresponding to the information resources of the first group having the same number of edges between each of the nodes and the transition node to the first group.

[0020] In some embodiments, the method further includes determining, by the data processing system, a click-through rate threshold, and assigning, by the data processing system, to the first group, one or more information resources associated with edges linked to edges having the same reference node as an edge having a click-through rate greater than the click-through rate threshold.

[0021] In some embodiments, the method further includes clustering, by the data processing system, the one or more information resources based on proximity of one or more nodes corresponding to the one or more information resources to each other in the graph and based on weights of click-through rates between the one or more nodes.

[0022] In some embodiments, assigning one or more of the plurality of information resources to the first group based on click-through rates of node pairs including the same reference node and different destination nodes is based on a predetermined hierarchical model of pages of a website. In some embodiments, constructing the graph is based on a predetermined hierarchical model of pages of a website.

[0023] At least one aspect relates to a system for segmenting client sessions of a website using a website script based on conversion metrics of information resources of the website. The system may include an information resource management module, executing on a data processing system and configured to identify, for a website corresponding to an advertiser, a plurality of information resources included in the website, at least one of a plurality of information resources corresponding to a landing page of a content item of the advertiser provided for display as a third-party content item, at least one of a plurality of information resources identified as a conversion information resource presented in response to a conversion event, and at least one information resource of the website including an online activity reporting script for determining a conversion rate for the at least one information resource. The system may also include a visitor management module, executing on the data processing system and configured to store, via the online activity reporting script, a plurality of entries corresponding to information resource accesses by client devices for each of the at least one information resource of the website including the online activity reporting script in a data structure, each of the plurality of entries identifying (i) a resource identifier identifying the information resource and (ii) a client identifier identifying a client device accessing the information resource. The system may include a conversion rate determination module executing on the data processing system and configured to determine, for the information resource, a conversion ratio based on a first number of client identifiers that accessed the information resource and also accessed the conversion information resource relative to a second number of client identifiers that accessed the information resource. The system may also include a remarketing list generation module executing on the data processing system and configured to assign, based on the determined conversion rate for the first information resource, a first information resource of at least one information resource of the website including the online activity reporting script to a first remarketing list, the first remarketing list including client identifiers of client devices that accessed the first information resource. The remarketing list generation module may also be configured to use the remarketing list to select content items for display in response to receiving a content request from one of the client identifiers that accessed the first information resource.

[0024] In some implementations, the conversion rate determination module may determine at least one conversion rate threshold, and the remarketing list generation module may assign the first information resource to the first remarketing list based on a conversion rate of the first information resource relative to the at least one conversion rate threshold.

[0025] In some implementations, the conversion rate determination module may assign a second information resource from the at least one information resource of the website including the online activity reporting script to a second remarketing list based on the determined conversion rate of the second information resource relative to at least one conversion rate threshold, the second remarketing list being different from the first remarketing list and including a client identifier of a client device accessing the second information resource.

[0026] In some embodiments, the conversion rate of the first information resource is greater than the at least one conversion rate threshold, and the conversion rate of the second information resource is less than the at least one conversion rate threshold. In some embodiments, the conversion rate determination module may calculate an intermediate conversion rate based on the conversion rate of each of the at least one information resource of the website including the online activity reporting script, and the conversion rate determination module may assign at least one conversion rate threshold as a multiple of the intermediate conversion rate.

[0027] In some implementations, the conversion rate determination module may calculate an average conversion rate based on the conversion rate of each of the at least one information resource of the website including the online activity reporting script, and the conversion rate determination module may assign the at least one conversion rate threshold based on a multiple of a standard deviation of the average conversion rate.

[0028] In some implementations, the conversion rate determination module may assign one or more information resources having a range of conversion rates bounded by two of at least one conversion rate threshold value to the first remarketing list.

[0029] In some embodiments, the conversion rate determination module may determine the conversion rate for each of at least one information resource of the website including the online activity reporting script based on a time limit of conversions between when the client identifier accesses the information resource and when the client identifier accesses the conversion information resource.

[0030] In some embodiments, the information resource management module may receive, for at least one information resource of the website including the online activity reporting script, data corresponding to at least one of an identifier of the at least one information resource of the website, an identifier of a client device accessing the at least one information resource of the website, and a number of times the client device accessed the at least one information resource of the website. In some embodiments, the information resource management module may receive, for a conversion information resource of the website, data corresponding to at least one of an identifier of the conversion information resource of the website, an identifier of a client device accessing the conversion information resource of the website, and a number of times the client device accessed the conversion information resource of the website.

[0031] At least one aspect relates to a method for segmenting client sessions of a website using a website script based on conversion metrics of information resources of the website. The method includes, by a data processing system including one or more processors, identifying, for a website corresponding to an advertiser, a plurality of information resources included in the website, at least one of a plurality of information resources corresponding to a landing page of a content item of the advertiser provided for display as a third-party content item, at least one of a plurality of information resources identified as a conversion information resource presented in response to a conversion event, and at least one information resource of the website including an online activity reporting script for determining a conversion rate for the at least one information resource. The method includes, for each of the at least one information resource of the website including the online activity reporting script, storing, by the data processing system, via the online activity reporting script, a plurality of entries corresponding to information resource accesses by client devices in a data structure, each of the plurality of entries identifying (i) a resource identifier identifying the information resource and (ii) a client identifier identifying a client device accessing the information resource. The method includes determining, by the data processing system, for each of at least one information resource of the website including the online activity reporting script, a conversion rate for the information resource based on a first number of client identifiers that accessed the information resource and also accessed the conversion information resource relative to a second number of client identifiers that accessed the information resource. The method includes assigning, by the data processing system, a first information resource of the at least one information resource of the website including the online activity reporting script to a first remarketing list based on the determined conversion rate for the first information resource, the first remarketing list including client identifiers of client devices that accessed the first information resource. The method includes selecting a content item for display using the remarketing list in response to receiving a content request from one of the client identifiers that accessed the first information resource.

[0032] In some implementations, the method further includes determining, by the data processing system, at least one conversion rate threshold, and assigning, by the data processing system, the first information resource to the first remarketing list based on a conversion rate of the first information resource relative to the at least one conversion rate threshold.

[0033] In some embodiments, the method further includes assigning, by the data processing system, a second information resource of the at least one information resource of the website including the online activity reporting script to a second remarketing list based on the determined conversion rate of the second information resource relative to at least one conversion rate threshold, the second remarketing list being different from the first remarketing list and including a client identifier of a client device accessing the second information resource.

[0034] In some embodiments, the conversion rate of the first information resource is greater than the at least one conversion rate threshold, and the conversion rate of the second information resource is less than the at least one conversion rate threshold.

[0035] In some embodiments, the method further includes calculating, by the data processing system, an intermediate conversion rate based on a conversion rate for each of at least one information resource of the website that includes the online activity reporting script, and assigning, by the data processing system, at least one conversion rate threshold that is a multiple of the intermediate conversion rate.

[0036] In some embodiments, the method further includes calculating, by the data processing system, a median conversion rate based on the conversion rate of each of at least one information resource of the website that includes the online activity reporting script, and assigning, by the data processing system, the at least one conversion rate threshold based on a multiple of a standard deviation of the average conversion rate.

[0037] In some implementations, the method further includes assigning, by the data processing system, one or more information resources having a range of conversion rates bounded by two of at least one conversion rate threshold to the first remarketing list.

[0038] In some embodiments, the method further comprises determining, for each of at least one information resource of the website including the online activity reporting script, the conversion rate based on a time limit of conversions between when the client identifier accesses the information resource and when the client identifier accesses the conversion information resource.

[0039] In some embodiments, the method further includes receiving, by the data processing system, data corresponding to at least one of an identification of the at least one information resource of the website, an identification of a client device accessing the at least one information resource of the website, and a number of times the at least one information resource of the website has been accessed by the client device, for at least one information resource of the website that includes the online activity reporting script. In some embodiments, the method further includes receiving, by the data processing system, data corresponding to at least one of an identification of the conversion information resource of the website, an identification of a client device accessing the conversion information resource of the website, and a number of times the conversion information resource of the website has been accessed by the client device, for a conversion information resource of the website.

[0040] These and other aspects and embodiments are discussed in detail below. The above information and the following detailed description include illustrative examples of the various aspects and embodiments and provide an overview or framework for understanding the nature and character of the claimed aspects and embodiments. The accompanying drawings provide illustration and a further understanding of the various aspects and embodiments and are incorporated into and constitute a part of this specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are not intended to be drawn to scale. Like reference numbers and names in the various figures indicate like elements. For clarity, not every component may be labeled in every figure.

[0042] In the attached figure:

[0043] Figure 1 is a block diagram depicting one embodiment of an environment for automatically generating remarketing lists according to an illustrative embodiment.

[0044] Figure 2 is a block diagram depicting one implementation of a campaign management module in accordance with an illustrative embodiment.

[0045] Figure 3 is a diagram depicting a page of a website for automatically generating remarketing lists according to an exemplary embodiment.

[0046] Figure 4 is a flow chart depicting a method of automatically generating remarketing lists according to an exemplary embodiment.

[0047] Figure 5 is a block diagram depicting one implementation of a campaign management module in accordance with an illustrative embodiment.

[0048] Figure 6 is a table depicting a page of a website for automatically generating remarketing lists according to an exemplary embodiment.

[0049] Figure 7 is a flow chart depicting a method of automatically generating remarketing lists according to an exemplary embodiment.

[0050] Figure 8 is a block diagram depicting an exemplary embodiment of the overall architecture of a computer system that can be used to implement the elements of the systems and methods described and illustrated herein. DETAILED DESCRIPTION

[0051] The following is a more detailed description of various concepts related to methods, devices, and systems for automatically generating remarketing lists for websites and their implementation. The various concepts introduced above and discussed in more detail below can be implemented in any of numerous ways, as the described concepts are not limited to any particular implementation.

[0052] Aspects of the present disclosure generally relate to systems and methods for automatically generating remarketing lists for content providers (eg, advertisers) based on segments of web pages of the content provider's website.

[0053] In existing solutions, content providers may first add an online reporting script (e.g., a small snippet of code for an online activity reporting script) to each of the content provider's web pages. The online reporting script may then report information about users who visited the tagged pages (e.g., the user's cookie ID, the page's URL, whether the user converted, etc.). The content provider may then create remarketing lists based on rules defined by the content provider, and the rules may be manually entered into the remarketing tool. For example, a content provider may generate a remarketing list based on customers who added products to their shopping carts but did not proceed with a purchase. For another example, a content provider may generate a remarketing list based on customers who visited a specific category of products (e.g., shoes). Therefore, due to the manual nature of this process, generating remarketing lists can become tedious and time-consuming for content providers. Furthermore, the granularity and effectiveness of the remarketing lists may be limited by the skills of the individual content providers.

[0054] The present disclosure addresses these challenges by segmenting content provider websites using user and page information reported by online reporting scripts and creating high-value remarketing lists based on the segments. The present disclosure can generate remarketing lists based on 1) a mapping of content provider websites and 2) conversion metrics corresponding to individual pages on the content provider's websites.

[0055] In one embodiment of a mapping based on a content provider website, a sequence of URLs visited by all users can be constructed using information reported by an online reporting script on the website page (e.g., source-destination URLs of links or the number of times a user visited each tagged URL). Using this information, click-through rates can also be determined between all tagged URL pairs, and a directed (based on URL links) and weighted (based on link click-through rates) graph of website URLs can be constructed. Using this graph, various URL paths leading to purchase page URLs (indicating user conversions) can be determined. Consequently, corresponding remarketing lists can then be associated with URLs that are the same distance from the conversion URL. The online reporting script can be an online activity reporting script embedded in one or more information resources on the website. The online activity reporting script includes one or more computer-executable instructions that can be configured to execute on client device 125. The computer-executable instructions of the online reporting script can be configured to instruct client device 125 to report its online activity data to content provider 115 or data processing system 110.

[0056] In another embodiment based on mapping content provider websites, a directed and weighted graph of website pages can be used to determine the website's product categories. For example, the graph's links and click-through rates can be analyzed to infer user visit patterns, and URLs with strong links can be clustered as corresponding to a product. The corresponding remarketing lists can then be associated with different clusters of the graph to obtain product-specific remarketing lists.

[0057] At least some aspects of the present disclosure relate to systems and methods for automatically generating remarketing lists based on client sessions. The method includes identifying, by a data processing system including one or more processors, a plurality of client sessions established at a website, each client session corresponding to a client device accessing one or more information resources of the website. The method includes determining, by the data processing system, one or more node pairs for each of the plurality of client sessions, each of the one or more node pairs including a destination node representing a destination information resource and a reference node representing a reference information resource corresponding to an information resource accessed by the client device from the destination information resource. The method includes using the plurality of client sessions to construct, by the data processing system, a graph including the determined node pairs and a plurality of edges between the destination nodes of the node pairs and the corresponding reference nodes. Each of the plurality of edges between the target nodes and the reference nodes of the corresponding node pairs may represent a ratio of a first number of client devices accessing the destination information resource from the reference information resource to a second number of client devices accessing the reference information resource. The method includes assigning one or more information resources from a plurality of information resources to a first set of information resources based on one of: i) a number of edges between nodes corresponding to the first set of information resources and conversion nodes representing information resources presented in response to conversion events or ii) click-through rates for node pairs including the same reference node and different destination nodes. The method includes assigning the first set of information resources to a remarketing list including client identifiers of client devices that accessed the information resources assigned to the first set of information resources.

[0058] In other embodiments, because the system can determine whether a user has converted based on information reported by an online reporting script, the conversion rate of users who visit a particular page can be determined (i.e., the "tag conversion rate" of each page can be determined). Thus, URLs on a website can be segmented or sorted based on their tag conversion rates. Based on a tag conversion rate threshold, each of the URLs can be grouped into a remarketing list. For example, URLs with relatively high tag conversion rates (e.g., having a tag conversion rate higher than a median tag conversion rate threshold for all pages) can belong to one remarketing list or be otherwise associated with one remarketing list, and URLs with relatively low conversion rates (e.g., having a tag conversion rate lower than the median conversion rate threshold) can belong to another remarketing list.

[0059] At least some aspects of the present disclosure relate to systems and methods for automatically generating remarketing lists using conversion metrics for information resources of a content provider's website. The method includes, by a data processing system including one or more processors, identifying, for a website corresponding to a content provider, a plurality of information resources included in the website, at least one of a plurality of information resources corresponding to a landing page for a content item of the content provider provided for display as a third-party content item, at least one of a plurality of information resources identified as a conversion information resource presented in response to a conversion event, and at least one information resource of the website including an online activity reporting script for determining a conversion rate for the at least one information resource. The method includes, for each of the at least one information resource of the website including the online activity reporting script, storing, by the data processing system, via the online activity reporting script, a plurality of entries corresponding to information resource accesses by client devices in a data structure, each of the plurality of entries identifying (i) a resource identifier identifying the information resource and (ii) a client identifier identifying a client device accessing the information resource. The method also includes, for each of the at least one information resource of the website including the online activity reporting script, determining, by the data processing system, a conversion rate for the information resource based on a first number of client identifiers that accessed the information resource and also accessed the conversion information resource relative to a second number of client identifiers that accessed the information resource. The method includes assigning, by a data processing system, a first information resource of at least one information resource of a website including an online activity reporting script to a first remarketing list based on a determined conversion rate of the first information resource, the first remarketing list including a client identifier of a client device accessing the first information resource.

[0060] The present solution aims to solve an internet-centric problem, also rooted in computer technology, with a solution rooted in computer technology. Specifically, the present solution aims to categorize client device identifiers based on certain online activities performed by the client devices associated with the client device identifiers in website client sessions, such as click-through rates between referring nodes and destination nodes. In some embodiments, the internet-centric problem involves categorizing client devices that visit one or more web pages of a content provider's website into one or more different lists. These lists can be used for remarketing, that is, directing additional content items to the client device identifiers based on the list to which the client device identifiers are categorized. The additional content items can be used to direct the client device identifiers to the content provider's website to perform additional online activities that can ultimately lead to conversions. In some embodiments, the present solution can solve the technical problem of data categorization by categorizing pages of a website into various lists, such that client device identifiers (or users of client devices) that visit one or more of the pages are included in the remarketing list corresponding to the list to which the page was categorized, which includes users who previously visited the page through the navigation behavior of users who visited the website and characteristics of the page when the remarketing list was generated.

[0061] Figure 11 is a block diagram depicting one embodiment of an environment for automatically generating remarketing lists based on client sessions, according to an exemplary embodiment. Environment 100 includes at least one data processing system 110. Data processing system 110 may include at least one processor (or processing circuit) and memory. The memory stores processor-executable instructions that, when executed on the processor, cause the processor to perform one or more of the operations described herein. The processor may include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like, or a combination thereof. The memory may include, but is not limited to, electrical, optical, magnetic, or any other storage or transmission device capable of providing program instructions to the processor. The memory may also include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, FPGA, read-only memory (ROM), random-access memory (RAM), electrically erasable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor can read instructions. The instructions may include code from any suitable computer programming language. Data processing system 110 may include one or more computing devices or servers capable of performing various functions. In some implementations, data processing system 110 may include an advertising auction system configured to host the auction. In some implementations, data processing system 110 does not include an advertising auction system, but is configured to communicate with the advertising auction system via network 105 .

[0062] Network 105 may include a computer network such as the Internet, a local area network, a wide area network, a metropolitan area network, or other regional network, an intranet, a satellite network, other computer networks such as voice or data mobile phone communication networks, and combinations thereof. Data processing system 110 of environment 100 may communicate via network 105, for example, with at least one content provider 115, at least one content publisher computing device 120, or at least one client device 125. Network 105 may be any form of computer network that relays information between client device 125, data processing system 110, and one or more content sources, such as web servers, ad servers, and others. For example, network 105 may include the Internet and / or other types of data networks (such as a local area network (LAN), a wide area network (WAN), a cellular network, a satellite network, or other types of data networks). Network 105 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within network 105. Network 105 may also include any number of hardwired and / or wireless connections. For example, client device 125 may communicate wirelessly (eg, via WiFi, cellular, radio, etc.) with network 105 and transceivers that are hardwired to other computing devices (eg, via fiber optic cables, CAT5 cables, etc.).

[0063] Content providers 115 may include servers or other computing devices operated by a content provider entity to provide content items, such as advertisements, for display on information resources at client devices 125. Content provided by content providers 115 may include third-party content items or creatives (e.g., advertisements) for display on information resources, such as websites or web pages that include primary content, such as content provided by content publisher computing devices 120. Content items may also be displayed on search results web pages. For example, content providers 115 may provide or be the source of advertisements (ads) or other content items for display within content slots on content web pages, such as company web pages on which a company provides primary content for a web page, or for display on search results landing pages provided by search engines. Content items associated with content providers 115 may be displayed on information resources other than web pages on smartphones or other client devices 125, such as content displayed as part of an application (e.g., a gaming application, a global positioning system (GPS) or mapping application, or other types of applications). Content providers 115 may be configured to act as a web server for hosting one or more information resources of a content provider's website. The one or more information resources may be landing pages to which content items provided by content provider 115 are linked, such that when a client device interacts with a content item from content provider 115 , the client device is directed to the information resource identified as the landing page for the content item.

[0064] The content publisher computing device 120 may comprise a server or other computing device operated by a content publisher entity to provide primary content for display via the network 105. For example, the content publisher computing device 120 may comprise a webpage operator that provides primary content for display on a webpage. The primary content may include content other than that provided by the content publisher computing device 120, and the webpage may include content slots configured to display third-party content items (e.g., advertisements) from a content provider 115. For example, the content publisher computing device 120 may operate a company's website and provide content related to the company for display on the website's webpage. The webpage may include content slots configured to display third-party content items, such as advertisements from the content provider 115. In some embodiments, the content publisher computing device 120 comprises a search engine computing device (e.g., a server) of a search engine operator operating a search engine website. The primary content of the search engine webpage (e.g., a results or landing webpage) may include search results and third-party content items, such as content items from the content provider 115, displayed in the content slots. In some embodiments, the content publisher computing device 120 may comprise a server for serving video content. In some implementations, the content publisher computing device 120 may be the same as the content provider 115 .

[0065] The client device 125 may include data configured to communicate via the network 105 to display content such as content provided by the content publisher computing device 120 (e.g., primary web page content or other information resources) and content provided by the content provider 115 (e.g., third-party content items such as advertisements configured to be displayed in content slots on a web page). The client device 125, the content provider 115, and the content publisher computing device 120 may include desktop computers, laptop computers, tablet computers, smartphones, personal digital assistants, mobile devices, consumer computing devices, servers, clients, digital cameras, set-top boxes for televisions, video game consoles, or any other computing devices configured to communicate via the network 105. The client device 125 may be a communication device through which an end user may submit a request to receive content. The request may be a request from a search engine, and the request may include a search query. In some embodiments, the request may include a request to access a web page.

[0066] The content provider 115, the content publisher computing device 120, and the client device 125 may include a processor and a memory—that is, a processing circuit. The memory stores machine instructions that, when executed on the processor, cause the processor to perform one or more of the operations described herein. The processor may include a microprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like, or a combination thereof. The memory may include, but is not limited to, an electrical, optical, magnetic, or any other storage or transmission device capable of providing program instructions to the processor. The memory may also include a floppy disk, a CD-ROM, a DVD, a magnetic disk, a memory chip, an ASIC, an FPGA, a read-only memory (ROM), a random access memory (RAM), an electrically erasable ROM (EEPROM), an erasable programmable ROM (EPROM), a flash memory, an optical medium, or any other suitable memory from which the processor can read instructions. The instructions may include code from any suitable computer programming language.

[0067] The content provider 115, the content publisher computing device 120, and the client device 125 may also include one or more user interface devices. Generally, a user interface device refers to any electronic device (e.g., a keyboard, a mouse, a pointing device, a touch screen display, a microphone, etc.) that transmits data to a user by generating sensory information (e.g., visual information on a display, one or more sounds, etc.) and / or converts received sensory information from a user into electronic signals. According to various embodiments, the one or more user interface devices may be external to the housing of the content provider 115, the content publisher computing device 120, and the client device 125 (e.g., a built-in display, a microphone, etc.) or external to the housing of the content provider 115, the content publisher computing device 120, and the client device 125 (e.g., a monitor connected to the user computing device 115, speakers connected to the user computing device 115, etc.). For example, the content provider 115, the content publisher computing device 120, and the client device 125 may include an electronic display that visually displays a web page using web page data received from one or more content sources and / or from the data processing system 110 via the network 105. In some implementations, a content placement campaign manager or a third-party content provider such as an advertiser can communicate with data processing system 110 via content provider 115. In some implementations, the advertiser can communicate with data processing system 110 via a user interface displayed on a user interface device of content provider 115.

[0068] The data processing system 110 may include at least one server. For example, the data processing system 110 may include multiple servers in at least one data center or server farm. In some embodiments, the data processing system 110 may include a third-party content placement system, such as an ad server or ad placement system. The data processing system 110 may include at least one script provider module 130, at least one campaign management module 140, and at least one database 145. The script provider module 130 and the campaign management module 140 may each include at least one processing unit, server, virtual server, circuit, engine, agent, device, or other logic device such as a programmable logic array configured to communicate with the database 145 and other computing devices (e.g., content provider 115, content publisher computing device 120, or client device 125) via the network 105.

[0069] The script provider module 130 and the activity management module 140 may include or execute at least one computer program or at least one script. The script provider module 130 and the activity management module 140 may be separate components, a single component, or part of the data processing system 110. The script provider module 130 and the activity management module 140 may include a combination of software and hardware, such as one or more processors configured to execute one or more scripts.

[0070] Data processing system 110 may also include one or more content repositories or databases 145. Databases 145 may be local to data processing system 110. In some implementations, database 145 may be remote from data processing system 110 but may be in communication with data processing system 110 via network 105. Database 145 may include web pages, portions of web pages, third-party content items (e.g., advertisements), online reporting scripts, and the like to serve content providers 115 or client devices 125.

[0071] The script provider module 130 may be configured to retrieve an online reporting script (or online activity reporting script) from the database 145 and send the retrieved script to the content provider 115 (or another entity for sending to the content provider 115). The script may be embedded in the code of each web page on the website of the content provider 115. For example, an administrator of the content provider 115 may manually insert the script into the code used to generate or render each page of the content provider's website or into a selected page of the content provider, as desired.

[0072] The online report script may include computer-executable instructions that can be executed by one or more processors. The computer-executable instructions may include scripts such as Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), Extensible Markup Language (XML), Cascading Style Sheets (CSS), and JavaScript. The computer-executable instructions may be executed within an application of the content provider 115, such as an application that causes the content provider 115 to generate a web page that is transmitted to the client device 125 via the network 105. The application may include, for example, an Internet browser, a mobile application, a gaming application, a GPS application, or any other computer program that can read and execute computer-executable instructions.

[0073] The computer-executable instructions of the online reporting script, when executed by the processor of content provider 115, may cause the application of content provider 115 to transmit data regarding the website pages of content provider 115 (e.g., those pages that include the script) to data processing system 110. For example, the computer-executable instructions may transmit data such as, but not limited to, an identifier of a current information resource of the website (e.g., a URL), an identifier of an information resource of the website that refers to the current information resource (e.g., the URL a user used to reach the current information resource), an identifier of a converted information resource of the website (e.g., the URL presented to client device 125 in response to a conversion event initiated by client device 125), or the number of times a client device has accessed the current information resource, an identifier of a client device that accessed the current information resource, the referenced information resource, or the converted information resource (e.g., a cookie ID assigned to the accessing client device or an associated entity, etc.). In some embodiments, the information resource identifier may be a URL. In some embodiments, the information resource identifier may be a domain to which the information resource belongs. In some embodiments, the information resource identifier may be an IP address corresponding to a server hosting the information resource.

[0074] In some implementations, script provider module 130 may receive a request for an online reporting script from content provider 115 via network 105. In response to receiving the request, script provider module 130 may access database 145 to retrieve the online reporting script and may transmit data including the online reporting script to content provider 115.

[0075] In some embodiments, as described above, each page of a content provider's website may include an online reporting script. In some embodiments, the online reporting script may be directly inserted by data processing system 110 without requiring any action by the content publisher's computing device or the content provider. The online reporting script may be configured to execute on a client device 125 of a client accessing the content provider's website. In some embodiments, a browser may be executed on client device 125. The browser may be configured to execute one or more computer-executable instructions corresponding to the online reporting script embedded or inserted within the content provider's web page. The online reporting script may cause client device 125 to transmit data to one or more of content provider 115 and data processing system 110. In some embodiments, the online reporting script inserted within a web page of the content provider's computing system 115 website may be executed by a processor of client device 125. When the processor of client device 125 executes one or more other computer-executable instructions included in the web page, the processor may execute the online reporting script. Execution of the online reporting script may cause the processor to identify access-related data regarding the web page visit. Specifically, the data may include an identifier of a current information resource on the website (e.g., a URL), an identifier of an information resource on the website that refers to the current information resource (e.g., the URL a user used to reach the current information resource), an identifier of a converted information resource on the website (e.g., a URL presented to the client device 125 in response to a conversion event initiated by the client device 125), or the number of times a client device 125 has accessed the current information resource, an identifier of a client device that accessed the current information resource, the referencing information resource, or the converted information resource (e.g., a cookie ID assigned to the accessing client device or an associated entity). In some embodiments, the information resource identifier may be a URL. In some embodiments, the information resource identifier may be a domain to which the information resource belongs. In some embodiments, the information resource identifier may be an IP address corresponding to a server hosting the information resource. Execution of the online reporting script may cause the processor to transmit access-related data to the data processing system 110 or the content provider 115. The processor may transmit the access-related data via a BEACON request. In some embodiments, the processor may automatically transmit the access-related data without requiring any additional action by the user of the computing device.

[0076] In some embodiments, the campaign management module 140 may be configured to receive data from one or more client devices 125 that access an information resource including an online reporting script provided by the script provider module 125. In some embodiments, the campaign management module 140 may be configured to receive data corresponding to an information resource including an online reporting script for a website of the content provider 115 from the content provider 115. The campaign management module 140 may perform various operations and functions on the received data to generate optimally customized remarketing lists based on the information resource of the website of the content provider 115. In particular, the following Figure 2 and 5 140.

[0077] A. Site Map

[0078] Figure 2 is a block diagram depicting one embodiment of a campaign management module according to an exemplary embodiment. The campaign management module 140 may include multiple submodules, including a visitor management module 205, a graph construction module 210, a graph clustering module 215, and a remarketing list generation module 220. The campaign management module 140 may be executed by one or more processors of a computing device, such as the following: Figure 8 those processors described in to run or otherwise execute.

[0079] In some embodiments, upon accessing a website of content provider 115, client device 125 may execute an online reporting script embedded or included in the website's page code. For example, a web browser of client device 125 may load a page with an embedded online reporting script, and when the page is loaded on client device 125, the instructions of the online reporting script may be executed at client device 125. These instructions may cause client device 125 to identify visit-related data related to the page visit and transmit it to data processing system 110.

[0080] In some embodiments, data processing system 110 may provide a system for automatically generating remarketing lists based on client sessions. Visitor management module 205 may be executed on data processing system 110 and configured to identify multiple client sessions of client devices accessing a website of content provider 115. A client session may correspond to a client device 125 accessing one or more information resources of the website of content provider 115. In some embodiments, a client session corresponding to a client device 125 may be determined (e.g., by data processing system 110 or by content provider 115) based on the number of web pages accessed by the client device 125. For example, a client session may be identified in response to a client device 125 accessing a single web page of a website, or in response to a client device 125 accessing a number of pages greater than a threshold (e.g., two or more pages of a website). In some embodiments, information resources including embedded online reporting scripts may trigger a client device 125 to transmit (e.g., to data processing system 110) identification information of the client device 125 corresponding to the client session based on the client device's access to those information resources. Thus, the online reporting script provides access-related data to the data processing system 110 so that the data processing system 110 can determine navigation path information of the client device 125 during the client session.

[0081] In some embodiments, the online reporting script is executed on the client device and can trigger the client device 125 to transmit an identifier of the information resource that was accessed, a number of times when the information resource was accessed, or an identifier of the client device 125 that accessed the information resource, etc. The data processing system 110, in conjunction with the information resource including the online reporting script, can determine multiple navigation paths corresponding to multiple client sessions, and the visitor management module 205 can aggregate the navigation path data (e.g., by sorting and organizing the data into a table of entries).

[0082] The visitor management module 205 may determine one or more node pairs for each of the plurality of client sessions, each of the one or more node pairs including a destination node representing a destination information resource and a reference node representing a reference information resource corresponding to an information resource accessed by the client device. In some embodiments, because the online reporting script instructs the client device 125 to transmit identifier information for a reference information resource and an information resource to which the reference information resource is linked based on the determined navigation path of the client device, the visitor management module 205 may determine a pair of discrete information resources of the directly linked content provider 115. For example, a pair of information resources may include a reference information resource and a destination information resource to which the reference information resource is linked, and the visitor management module 205 may enter the reference information resource and the destination information resource into a table of entries for subsequent access (described further below). Accordingly, the visitor management module 205 may assign information resource pairs to corresponding node pairs.

[0083] In some embodiments, the graph construction module 210 may be executed on the data processing system 110 and configured to use multiple client sessions to construct a graph comprising the determined node pairs and a plurality of edges between the destination nodes of the node pairs and the corresponding reference nodes. In addition to determining the linked nodes based on the determined multiple client sessions, the graph construction module 210 may also determine links between the determined nodes. Links between nodes may be referred to as edges. For example, in response to a client device 125 accessing a first information resource of a website of a content provider 115, and in response to the client device 125 accessing a second information resource of the website via the first information resource (e.g., due to the client device 125 accessing a hyperlink on the first information resource), the graph construction module 210 (or the visitor management module 205) may determine a first node corresponding to the first information resource and a second node corresponding to the second information resource, as well as the presence of an edge or link between the first information resource and the second information resource. Once the graph construction module 210 (e.g., from the visitor management module 205) receives sufficient client session information and sufficient node pairs, the graph construction module 210 may construct a graph comprising nodes corresponding to the visited information resources of the website and edges between the nodes. For example, if graph construction module 210 receives information identifying multiple information resources, graph construction module 210 may compare the number of identified information resources with a threshold before constructing the graph. If the number exceeds the threshold, graph construction module 210 may initiate graph construction.

[0084] In some embodiments, the graph construction module 210 (or visitor management module 205) may receive, for example, an identifier of a website page visited, a client identifier of a client device 125 accessing the page, a number of visits, a referring page, and the like. In some embodiments, in response to receiving this information, the graph construction module 210 (or visitor management module 205) may organize the data into entries in a table. For example, a table of online activities stored by the data processing system 110 may have entries corresponding to a referring URL, a destination URL accessed from the referring URL, the number of times a client device 125 accessed the referring and destination URLs, an identifier of the client device 125, and the like. For example, each time a client device 125 accesses a page of a website, the client device 125 may send visit-related data to the data processing system 110. In response to the received data, the data processing system 110 may create an entry in a stored table corresponding to the received visit-related data.

[0085] In some embodiments, data processing system 110 may generate a second table for storing information based on an analysis of the first table containing the received data. For example, based on an analysis of the referring URL and the destination URL, graph construction module 210 may determine whether an edge exists between specific URLs. In some embodiments, further based on the client identifier of client device 125, graph construction module 210 may determine which client devices reached the referring URL instead of the destination URL, and which client devices reached the destination URL from the referring URL, to determine click-through rates. Thus, the second table may include calculation results based on the data in the first table (e.g., click-through rates between nodes).

[0086] In some embodiments, each of the plurality of edges between the target node and the referencing node of a corresponding node pair may represent a ratio of a first number of client devices that accessed the destination information resource from the referencing information resource to a second number of client devices that accessed the referencing information resource. The graph construction module may further determine a click-through rate corresponding to each edge. For example, because the activity management module 140 receives information identifying the client device 125 and information about which information resources the client device 125 navigated to, the graph construction module 210 may determine a ratio of visitors who accessed the first information resource (e.g., the referencing information resource) to visitors who accessed the destination information resource (e.g., the information resource to which the referencing information resource links). In other words, the graph construction module 210 may determine a click-through rate for the node pair based on the aggregated navigation path of the client sessions to determine the percentage of visitors who accessed one information resource from another referencing information resource. In this manner, the graph construction module 210 may assign a click-through rate to each edge of the graph represented by the determined ratio (e.g., inputting the determined click-through rate into a stored table).

[0087] In some embodiments, the graph construction module 210 may omit at least one of the plurality of edges and a node pair associated with the at least one edge whose ratio is below a ratio threshold in the graph. The graph construction module 210 may store a predetermined threshold, or may automatically set a threshold for the click-through rates of the edges in the graph. The graph construction module 210 may compare each of the click-through rates corresponding to each edge, and omit or delete the edges whose click-through rates are below the threshold in the graph. In some embodiments, the graph construction module 210 may also delete the source node and the destination node at the end of the edge whose click-through rate is below the threshold in the graph. Thus, the graph construction module 210 may construct a graph with nodes that are frequently visited by visitors (e.g., nodes associated with edges whose click-through rates exceed the threshold), and may delete those nodes that are infrequently visited (e.g., nodes associated with edges whose click-through rates are below the threshold), because relatively infrequently visited nodes may produce inaccurate website mapping and clustering (if included in the graph).

[0088] In some embodiments, graph clustering module 215 may be executed on data processing system 110 and may be configured to assign one or more of the plurality of information resources to a first group of information resources. Graph clustering module 215 may analyze the information resources or nodes represented in the graph using the table using entries corresponding to the websites of content providers 115 to form groups or clusters based on patterns or related characteristics.

[0089] In some embodiments, the graph clustering module 215 may cluster the graph based on the number of edges between nodes corresponding to the first set of information resources and transition nodes representing information resources presented in response to a transition event. The visitor management module 205 may identify transition information resources for the website of the content provider 115 based on multiple client sessions. The transition information resources may correspond to pages of the website that indicate transitions by the client device 125. For example, the website may include a purchase confirmation page that indicates transitions by the client device 125 (to purchase an item) during the client session.

[0090] In some embodiments, the visitor management module 205 may identify the conversion information resource based on the number of reference information resources that link to the conversion page. For example, the conversion page of a website may have most of the reference pages that link to the conversion page (directly or indirectly) because all product pages end at a purchase confirmation or conversion page. In addition, the conversion information resource may be a page of a website that is relatively downstream in the node sequence because the client session typically ends at the purchase confirmation screen. Therefore, the visitor management module 205 may identify the conversion information resource based on the structure of the graph (e.g., based on the number of reference pages of the page or based on how far downstream the page is in the graph). In some embodiments, the administrator of the content provider 125 may mark or indicate the conversion information resource, and the visitor management module 205 may receive this information identifying the conversion page.

[0091] In some embodiments, the graph clustering module 215 may group or cluster the nodes of the graph based on the distance between the nodes and the transition nodes corresponding to the transition information resources. For example, the graph clustering module 215 may group or cluster the nodes in the graph based on the number of levels or layers of edges between the node or a given node and the transition nodes. For example, the graph clustering module 215 may group or cluster all nodes with three levels or layers of edges between the node and the transition nodes into a first group, group or cluster all nodes with five levels of edges between the node and the transition nodes into a second group, group or cluster all nodes with two levels of edges between the node and the transition nodes into a third group, and so on. In some embodiments, the graph clustering module 215 may cluster based on a range of distances between the nodes and the transition nodes. For example, the graph clustering module 215 may group all nodes with three to five levels of edges between the node and the transition nodes, group all nodes with six to nine levels of edges between the node and the transition nodes, and so on. In some embodiments, the distance between a node and a transition node may be the minimum number of edges from the node to the transition node.

[0092] In some embodiments, the graph clustering module 215 may cluster the graph based on the click-through rates of node pairs that include the same reference node and different destination nodes. For example, the graph clustering module 215 may identify a reference node and multiple different destination nodes that are linked to the common reference node via different edges. The graph clustering module 215 may compare each of the click-through rates of different edges with a threshold when determining whether the nodes associated with the edge should be clustered. For example, the higher the click-through rate, the more strongly the node pairs associated with the edge are likely to be related. Therefore, if each of the click-through rates of multiple edges between a common reference node and a destination node exceeds a threshold, the graph clustering module 215 may cluster the common reference node and the destination node into a group. The graph clustering module 215 may optionally group the nodes in the graph based on the distance to the nodes in other nodes, the click-through rate of the edges, the proximity of the nodes to each other, etc., to group nodes that are similar to each other (for example, grouping of nodes corresponding to categories of products or items sold on the website). In some embodiments, the graph clustering module 215 may utilize a clustering algorithm to identify groups of similar information resources. The graph clustering module 215 can utilize clustering algorithms such as, but not limited to, connectivity-based clustering (hierarchical clustering), centroid-based clustering, distribution-based clustering, density-based clustering, etc. In some implementations, cluster analysis can calculate a clustering score for a group of nodes and compare the score to a threshold to determine whether the group of nodes should be clustered.

[0093] Connection-based clustering is based on the core idea that objects that are more related to nearby objects are more related than objects that are farther away. Connection-based clustering can connect "objects" to form "clusters" based on their distances. Clustering can be described primarily by the maximum distance required to connect the parts of the cluster. At different distances, different clusters can be formed. Connection-based clustering is a whole set of methods that differ in the way the distance is calculated. In some embodiments, in addition to the distance function, the link criterion to be used must also be identified (for example, since a cluster consists of multiple objects, there are multiple candidates for calculating the distance). In some embodiments, the link criterion selection can be single link clustering (for example, the minimum value of the object distance), complete link clustering (for example, the maximum value of the object distance), or unweighted group pair method with arithmetic mean (UPGMA), also known as average link clustering. In addition, hierarchical clustering can be synthetic (for example, starting from single elements and clustering them into clusters) or divisive (for example, starting from a complete data set and dividing it into partitions).

[0094] In centroid-based clustering, clusters are represented by center vectors, which may not necessarily be members of the dataset. When the number of clusters is fixed to k, k-means clustering is formally defined as an optimization problem: find the centers of k clusters and assign the objects to the nearest cluster center such that the square of the distance to the cluster is minimized.

[0095] Distribution-based clustering can be based on a distribution model. For example, in some embodiments, the graph clustering module 215 can use a predetermined hierarchical model of website pages as a distribution model. For example, a typical website may follow a diamond shape, such as Figure 3 As shown in , it can be used as a hierarchical model. In some embodiments, clusters can then be defined as objects that are most likely to belong to the same distribution. A convenient feature of this approach is that it is very similar to how artificial datasets are generated: by sampling random objects from a distribution.

[0096] In density-based clustering, clusters can be defined as areas with a higher density than the rest of the data set. Objects in these sparse areas (the objects needed to separate the clusters) are typically considered noise and boundary points. In some embodiments, density-based clustering can be characterized by a well-defined clustering model called "density reachability." Similar to link-based clustering, density-based clustering can be based on connected points within a specific distance threshold.

[0097] The remarketing list generation module 220 may be executed on the data processing system 110 and may be configured to assign a first set of information resources to a remarketing list. In some embodiments, the remarketing list may include client identifiers of client devices that accessed the information resources assigned to the first set of information resources. The remarketing list generation module 220 may assign groups of related information resources identified by the graph clustering module 215 to separate the remarketing lists. The remarketing list generation module 220 may then assign a list of client identifiers of client devices that accessed the first set of information resources to a particular remarketing list assigned the first set of information resources, as determined by the visitor management module in conjunction with an online reporting script embedded in the code of the website page. In some embodiments, the remarketing list generation module 220 may assign information resources to a remarketing list based on the distance of the information resource from the identified conversion page. In some embodiments, the remarketing list generation module 220 may assign information resources to a remarketing list based on grouping or clustering of the information resources (e.g., based on click-through rate).

[0098] The remarketing list generation module 220 may be executed on the data processing system 110 and may be configured to assign client identifiers that accessed the first set of information resources to a remarketing list. The remarketing list generation module 220 may access the table of entries to determine the client identifiers that accessed the first set of information resources, and the remarketing list generation module 220 may assign these client identifiers to the remarketing list. In some embodiments, when the client device 125 accesses multiple information resources of the website, the remarketing list generation module 220 may assign the client identifier of the client device 125 to a remarketing list based on the information resource accessed by the client device 125 that is closest to the converted information resource.

[0099] In some embodiments, the remarketing list generation module 220 may use the remarketing list to select content items for display in response to receiving a content request from one of the client identifiers assigned to the remarketing list that accesses the first set of information resources. The remarketing list generation module 220 may compare the client identifier (e.g., a cookie ID) of the requesting client device 125 with the client identifiers of one or more remarketing lists stored on the data processing system 110. If the requesting client identifier matches the client identifier of the stored remarketing list, the data processing system 110 may perform a remarketing function for the client device 125 identified by the client identifier. For example, the data processing system 110 may send a content item to the client device 125 for display based on the remarketing list on which the client identifier of the client device 125 is found (e.g., the data processing system 110 may send an advertisement to the client device 125 based on the remarketing list associated with the client device 125).

[0100] Figure 3 300 is a diagram depicting a page of a website for automatically generating remarketing lists according to an exemplary embodiment. Diagram 300 includes a plurality of information resource nodes 302 representing a presented website, which may be generated by campaign management module 140. One of these nodes may be identified as a transition node 308. Each of nodes 302 is attached to one or more edges 304. Each edge 304 includes a corresponding click-through rate 306, which indicates the rate of click-throughs from a reference node attached to edge 304 to a destination node attached to the same edge 304. Additionally, diagram 300 illustrates two different variations of clustering or grouping of nodes: horizontal clustering 310 and vertical clustering 312.

[0101] In some embodiments, the graph construction module 210 may identify a conversion node 308 based on a number of reference nodes that directly or indirectly link to the destination node. For example, the graph construction module 210 may identify a conversion node 308 for a website by determining which of the multiple nodes of the website links (directly or indirectly) the most reference pages. In some embodiments, an administrator of the content provider 125 may mark or indicate the conversion nodes, and the visitor management module 205 may receive this information identifying the conversion pages. In some embodiments, the graph construction module 210 may identify a conversion node 308 based on a ratio of reference nodes to destination nodes for each node, and if the node's ratio exceeds a threshold or if the node's ratio is the highest among all the nodes' ratios, the node may be identified as a conversion node 308.

[0102] Graph 300 illustrates an example of a directed and weighted graph representing a website of content provider 115. Furthermore, graph 300 illustrates how graph 300 can be used to generate remarketing lists. For example, each of two horizontal clusters 310 corresponds to a remarketing list in which the circled nodes are the same distance from the conversion page (e.g., the pages "shoes," "clothing," and "men's" are each the same distance from the conversion page "purchase," and each node in the cluster is a third-level edge from conversion node 308). Thus, horizontal clusters 310 are determined based on the distance from the node conversion. Alternatively, each of three vertical clusters 312 corresponds to a remarketing list in which the circled nodes are associated with the same product category, based on the clustering of similar pages represented by the nodes (e.g., the pages "shoes," "shoe 1," "shoe 2," and "shoe 3" are each associated with the same product category).

[0103] After constructing graph 300, graph 300 may be stored at activity management module 140 for cluster analysis. In some embodiments, any suitable graph data model may be used to store graph 300, such as, but not limited to, Resource Description Framework (RDF). The RDF data model is based on the idea of ​​stating resources (particularly web resources) in the form of subject-predicate-object representations. These representations are referred to as triples in RDF terminology. The subject represents the resource, and the predicate represents a feature or aspect of the resource and represents the relationship between the subject and the object. A collection of PDF representations essentially represents a labeled directed multigraph.

[0104] Figure 4 is a flow chart depicting a method 400 for automatically generating a remarketing list according to an exemplary embodiment. Briefly, the method 400 may include a data processing system identifying a plurality of client sessions (block 405). The method 400 may include the data processing system determining one or more node pairs based on the client sessions (block 410). The method 400 may include the data processing system constructing a graph including the node pairs and edges between the nodes (block 415). The method 400 may include the data processing system assigning information resources to a first set of information resources based on the graph (block 420). The method 400 may include the data processing system assigning the first set of information resources to a remarketing list (block 425). The method 400 may include the data processing system assigning a client identifier to access the first set of information resources (block 430). The method 400 may include the data processing system selecting content items for display using the remarketing list (block 435).

[0105] In more detail, method 400 may include a data processing system identifying a plurality of client sessions (block 405). In some implementations, visitor management module 205 may identify the plurality of client sessions. The client sessions may correspond to client devices accessing one or more information resources of a content provider's website. The client sessions allow the data processing system to determine navigation path information on the website of the client device during the client sessions.

[0106] Method 400 may include a data processing system determining one or more node pairs based on a client session (block 410). The data processing system may determine the one or more node pairs for each of a plurality of client sessions. Each of the one or more node pairs may include a destination node representing a destination information resource and a reference node representing a reference information resource corresponding to an information resource from which a client device accessed the destination information resource. For example, a pair of information resources may include a reference information resource and a destination information resource to which the reference information resource links.

[0107] In some embodiments, method 400 may include creating a table of entries based on the received access-related information. The table may include data related to online activity and may be maintained by data processing system 110. The table may have entries corresponding to a referring URL, a destination URL accessed from a referring URL, the number of times a client device 125 accessed the referring and destination URLs, an identifier of client device 125, and the like. Method 400 may include identifying a conversion node based on the table of entries based on the number of referring nodes linked to the destination node, and method 400 may include identifying a conversion node based on the maximum number of referring nodes linked to the destination node. In some embodiments, identifying the conversion node may be based on a ratio of referring nodes to destination nodes for each node.

[0108] Method 400 may include the data processing system constructing a graph including node pairs and edges between the nodes (block 415). The data processing system may use multiple client sessions to construct the graph including the determined node pairs and multiple edges between the destination nodes of the node pairs and the corresponding reference nodes. For example, in response to a client device accessing a first information resource of a website of a content provider, and in response to the client device accessing a second information resource of the website via the first information resource, the data processing system may determine that a first node corresponding to the first information resource and a second node corresponding to the second information resource exist, and that an edge or link exists between the first information resource and the second information resource.

[0109] The data processing system may also determine a ratio of a first number of client devices that accessed the destination information resource from the reference information resource to a second number of client devices that accessed the reference information resource for each of a plurality of edges between the destination node and the reference node of the corresponding node pair. The ratio may represent a click-through rate between the nodes in the node pair. The graph processing system may omit at least one of the plurality of edges and node pairs associated with the at least one edge whose ratio is below a ratio threshold from the graph. In some embodiments, the method 400 may generate a second table for storing information based on an analysis of a first table of entries including the received data. For example, based on an analysis of the reference URL and the destination URL, the method 400 may include determining whether an edge exists between specific URLs. In some embodiments, further based on a client identifier of the client device 125, the method 400 may include determining client devices that reach the reference URL instead of the destination URL and client devices that reach the destination URL from the reference URL to determine the click-through rate.

[0110] Method 400 may include a data processing system assigning information resources to a first set of information resources based on a graph (block 420). The data processing system may cluster the graph based on the number of edges between nodes corresponding to the first set of information resources and transition nodes representing information resources presented in response to transition events. Alternatively, the graph clustering module may cluster the graph based on click-through rates for pairs of nodes that include the same reference node and different destination nodes. For example, the data processing system may cluster the nodes based on the location of the nodes, the click-through rates of the edges, the proximity of the nodes to each other, etc. to group nodes that are similar to each other. In some embodiments, the data processing system may utilize a clustering algorithm to identify groups of similar information resources of the graph.

[0111] Method 400 may include the data processing system assigning a first set of information resources to a remarketing list (block 425). In some implementations, the remarketing list may include a client identifier of a client device that accessed the first set of information resources. The data processing system may separate the remarketing lists by assigning different sets of related information resources.

[0112] Method 400 may include assigning, by the data processing system, client identifiers that accessed the first set of information resources to a remarketing list (block 430). Method 400 may include accessing a table of entries to identify those client identifiers that accessed the first set of information resources. Method 400 may include assigning those client identifiers to the remarketing list. In some implementations, method 400 may include assigning, when a client device accesses multiple information resources of a website, a client identifier of a client device to a remarketing list based on an information resource that the client device accessed that is closest to a conversion resource.

[0113] Method 400 may include the data processing system selecting a content item for display using a remarketing list (block 435). Method 400 may include comparing a client identifier (e.g., a cookie ID) of a requesting client device with client identifiers of one or more remarketing lists stored on the data processing system. If the requesting client identifier matches a client identifier of a stored remarketing list, the method may include performing a remarketing function with respect to the client device identified by the client identifier. For example, method 400 may include sending a content item for display to the client device based on the remarketing list on which the client identifier of client device 125 is found. For example, the content item may be an advertisement.

[0114] Website Conversion Metrics

[0115] Figure 5is a block diagram depicting one embodiment of a campaign management module according to an exemplary embodiment. In some embodiments, the campaign management module 140 includes an information resource management module 505, a visitor management module 205, a conversion rate determination module 515, and a remarketing list generation module 220. In addition to modules 505, 205, 515, and 220, the campaign management module 140 may include a combination of Figure 2 Some or all of the submodules described.

[0116] In some embodiments, the information resource management module 505 can be executed on the data processing system 110 and can be configured to identify, for a website corresponding to the content provider 115, a plurality of information resources included in the website. The information resource management module 505 can provide the same functionality and operations as the visitor management module 205 and can thus identify the client device 125 that accesses the website, a page of the website, etc.

[0117] At least one of the plurality of information resources may correspond to a landing page of an advertiser's content item provided for display as a third-party content item. The landing page may be one of the information resources of the advertiser's website. The advertisement for display may be generated by the content provider 115 at the content publisher computing device 120.

[0118] In some embodiments, at least one of the plurality of information resources may be identified as a conversion information resource presented in response to a conversion event. As described above in conjunction with the visitor management module 205, the information resource management module 505 may identify a conversion information resource for the website of the content provider 115 (e.g., by being specified by the content provider 115 or by analyzing a mapping of the website of the content provider 115). In some embodiments, at least one information resource of the website may include an online activity reporting script to report online activity corresponding to the client device 125 to the data processing system or the content provider 125 in order to determine a conversion rate for the at least one information resource. In some embodiments, the online activity reporting script may correspond to the online activity reporting script described above and may be embedded in the code of one or more pages of the website.

[0119] In some embodiments, the visitor management module 205 may be executed on the data processing system 110 and may store, via the online activity reporting script, a plurality of entries corresponding to information resource accesses by client devices in a data structure for each of at least one information resource of a website including the online activity reporting script. The plurality of entries may be stored as a table. In some embodiments, each of the plurality of entries may identify a resource identifier for identifying the information resource and a client identifier for identifying the client device accessing the information resource. As described above in conjunction with Figure 2As described above, the visitor management module 205 can identify one or more of the client devices 125 that access pages of the website including the online activity reporting script, and identify the information resources accessed by the client devices 125 and other information associated with the access (e.g., number of visits, and referring information resources, etc.). This information corresponds to the above-mentioned information in conjunction with Figure 2 The visitor management module 205 receives the visit-related information.

[0120] In some embodiments, the information resource management module 505 may receive, for at least one information resource of a website that includes an online activity reporting script, data corresponding to an identification of the at least one information resource of the website, an identification of a client device 125 that accessed the at least one information resource of the website, and a number of times the client device 125 accessed the at least one information resource of the website. In some embodiments, the information resource management module 505 may receive, for a conversion information resource of the website, data corresponding to an identification of the conversion information resource of the website, an identification of the client device 125 that accessed the conversion information resource of the website, and a number of times the client device 125 accessed the conversion information resource of the website.

[0121] In some embodiments, the conversion rate determination module 515 may be executed on the data processing system 110 and may be configured to determine, for an information resource, a conversion rate based on a first number of client identifiers relative to a second number of client identifiers of the client devices 125 that accessed the information resource and also accessed the converted information resource relative to a first number of client identifiers of the client devices 125 that accessed the information resource. As described above, the data processing system 110 may store the access-related information in a table of entries. The conversion rate determination module 515 may determine the conversion rate by accessing the table of entries, determining the first number of client identifiers and the second number of client identifiers, and calculating a ratio between the first number and the second number.

[0122] The visitor management module 205 can identify whether the client device 125 visits the conversion page based on the table of entries. Figure 2 and Figure 3 to identify conversion pages. Thus, the conversion rate determination module 515 can determine the ratio of visitors who visited both a specific resource and a conversion resource on the website to all visitors who visited the specific resource. Thus, the conversion rate determination module 515 can determine a conversion rate for each resource on the website, with each conversion rate indicating the percentage of visitors who visited a specific resource who ultimately converted. In some embodiments, the conversion rate determination module 515 can normalize conversion rates across websites. For example, a website selling cars may have a much lower conversion rate than a website selling flowers, so the conversion rate determination module 515 can normalize website conversion rates across different websites to achieve consistency.

[0123] In some embodiments, the conversion rate determination module 515 may determine the conversion rate based on a conversion time limit between when the client device 125 accesses the information resource and when the client device 125 accesses the converted information resource. For example, in determining the conversion rate, the conversion rate determination module 515 may set a time limit from when the client device 125 accesses the initial information resource to when the client device 125 accesses the converted information resource. If the client device 125 accesses the converted information resource outside of the time limit (e.g., two days), the conversion rate determination module 515 may not count the visitor as a conversion for the purpose of calculating the conversion rate of the initial information resource.

[0124] In some embodiments, the remarketing list generation module 220 may be executed on the data processing system 110 and may be configured to assign a first information resource of at least one information resource of a website including an online activity reporting script to a first remarketing list based on the determined conversion rate of the first information resource. The first remarketing list may include a client identifier of a client device that accessed the first information resource. The remarketing list generation module 220 may organize or sort the information resources of the website according to their respective conversion rates. Based on the sorting, the remarketing list generation module 220 may assign subsets of the information resources to different remarketing lists based on the conversion rates.

[0125] For example, the remarketing list generation module 220 may assign information resources with conversion rates greater than a threshold and information resources with conversion rates less than a threshold to different remarketing lists. In some embodiments, the remarketing list generation module 220 may determine one or more conversion rate thresholds for assigning information resources to remarketing lists. In some embodiments, the conversion rate determination module 515 may calculate a median conversion rate based on the conversion rate of each of at least one information resource of a website that includes an online activity reporting script, and may assign at least one conversion rate threshold as a multiple of the median conversion rate. For example, the conversion rate determination module 515 may assign a threshold to the median conversion rate among all conversion rates of the website's information resources.

[0126] In some embodiments, the conversion rate determination module 515 may assign multiple conversion rate thresholds as various multiples of the median conversion rate. For example, the conversion rate determination module 515 may assign separate thresholds corresponding to 0, 0.5, 1, 1.5, and 2 times the median conversion rate for a total of five different thresholds. Thus, the remarketing list generation module 220 may assign information thresholds to different remarketing lists bounded by the thresholds (e.g., information resources with conversion rates between 1 and 1.5 times the median conversion rate may be assigned to a remarketing list).

[0127] In some embodiments, the conversion rate determination module 515 may calculate an average conversion rate based on the conversion rate of each of at least one information resource of the website including the online activity reporting script, and the conversion rate determination module 515 may assign at least one conversion rate threshold based on a multiple of the standard deviation of the average conversion rate. In some embodiments, the conversion rate of the information resource may follow a Poisson distribution, in which case the conversion rate threshold may be derived based on the location of the information resource (e.g., the threshold may be determined based on the conversion rate of the information resource close to the conversion page).

[0128] Figure 6 300 is a table depicting pages of a website for automatically generating remarketing lists according to an exemplary embodiment. Table 600 depicts multiple information resources (web pages) with corresponding conversion rates. The table also illustrates different shaded rows corresponding to different remarketing lists 602, 604, 606, 608, and 609. The depicted information resources (corresponding to some of the information resources depicted in diagram 300) are sorted in descending order according to their corresponding conversion rates. In some embodiments, conversion rate determination module 515 may calculate an intermediate conversion rate based on the conversion rates of the information resources in table 600. Conversion rate determination module 515 may set multiple conversion rate thresholds based on the determined intermediate conversion rates. For example, conversion rate determination module 515 may assign separate thresholds corresponding to 0, 0.5, 1, 1.5, and 2 times the intermediate conversion rate for a total of five different thresholds.

[0129] For example, the conversion rate determination module 515 may determine that the information resources depicted in table 600 have a median conversion rate of 0.26. The conversion rate determination module 515 may use the determined median conversion rates to determine multiple conversion rate thresholds of 0.00 (0 times the median conversion rate), 0.13 (0.5 times the median conversion rate), 0.26 (1.0 times the median conversion rate), 0.39 (1.5 times the median conversion rate), and 0.52 (2.0 times the median conversion rate). If multiple thresholds are determined, the remarketing list generation module 220 may assign each of the depicted information resources to a remarketing list based on a range of conversion rates bounded by the multiple thresholds and the actual conversion rates of the information resources. For example, because the information resource "shopping cart" is the only information resource with a conversion rate above the 0.52 conversion rate threshold (2 times the median conversion rate), the remarketing list generation module 220 may assign only the "shopping cart" information resource to the remarketing list 602. Similarly, because the information resources "Dress 1," "Shoe 1," "Item 1," and "Dress 2" are the only information resources with conversion rates between a 0.26 conversion rate threshold (1 times the median conversion rate) and a 0.39 conversion rate threshold (1.5 times the median conversion rate), the remarketing list generation module 220 may assign only the information resources "Dress 1," "Shoe 1," "Item 1," and "Dress 2" to the remarketing list 606. Thus, the information resources of table 600 may be segmented and organized into different remarketing lists 602, 604, 606, 608, and 610.

[0130] Figure 7 7 is a flow chart depicting a method for automatically generating remarketing lists according to an exemplary embodiment. Briefly, method 700 may include a data processing system identifying a plurality of information resources (block 705). Method 700 may include the data processing system storing data corresponding to information resource accesses (block 710). Method 700 may include the data processing system determining conversion rates for the plurality of information resources (block 715). Method 700 may include the data processing system assigning a first information resource to a first remarketing list based on its conversion rate (block 720).

[0131] In more detail, method 700 may include the data processing system identifying a plurality of information resources (block 705). The information resource management module 505 may be executed on the data processing system 110 and may be configured to identify, for a website corresponding to a content provider 115, a plurality of information resources included in the website. The information resource management module 505 may provide the same functionality and operations as the visitor management module 205 and, thus, may identify a client device 125 that accesses a website, a page of a website, etc. At least one of the plurality of information resources may be identified as a conversion information resource to be presented in response to a conversion event.

[0132] Method 700 may include a data processing system storing data corresponding to information resource accesses (block 710). In some implementations, visitor management module 205 may be executed on data processing system 110 and may, via an online activity reporting script, store, in a data structure, a plurality of entries corresponding to information resource accesses by a client device for each of at least one information resource of a website including the online activity reporting script. Each of the plurality of entries may identify a resource identifier for identifying the information resource and a client identifier identifying a client device accessing the information resource.

[0133] In some embodiments, the information resource management module 505 may receive, for at least one information resource of a website that includes an online activity reporting script, data corresponding to an identifier of the at least one information resource of the website, an identifier of a client device that accessed the at least one information resource of the website, or the number of times the client device accessed the at least one information resource of the website. In some embodiments, the information resource management module 505 may receive, for a conversion information resource of the website, data corresponding to an identifier of the conversion information resource of the website, an identifier of a client device that accessed the conversion information resource of the website, or the number of times the client device accessed the conversion information resource of the website.

[0134] Method 700 may include the data processing system determining conversion rates for a plurality of information resources (block 715). Visitor management module 205 may further identify whether client device 125 accessed a conversion page. Thus, conversion rate determination module 515 may determine a ratio of visitors who visited both a particular information resource of the website and a converted information resource to all visitors who visited the particular information resource. In some embodiments, conversion rate determination module 515 may normalize conversion rates across websites. Conversion rate determination module 515 may determine the conversion rate based on a time period between when a client device 125 accessed an information resource and when a client identifier accessed a converted information resource.

[0135] Method 700 may include the data processing system assigning a first information resource to a first remarketing list based on its conversion rate (block 720). The remarketing list generation module 220 may be executed on the data processing system 110 and may be configured to assign a first information resource of at least one information resource of a website including an online activity reporting script to a first remarketing list based on the determined conversion rate of the first information resource. The remarketing list generation module 220 may assign information resources with relatively high conversion rates and information resources with relatively low conversion rates to different remarketing lists. In some embodiments, the remarketing list generation module 220 may determine one or more conversion rate thresholds for assigning information resources to remarketing lists. In some embodiments, the conversion rate determination module 515 may calculate an intermediate conversion rate based on the conversion rate of each of the at least one information resource of the website including the online activity reporting script and may assign at least one conversion rate threshold as a multiple of the intermediate conversion rate.

[0136] Figure 8 1. The overall architecture of an exemplary computer system 800 is shown, in accordance with some embodiments, which may be used to implement any of the computer systems discussed herein, including the system 110 and its components such as the script provider module 130 and the activity management module 140. The computer system 800 may be used to provide information for display via the network 105. Figure 8 The computer system 800 includes one or more processors 820 communicatively coupled to a memory 825, one or more communication interfaces 805, and one or more output devices 810 (e.g., one or more display units) and one or more input devices 815. The processor 820 may be included in the data processing system 110 or other components of the system 110, such as the script provider module 130 and the activity management module 140.

[0137] exist Figure 8 In the computer system 800 of FIG. 8 , the memory 825 may include any computer-readable storage medium and may store computer instructions, such as processor-executable instructions for implementing the various functions described herein for the corresponding system, as well as any data related thereto, generated thereby, or received via (one or more) communication interfaces or (one or more) input devices (if present). Referring again to FIG. Figure 1 In the system 110 of FIG. 1 , the data processing system 110 may include a memory 825 to store information related to the availability of a repository of one or more content units and the retention of one or more content units. The memory 825 may include a database 145 . Figure 8 The processor(s) 820 shown in FIG. 8 may be used to execute instructions stored in a memory 825 , and in doing so may also read from or write to the memory various information processed and / or generated according to the execution of the instructions.

[0138] Figure 8 The processor 820 of the computer system 800 shown in FIG. 1 may also be communicatively coupled to or control the communication interface(s) 805 to send or receive various information in accordance with the execution of instructions. For example, the communication interface(s) 805 may be coupled to a wired or wireless network, bus, or other communication means, and thus may allow the computer system 800 to send information to or receive information from other devices (e.g., other computer systems). Although not shown in FIG. Figure 1 Although not explicitly shown in the system 800, one or more communication interfaces facilitate the flow of information between the components of system 800. In some implementations, the communication interface(s) may be configured (e.g., via various hardware components or software components) to provide a website as an access portal to at least some aspects of computer system 800. Examples of communication interface 805 include a user interface (e.g., a web page) through which client device 125 can communicate with data processing system 110.

[0139] Figure 8 The output device 810 of the computer system 800 shown in FIG. 1 can be configured to, for example, allow various information to be observed or otherwise perceived in conjunction with the execution of instructions. The input device(s) 815 can be configured to, for example, allow the client device 125 to manually adjust, make selections, input data, or interact with the processor in any of a variety of ways during the execution of instructions. Additional information regarding the overall computer system architecture that can be employed by the various systems discussed herein is further provided herein.

[0140] The subject matter and implementations of the operations described in this specification may be implemented using digital electronic circuitry, or computer software, firmware, or hardware implemented on tangible media, including the structures disclosed in this specification, their structural equivalents, or a combination of one or more thereof. The implementations of the subject matter described in this specification may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, which are executed by a data processing device or used to control the operation of the data processing device. The program instructions may be encoded on an artificially generated propagated signal—e.g., a machine-generated electrical, optical, or electromagnetic signal—generated to encode information for transmission to a suitable receiver device for execution by the data processing device. A computer storage medium may be, or may be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Furthermore, while a computer storage medium is not a propagated signal, a computer storage medium may include a source or destination of computer program instructions encoded in an artificially generated propagated signal. A computer storage medium may also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0141] The features disclosed herein can be implemented on a smart TV module (or a connected TV module, a hybrid TV module, etc.), which may include a processing module configured to integrate an Internet connection with a more traditional TV program source (e.g., via cable, satellite, air or other signals). The smart TV module may be physically assembled into a TV set, or may include a separate device such as a set-top box, a Blu-ray or other digital media player, a game console, a hotel TV system, and other supporting devices. The smart TV module may be configured to allow viewers to search and find videos, movies, photos, and other content on the Internet, on local cable channels, on satellite TV channels, or stored on a local hard drive. A set-top box (STB) or a set-top unit (STU) may include an information appliance device, which may include a tuner and be connected to a TV set and an external signal source, converting the signal into content that is subsequently displayed on a TV screen or other display device. The smart TV module may be configured to provide a home screen or top screen including icons for multiple different applications such as a web browser and multiple streaming media services, connected cable or satellite media sources, other network "channels," etc. The smart TV module may also be configured to provide an electronic program guide to the user. A companion application for the Smart TV module may be operable on the mobile computing device to provide the user with additional information about available programming, thereby allowing the user to control the Smart TV module, etc. In alternative embodiments, these features may be implemented on a laptop or other personal computer, smartphone, other mobile phone, handheld computer, tablet PC, or other computing device.

[0142] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0143] The terms "data processing device," "data processing system," "user device," or "computing device" encompass all types of devices, apparatuses, and machines for processing data, including, for example, a programmable processor, a computer, a system-on-chip, or multiple or a combination thereof. The device may include specialized logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, such as processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The device and execution environment may implement various computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. The script provider module 130 and the activity management module 140 may include or share one or more data processing devices, computing devices, or processors.

[0144] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, descriptive or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or code). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0145] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and devices can be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0146] For example, processors suitable for executing computer programs include general-purpose microprocessors and special-purpose microprocessors and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for performing actions according to instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include a large-capacity storage device for storing data, such as a magnetic disk, a magneto-optical disk, or an optical disk, or be operatively coupled to the large-capacity storage device to receive data from the large-capacity storage device or transfer data to the large-capacity storage device, or both. However, a computer does not necessarily have such a device. In addition, for example, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0147] To provide for interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device, such as a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, with which the user can provide input to the computer. Other types of devices may also be used to provide for interaction with the user; for example, feedback provided to the user may include any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and any form of input may be received from the user, including sound, speech, or tactile input. Additionally, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0148] Implementations of the subject matter described in this specification may be implemented in a computer that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a client computer having a graphical user interface or a web browser through which a user can interact with implementations of the subject matter described in this specification, or any combination of such back-end components, middleware components, or front-end components. The components in the system may be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., point-to-peer networks).

[0149] A computing system such as system 600 or system 110 may include a client and a server. For example, data processing system 110 may include one or more servers in one or more data centers or server farms. The client and server are typically remote from each other and typically interact via a communication network. The relationship between client and server arises from the fact that computer programs run on respective computers and have a client-server relationship with each other. In some embodiments, a server sends data (e.g., an HTML page) to a client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input from the user). Data generated at the client device (e.g., a result of a user interaction) may be received from the client device at the server.

[0150] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or the scope of what may be claimed, but rather as descriptions of features specific to particular implementations of the systems and methods described herein. Certain features described in this specification in the context of separate implementations may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually in multiple implementations or in any suitable subcombination. Furthermore, although features may be described above as being in a particular combination or even as initially claimed actions, in some cases one or more features in a claimed combination may be practiced with that combination, and a claimed combination may involve subcombinations or variations of subcombinations.

[0151] Similarly, while operations may be depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown, or in sequential order, or that all illustrated operations be performed, in order to achieve the desired results. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the particular order shown, or in sequential order, to achieve the desired results.

[0152] In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can be integrated together in a single software product or packaged into multiple software products. For example, the script provider module 130 and the activity management module 140 can be part of the data processing system 110, a single module, a logical device having one or more processing modules, one or more servers, or part of a search engine.

[0153] Now that some exemplary embodiments and implementations have been described, it will be apparent that the foregoing is illustrative and not restrictive, and has been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method actions or system elements, these actions and those elements can be combined in other ways to achieve the same goals. Actions, elements, and features discussed in connection with only one embodiment are not intended to be excluded from similar effects in other embodiments.

[0154] The phraseology and terminology used herein are for descriptive purposes and should not be construed as limiting. The use of "including," "comprising," "having," "containing," "involving," "characterized by," "characterized by," and variations thereof herein are intended to encompass alternative embodiments consisting of each of the items listed thereafter, their equivalents and additions, and the exclusive list thereof. In one embodiment, the systems and methods described herein consist of one, more than one, or all of the elements, acts, or components described.

[0155] Any reference herein to an embodiment, element, or action of a system or method expressed in the singular may also encompass embodiments that include a plurality of such elements, and any plural reference herein to any embodiment, element, or action may also encompass embodiments that include only a single element. References in the singular or plural are not intended to limit the presently disclosed systems or methods, their components, actions, or elements to singular or plural configurations. References to any action or element that is based on any information, action, or element may include embodiments in which the action or element is based, at least in part, on any information, action, or element.

[0156] Any embodiment disclosed herein may be combined with any other embodiment, and references to "an embodiment," "some embodiments," "alternative embodiments," "various embodiments," "one embodiment," etc. are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment. These terms as used herein are not necessarily all referring to the same embodiment. Any embodiment may be combined, inclusively or exclusively, with any other embodiment in any manner consistent with the aspects and embodiments disclosed herein.

[0157] References to "or" may be construed as inclusive such that any term described using "or" may refer to any of a single, more than one, and all of the described items.

[0158] Where reference numerals are appended to technical features in the drawings, detailed description, or any claims, such reference numerals are included solely for the purpose of enhancing the intelligibility of the drawings, detailed description, and claims. Therefore, neither the reference numerals nor their absence shall have any limiting effect on the scope of any claim.

[0159] The system and method described herein may be implemented in other specific forms without departing from its characteristics. Although the examples provided herein relate to controlling the display of content of information resources, the system and method described herein may include applications in other environments. The above embodiments are exemplary rather than limiting the described system and method. The scope of the system and method described herein is therefore indicated by the appended claims rather than the preceding description, and changes falling within the meaning and scope of equivalents of the claims are included therein.

Claims

1. A system for automatically generating remarketing lists based on sessions, comprising: One or more processors and memory, the one or more processors configured to: identifying one or more client sessions, each of the one or more client sessions corresponding to a client device accessing one or more information resources; determining, from each of the one or more client sessions, a plurality of information resource pairs, each of the plurality of information resource pairs comprising a destination information resource and a reference information resource from which the client device accesses the destination information resource; identifying, using the one or more client sessions, an interaction metric for each of the plurality of information resource pairs based on a number of client devices accessing a respective destination information resource of the information resource from a corresponding reference information resource of the information resource pair; selecting a subset of information resource pairs from the plurality of information resource pairs, each information resource pair in the subset of information resource pairs having a corresponding interaction metric that satisfies a threshold; generating a client identifier list comprising one or more client device identifiers, each client device identifier being associated with a respective client session of the one or more client sessions that accessed a corresponding reference information resource of at least one information resource pair of the subset of information resource pairs; responsive to receiving a request for content from a first client device having a client device identifier included in the client identifier list, selecting a content item corresponding to the destination information resource of the at least one information resource pair for display at the client device; and The content item is transmitted to the client device for display.

2. The system according to claim 1, wherein: The one or more processors are further configured to maintain an association between the content item and the list of client identifiers in one or more data structures in response to a request from a content provider of the content item.

3. The system according to claim 1, wherein: The one or more processors are further configured to receive one or more of an identification of a destination information resource, an identification of a reference information resource, or a number of times a client device has accessed the destination information resource and the reference information resource.

4. The system according to claim 1, wherein: The one or more processors are further configured to limit the selection from the subset of information resource pairs to at least one information resource pair associated with a corresponding interaction metric below the threshold.

5. The system according to claim 1, wherein The one or more processors are further configured to identify a number of information resource pairs that include a same destination information resource and a different reference information resource.

6. The system according to claim 5, wherein: The one or more processors are further configured to identify a subset of the information resource pairs based on a number of the plurality of information resource pairs including the same destination information resource and different reference information resources exceeding the threshold.

7. The system according to claim 1, wherein: In identifying the subset of information resource pairs, the one or more processors are further configured to: Determining click-through rate thresholds; and One or more information resource pairs including the same referencing node and having a click-through rate greater than the click-through rate threshold are identified as part of the subset of information resource pairs.

8. The system according to claim 1, wherein: In identifying the subset of information resource pairs, the one or more processors are further configured to: determining weight values ​​of click-through rates between the reference information resource and the destination information resource of the plurality of information resource pairs; and A subset of the information resource pairs is identified based on weight values ​​of click-through rates between a referencing information resource and a destination information resource of the plurality of information resource pairs.

9. The system according to claim 1, wherein: The one or more processors identify, for each of the plurality of information resource pairs, that the interaction metric is based on a hierarchical model of pages of a website.

10. The system according to claim 9, wherein: The one or more processors identify the subset of pairs of information resources based on the hierarchical model of the pages of the website.

11. A method for automatically generating a remarketing list based on a session, comprising: identifying, by a data processing system including one or more processors and memory, one or more client sessions, each of the one or more client sessions corresponding to a client device accessing one or more information resources; determining, by the data processing system, a plurality of information resource pairs from each of the one or more client sessions, each information resource pair in the plurality of information resource pairs comprising a destination information resource and a reference information resource, the client device accessing the destination information resource from the reference information resource; identifying, by the data processing system, an interaction metric for each of the plurality of information resource pairs based on a number of client devices accessing a respective destination information resource of an information resource from a corresponding reference information resource of the information resource pair using the one or more client sessions; selecting, by the data processing system, a subset of information resource pairs from the plurality of information resource pairs, each information resource pair in the subset of information resource pairs having a corresponding interaction metric that satisfies a threshold; generating, by the data processing system, a client identifier list comprising client device identifiers, each client device identifier being associated with a respective client session of the one or more client sessions that accessed a corresponding reference information resource of at least one information resource pair of the subset of information resource pairs; selecting, by the data processing system, a content item corresponding to the destination information resource of the at least one information resource pair for display at the client device in response to receiving a request for content from the client device having a client device identifier included in the client identifier list; as well as The content item is transmitted by the data processing system to the client device for display.

12. The method of claim 11, further comprising maintaining an association between the content item and the list of client identifiers in one or more data structures in response to a request from a content provider of the content item.

13. The method of claim 11, further comprising receiving, by the data processing system, one or more of an identification of a destination information resource, an identification of a reference information resource, or a number of times a client device has accessed the destination information resource and the reference information resource.

14. The method of claim 11, further comprising limiting, by the data processing system, the selection from the subset of information resource pairs to at least one information resource pair associated with a corresponding interaction metric below the threshold.

15. The method of claim 11, further comprising identifying, by the data processing system, a number of information resource pairs that include a same destination information resource and different reference information resources.

16. The method according to claim 14, wherein Identifying the subset of information resource pairs is based on a number of the plurality of information resource pairs including the same destination information resource and different reference information resources exceeding the threshold.

17. The method according to claim 11, wherein Identifying the subset of the information resource pairs further comprises: determining, by the data processing system, a click-through rate threshold; and One or more information resource pairs including the same referencing node and having a click-through rate greater than the click-through rate threshold are identified by the data processing system as part of the subset of the information resource pairs.

18. The method according to claim 11, wherein Identifying the subset of the information resource pairs further comprises: Determining, by the data processing system, weight values ​​of click-through rates between a reference information resource and a destination information resource of the plurality of information resource pairs; and A subset of the information resource pairs is identified by the data processing system based on weight values ​​of click-through rates between a referencing information resource and a destination information resource of the plurality of information resource pairs.

19. The method according to claim 11, wherein Identifying the interaction metric for each of the plurality of information resource pairs is based on a hierarchical model of pages of a website.

20. The method according to claim 19, wherein Identifying the subset of the information resource pairs is based on the hierarchical model of the pages of the website.

Images