A Machine Learning Architecture for Defining End-User Audiences for Automated Online Content Selection
The system addresses irrelevant content delivery by using contextual terms and machine learning to define audiences, ensuring relevant content distribution and reducing computational overhead.
Patent Information
- Application Number
- JP2024532747
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-01
- Filing Date
- 2022-11-21
- Publication Date
- 2024-11-06
AI Technical Summary
Existing content distribution methods, such as keyword targeting and behavioral targeting, often result in irrelevant content delivery due to reliance on predefined groups and user input, limiting relevance and efficiency.
A system that uses contextual terms to define audiences by processing web pages to extract topic terms, applying machine learning to identify relevant terms, and associating users with these terms, allowing for dynamic content distribution based on user behavior and context.
Enables contextually relevant content delivery by identifying user interests through contextual terms, reducing computational resources, and improving human-computer interaction.
Smart Images

Figure 2024541109000001_ABST
Abstract
Description
[Technical field]
[0001] This application relates generally to systems and methods for defining audiences for content selection, and more particularly, to enabling content providers to efficiently identify audiences that have contextually relevant interests in particular types of content. [Background technology]
[0002] Prior art systems use keyword targeting to enable content from a content campaign to be matched with a content distributor based on a set of keywords. For example, a search engine may sell ads that appear when a search request contains a specified keyword. This basic method provides a relatively poor experience where the content distributed is often irrelevant to the search. For example, a content generator wanting to distribute information related to apple pies and bakeries may want to search and publish content on websites based on the keyword "apple," but because such sites also contain the keyword "apple," the content may be mistakenly placed on a technology or business website that discusses Apple®, a technology company of the same name.
[0003] A second method for placing advertisements may include behavioral targeting. Under this method, a content generator (e.g., an advertiser) may specify retargeting of online advertisements to users identified as having performed a conversion. Users with similar conversion characteristics (e.g., purchasing an item in the previous three months) may be grouped together. Users in the group may then be matched with specific online content for delivery. However, in this method, the content generator is limited to selecting only from predefined groups, and there are competing incentives between the content generator and the content publisher / provider, such that the content publisher / provider may want to place content, articles, or other material in as many or as few groups as possible, which may overly limit or overly satisfy the delivery of collocated content belonging to the content generator. Additionally, because there is only a limited number of groupings, the content delivered to the user may have limited relevance.
[0004] A content selection service (e.g., a placement bidding service) may distribute and deliver content on behalf of a content creator according to a content campaign according to keyword targeting. Under one approach, upon detecting input of a particular term in a search engine from a user, the service may provide content (e.g., an online advertisement) to the user based on the term. While this approach can deliver targeted content that is relevant to the user, the approach may depend on the user explicitly inputting a particular keyword into the search engine. Because of the dependency on specific search terms, this approach for keyword targeting for content delivery may be difficult to apply in situations where a user does not or cannot input a term. Summary of the Invention
[0005] What is desired is a manner for providing context and keyword targeted online content distribution independent of a search engine environment. The systems and methods disclosed herein are intended to address these shortcomings, but may also provide additional or alternative advantages. As described herein, the systems and methods may use context terms to define an audience to which the online content is provided. The audience may be defined and generated by a computing device according to user-inputted context terms, and the audience may represent users who have previously accessed a web page that includes topic words or phrases that are contextually relevant to the online content, and thus, the audience users may be targeted to provide the online content when viewing the web page. These systems and methods allow advertisers to discover groups of users to whom they can provide online content.
[0006] The service may collect and process online content. The service may aggregate content objects (e.g., web pages) from a content stream and process each of the content objects into topic terms. Once processed, the service may store the topics of each piece of content in a database. Each content object may be associated with a set of topic terms that are derived from the content object itself.
[0007] Upon determining the topic terms, the service may associate users with the topic terms. The service may associate users with content objects in the content stream and then use the associations to identify various topic terms to associate users with. For example, a first user may be associated with a first content object and a second content object, and a second user may be associated with the first content object. From these associations, the service may associate the first user with topic terms of the first content object and the second content object, and associate the second user with topic terms of the first content object.
[0008] Together, the service may receive in-context and out-of-context terms from the content provider via a user interface. The service may access a corpus database with a large set of terms that have been determined (either automatically by a computer or as indicated by user input) to be related to or co-occurring with each other in the corpus. The service may identify other contextual terms that are relevant to the in-context terms and filter the contextual terms using the out-of-context terms. With the identification of additional context terms, the service may determine an implication score for each topic term. Based on the implication score, the service may determine which topic terms are relevant to the context term. For example, the service may select the topic term with the highest implication score.
[0009] The service may generate content summaries for content objects associated with topic terms. To do so, the service may identify various phrases around occurrences of the topic terms in each associated content object. The service may apply a natural language processing model (e.g., a neural network or a knowledge graph) to determine the semantic relevance of the phrases to the topic terms. If the identified phrases are determined to be relevant, the service may select the phrases to use as the content summary for the content object.
[0010] Using the topic terms, the service may determine an audience that defines a set of users for the selected topic term. The service may provide the topic terms to a content provider's user interface for selection along with content summaries. The service may identify users associated with the selected topic terms and rank the users according to a relevance score, such as the number of times the users have accessed the content object from which the topic term was derived. The service may identify a subset of users with the highest relevance scores and use the subset as the audience for the topic term.
[0011] By definition, a service may use an audience to identify opportunities to deliver online content (e.g., online advertisements, web pages, and videos) for a content provider. When a user accesses a web page, the service may receive a request for one or more content selection parameters (e.g., a bid value) for inserting content into the web page. From the request, the service may determine that the user corresponds to one of the users defined in the content provider's audience. In response to this determination, the service may send the content provider's content selection parameters to a content exchange server. Using the parameters, the exchange server may perform a content exchange process (e.g., a real-time bidding process) to identify which content provider to select for content to be inserted into the web page.
[0012] In this way, the service may enable content providers to define an audience around contextually relevant terms. The audience definition may enable a selection of relevant content to be provided to end users. As a result, end user devices may consume less computational resources, such as processors, memory, and network bandwidth, in discovering relevant content. Additionally, human-computer interaction (HCI) between operators and end user devices may be improved.
[0013] Aspects of the present disclosure relate to a system, method, and non-transitory computer-readable medium for determining an audience for content delivery to determine users based on a database of a web page corpus for dynamically placing content on available web pages hosted by a third-party web server. A server may receive a context term from a client device, the context term identifying an additional term. The server may generate a set of context terms including the context term by selecting one or more of a plurality of corpus terms having a relationship with the context term from a corpus database, the corpus database storing text extracted from a plurality of past web pages. The server may calculate an implication score for each of a plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of past web pages associated with the topic term. The server may select a particular topic term from the plurality of topic terms based on the implication score of the particular topic term. The server may determine an audience representing a set of users who accessed at least one of the set of past web pages associated with the topic term. The server may then store campaign data in a campaign database, including a set of users, a set of context terms, and an audience representing the particular topic terms, and the campaign data is configured to perform a real-time bid selection operation during a real-time bid selection operation on one or more available web pages being accessed by users of the audience hosted by one or more third-party servers.
[0014] In some embodiments, the server may determine a plurality of topic terms from a plurality of terms on a plurality of web pages accessed by a plurality of users. In some embodiments, the server may maintain an association between each user of the plurality of users based on the user having accessed at least one of the plurality of web pages associated with the topic term. In some embodiments, the server may determine an audience using the association between each user of the plurality of users and the topic term.
[0015] In some embodiments, the server may identify a plurality of users of the topic term based on the number of times each user of the plurality of users accessed at least one of a plurality of web pages associated with the topic term. In some embodiments, the server may identify a user identifier from one or more users of the audience from the request for the selection value. In some embodiments, the server may transmit the content provider's selection value to a content exchange server in response to identifying the user identifier.
[0016] In some embodiments, the server may receive from the client device an audience size that defines a number of users to be selected from the plurality of users for the audience. In some embodiments, the server may determine an audience that identifies one or more users from the plurality of users based on the audience size.
[0017] In some embodiments, the server may generate multiple phrases of the topic term using the multiple terms on the multiple web pages from which the topic term is determined. In some embodiments, the server may calculate an implication score based on a number of occurrences of at least one of the set of context words on at least one of the multiple web pages associated with the topic term.
[0018] In some embodiments, the server may select a first subset of corpus terms from the plurality of corpus terms based on the context terms and remove a second subset of corpus terms from the first subset of corpus terms using the out-of-context terms received from the client device. In some embodiments, the server may transmit the plurality of topic terms for display on a graphical user interface (GUI) of the client device.
[0019] Both the foregoing general description and the following detailed description are examples and explanations and are intended to provide further explanation of the invention as claimed.
[0020] The accompanying drawings constitute a part of this specification, illustrate embodiments of the invention, and together with this specification explain the invention. [Brief description of the drawings]
[0021] [Figure 1] 1 illustrates a block diagram of a distributed computer system for audience determination for content delivery, according to one embodiment. [Diagram 2] 1 illustrates a flowchart of a method for determining an audience for content distribution, according to one embodiment. [Diagram 3] 1 illustrates an exemplary web page that allows a user to determine an audience for content distribution, according to one embodiment. [Figure 4] 1 illustrates an exemplary web page that allows a user to determine an audience for content distribution, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] Reference will now be made to various illustrated embodiments, where specific language will be used to describe the same. Nevertheless, it will be understood that no limitations of the claims or the present disclosure are intended thereby. Alterations and further modifications of the features of the invention illustrated herein, as well as additional applications of the principles of the subject matter illustrated herein, that would occur to one skilled in the relevant art and in possession of this disclosure, should be considered within the scope of the subject matter disclosed herein. The present disclosure will now be described in detail with reference to the embodiments illustrated in the drawings forming a part hereof. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the present disclosure. The illustrative embodiments described in the detailed description are not meant to limit the subject matter presented herein. It should be understood that the embodiments described herein are merely illustrative for the purpose of illustrating the techniques, technical components, and processes disclosed herein. However, some embodiments may implement aspects of the techniques of the present disclosure for other purposes or situations.
[0023] 1 depicts a block diagram of a distributed computer system 100 for determining audiences for content delivery. The system 100 includes, among other things, one or more placement servers 105 of a placement service, content provider devices 110 (sometimes referred to herein as client devices) associated with content providers preparing content delivery campaigns, a third party server 115, a content exchange server 120 (sometimes referred to herein as real-time bidding (RTB) servers), and end user devices 125a-125n (collectively referred to as end user devices 125), communicatively coupled to each other via one or more networks 130. The placement server 105 includes software programming that defines or otherwise functions as a web page indexer 135, a topic generator 140, an association evaluator 145, a term expander 150, an implication evaluator 155, a phrase generator 160, an audience generator 165, and a selection handler 170. The deployment server 105 includes or is in communication with a database 175. Each of the components in the system 100 may execute various software programming on one or more hardware processors coupled with memory to perform various operations. Particular components of the system 100 may be embodied in a single computing device or multiple computing devices.
[0024] Network 130 includes any number of private or public networks for hosting and conducting electronic communications between the electronic devices of system 100. Network 130 may include a telecommunications network or a data communications network comprising hardware or software components for exchanging data between the devices of system 100 pursuant to any number of telephony or networked communications.
[0025] The computing devices of system 100 (e.g., servers, end user devices 125, content provider devices 110) may include one or more computing devices and any type of computing device with hardware and software components configured to perform various processes and tasks described herein, including one or more processors or software that includes machine-executable instructions executed by one or more processors. Non-limiting examples of such computing devices of system 100 include server computers, laptop computers, desktop computers, tablet computers, and smartphone mobile devices, among others. One or more servers (or other devices of system 100), such as placement server 105, execute web server software for hosting one or more web pages according to web-related or data communication protocols and computing languages.
[0026] The web page indexer 135 executing on the deployment server 105 may collect, aggregate, or retrieve web pages hosted by the web server software executing on the third party server 115. Each web page is an online document in a markup language (e.g., Hypertext Markup Language (HTML)) that is stored or hosted by the web server software and database of the third party server 115 and displayed on the end user device 125. In retrieving, the web page indexer 135 may extract and download various types of data associated with each web page. The data extracted from the web page may include, for example, metadata, fingerprints, tags, scripts, images, text, and other content thereon. In addition, the web page indexer 135 may retrieve or identify a page identifier (e.g., Uniform Resource Locator (URL), web address) corresponding to the web page. Once identified, the web page indexer 135 may store and maintain the data extracted from the web page in the database 175. The web page indexer 135 may also store and maintain in a database 175 an association between the data and the page identifiers of the web pages.
[0027] The topic generator 140 executing on the placement server 105 may extract, identify, or otherwise determine a set of topic terms from the web pages of the third-party server 115. Each topic may define or identify the semantic meaning or subject matter of the content in the web page. For example, for a web page that includes content with a large number of words related to bicycles, the set of topic terms may include "bicycles." In some embodiments, the topic generator 140 may apply a machine learning architecture to the web pages to determine a set of topic terms for each web page. The machine learning architecture may include, for example, natural language processing algorithms, information retrieval models, topic models, tokenization models, and latent semantic analysis, among others. The topic generator 140 may scan the data extracted from the web pages indexed on the database 175. For the data from each indexed web page, the topic generator 140 may determine a set of topic terms for each web page from the machine learning architecture. Upon determination, the topic generator 140 may store and maintain an association between the set of topic terms and the page identifier of the web page in the database 175.
[0028] The association evaluator 145 executing on the placement server 105 may generate, determine, or otherwise identify an association between each end user device 125 and a web page. The association may identify an instance in which the end user device 125 accessed the web page. In generating, the association evaluator 145 may identify a device identifier of the end user device 125 that accessed the web page indexed on the database 175. For example, the web page indexer 135 may track the end user device 125 that accessed the web page using a cookie on the end user device 125. The identifier may be a network address, such as an Internet Protocol (IP) address or a Media Access Control (MAC) address, or a unique user identifier, such as a device identifier, account identifier, or an Identifier for Advertisers (IDFA), among others. The association evaluator 145 may store and maintain in the database 175 an association between the data or page identifier of the web page along with the device identifier of the end user device 125 that accessed the web page.
[0029] Additionally, the association evaluator 145 may generate, determine, or otherwise identify an association between each particular end user device 125 or user and one or more of the topic terms extracted from the web page. The association evaluator 145 identifies topic terms derived from the web pages indexed in the database 175. For each web page, the association evaluator 145 may also identify each user device 125 that accessed the web page. Through each web page, the association evaluator 145 may determine an association between a particular end user device 125, the web page, and the topic terms derived from the web page. Based on the determination, the association evaluator 145 may store and maintain the associations between the topic terms and each end user device 125 in the database 175.
[0030] In conjunction, the content provider device 110 may communicate and interface with the placement server 105 to define new content delivery campaigns or update previously defined delivery campaigns, including defining audiences of end users to be targeted for the content delivery campaign. The content provider device 110 may detect, identify, or otherwise receive a set of inputs from a user to define the user's content delivery campaign. Using the content provider device 110, the user accesses an audience (or campaign) configuration web page hosted by the placement server 105. The audience configuration web page may, for example, allow the user to enter inputs that configure and define the user's content delivery campaign.
[0031] Examples of audience configuration web pages are shown in Figures 3-4. The inputs for defining a campaign (or audience) may indicate a set of in-context terms or phrases and a set of out-of-context terms or phrases (collectively referred to as contextual terms), which may indicate in-context or out-of-context terms or phrases with respect to the target audience of the content delivery campaign. The in-context terms indicate words or phrases for which additional contextual terms are discovered or identified. The out-of-context terms may identify or include words or phrases that are used to filter or remove one or more of the additional contextual terms discovered using the in-context terms. For example, in Figure 3, the in-context terms include "Thor" and the out-of-context terms include "Marvel" to discover additional terms related to Norse mythology while avoiding references to comic book characters. The content provider device 110 sends a set of inputs including the in-context terms and out-of-context terms to the placement server 105.
[0032] The term expander 150 executing on the placement server 105 may initialize, train, or establish at least one machine learning architecture to generate additional context terms (sometimes referred to herein as beacon terms) using a corpus of documents (e.g., web pages, electronic documents) stored in an external or internal database (e.g., database 175). In general, the machine learning architecture receives a set of inputs corresponding to in-context and out-of-context terms and outputs a set of additional context terms and a set of word implications (or weights) that relate the inputs to outputs. The machine learning architecture includes any processor-executed machine learning techniques and algorithms, such as various types of neural networks (e.g., convolutional neural networks (CNNs), deep neural networks (DNNs)), linear regression, logistic regression, k-means, k-nearest neighbors (kNNs), or support vector machines (SVMs), among others. The corpus may be stored and maintained in the database 175 and may include a set of documents (e.g., web pages, articles, or other content) that include any number of potential terms and phrases. The set of documents used to train the machine learning architecture may be different from the web pages indexed on database 175. In some embodiments, term expander 150 may also use additional natural language processing and vectorized machine learning algorithms trained on the corpora maintained on database 175.
[0033] Using the corpus, the term expander 150 trains a machine learning architecture to calculate and determine various statistical associations between terms, phrases, and other latent information in the corpus. The statistical associations (sometimes referred to herein as relationships) may include or identify, among others, the probability of co-occurrence between any pair of terms or phrases in a set of documents in the corpus, the conditional likelihood (e.g., n-grams), and the distance between pairs of terms and phrases in the set of documents, or any combination thereof. To train, the term expander 150 may apply terms and phrases from the set of documents to the machine learning architecture. By providing the terms and phrases, the term expander 150 may process the terms and phrases using a set of weights in the machine learning architecture to generate additional context terms. The term expander 150 may calculate an error or loss metric for the additional context terms by comparing them to a measure derived from the set of terms in the corpus. The term expander 150 may iteratively set, adjust, or update the set of weights in the machine learning architecture based on the loss metric until the machine learning architecture reaches convergence. By adjusting the set of weights, the term expander 150 can update the machine learning architecture to produce additional context terms that reflect statistical associations between terms within a set of documents of the corpus on the database 175.
[0034] The term expander 150 may use the set of in-context terms and the set of out-of-context terms received from the content provider device 110 to generate, determine, or identify additional context terms. In some embodiments, the term expander 150 may apply a machine learning architecture to the in-context terms and the out-of-context terms. In applying, the term expander 150 may provide the in-context terms and the out-of-context terms as inputs to the machine learning architecture. The term expander 150 may then process the input terms according to a set of weights in the machine learning architecture to generate additional context terms. In processing, the term expander 150 may select a set of terms from the corpus that have statistical associations with the in-context terms. The statistical associations may meet a threshold amount for inclusion (e.g., threshold co-occurrence). From the initial subset, the term expander 150 may filter, reduce weights, or otherwise remove a subset of terms that have statistical associations with the out-of-context terms, and may use the remaining set of terms as additional context terms. The statistical associations may also meet a threshold amount for removal (e.g., threshold co-occurrence). Upon generation, the term expander 150 may include additional terms as part of the set of in-context terms for use in the audience definition.
[0035] In some embodiments, the term expander 150 may provide, transmit, or send the additional context terms to the content provider device 110. The content provider device 110 may then obtain, identify, or otherwise receive the additional context terms from the placement server 105. The content provider device 110 may present or display the additional context terms in a graphical user interface used to input a set of in-context and out-of-context terms. The content provider device 110 may detect, identify, or receive a selection of one or more of the additional context terms via a graphical user interface for defining a content delivery campaign. For example, a user on the content provider device 110 may click on a subset of the additional context terms to facilitate definition of an audience for the content delivery campaign. Upon selection, the content provider device 110 may provide, transmit, or transmit the selection of the additional context terms to the placement server 105.
[0036] The implication evaluator 155 executing on the placement server 105 may generate, determine, or otherwise calculate an implication score for each topic term determined from the web page. The implication score may be based on statistical associations (e.g., co-occurrence, conditional probability, and distance) between the topic term and one or more of a set of in-context terms including additional context terms on the web page on which the topic term is generated. In some embodiments, the implication score of a topic term may be based on the number of occurrences of at least one of the context terms in the web page on which the topic term occurs. For each topic term, the implication evaluator 155 may identify a set of web pages indexed on the database 175 from which the topic term is derived. On each identified web page, the implication evaluator 155 may determine or identify the number of occurrences of the context term. Using the number of occurrences, the implication evaluator 155 may determine an implication score for the topic term.
[0037] Based on the suggestion scores of the set of topic terms, the suggestion evaluator 155 may identify or select one or more of the topic terms. The selection of the topic terms may be used to define an audience for a content distribution campaign. In general, the higher the suggestion score, the more relevant the topic term may be to the context terms generated from the in-context and out-of-context terms. Conversely, the lower the suggestion score, the less relevant the topic term may be to the context terms generated from the in-context and out-of-context terms. In some embodiments, the suggestion evaluator 155 may rank the set of topic terms by the corresponding suggestion scores. From the ranking, the suggestion evaluator 155 may select the N topic terms with the highest suggestion scores. In some embodiments, the suggestion evaluator 155 may compare the suggestion scores of the set of topic terms to a threshold score. The threshold score may define a value of the suggestion score at which the associated topic term is selected. When the suggestion score of a topic term meets the threshold score, the suggestion evaluator 155 may select the topic term. Alternatively, if the topic term's implicature score does not meet the threshold score, the implicature evaluator 155 may exclude the topic term. Optionally, the implicature evaluator 155 may provide, transmit, or send the set of topic terms to the content provider device 110.
[0038] In conjunction, the phrase generator 160 running on the placement server 105 may determine or generate a set of phrases for the topic terms using content within the web pages indexed on the database 175. In some embodiments, the generation of phrases may be for the selected topic terms based on their suggestive scores. To do so, the phrase generator 160 may identify web pages in which the topic terms occur and may identify the location of the topic terms within the web pages. On each identified web page, the phrase generator 160 may extract or identify one or more phrases having at least one term within a set distance (e.g., number of words) from the topic term. Upon identification, the phrase generator 160 may filter the set of phrases by applying a machine learning architecture to determine the semantic relevance of each phrase with the topic term. The machine learning architecture may be a natural language processing algorithm or model such as knowledge graphs, information retrieval, automatic summarization (e.g., PEGASUS), transformer language models (e.g., bidirectional encoder representations from transformers (BERT)), and semantic similarity networks, among others. The phrase generator 160 may select one or more phrases from applying the machine learning architecture to the phrases. Upon selection, the phrase generator 160 may provide, send, or transmit the identified phrases to the content provider device 110.
[0039] In addition, the phrase generator 160 may apply a machine learning architecture to the set of extracted phrases to generate a new set of phrases that is different from the set of phrases identified from the content of the indexed web pages. The new set of phrases may lack keywords found in the content of the web pages on the database 175 and may be newly generated by applying the machine learning architecture. In some embodiments, the phrase generator 160 may embed information from the set of extracted phrases into a high-dimensional vector space for the machine learning architecture. Using the embeddings from the set of extracted phrases, the phrase generator 160 may generate a new set of phrases according to the machine learning architecture. In general, the new set of phrases may contain similar information as the set of extracted phrases. For example, the phrase generator 160 may apply an abstraction summarization algorithm (e.g., PEGASUS) to the set of extracted phrases to generate a new set of phrases.
[0040] The content provider device 110 may obtain, identify, or otherwise receive a set of topic terms from the placement server 105. The content provider device 110 may present or display the set of topic terms on a graphical user interface used to define a content delivery campaign. The topic terms may be used to define an audience for the content delivery campaign. The content provider device 110 may detect, identify, or receive a selection of topic terms via a graphical user interface for defining a content delivery campaign. For example, the topic terms may be presented in the form of a word cloud with in-context terms on the graphical user interface. Using the interface, a user on the content provider device 110 may click on a subset of the topic terms to facilitate definition of an audience for the content delivery campaign. In some embodiments, the content provider device 110 may obtain, identify, or otherwise receive a set of phrases from the placement server 105. The content provider device 110 may display or present the set of phrases on a graphical user interface. The presentation of the phrases may assist a user of the content provider device 110 in selecting topic terms. In some embodiments, the content provider device 110 may present or display an element on a graphical user interface for defining an audience size. The audience size may identify a number of end user devices 125 included in the audience of the content delivery campaign. Through interaction with the element, the content provider device 110 may obtain, identify, or receive the audience size. Upon selection, the content provider device 110 may provide, send, or transmit the selection of topic terms to the placement server 105. The content provider device 110 may also transmit the audience size to the placement server 105.
[0041] The audience generator 165 executing on the placement server 105 may generate, identify, or determine an audience for the content delivery campaign based on the topic terms. The audience may define, specify, or identify one or more end user devices 125 covered by the content delivery campaign. In some embodiments, the audience generator 165 may determine the audience using topic terms selected via the content provider device 110. In some embodiments, the audience generator 165 may use a set of topic terms selected by the implication evaluator 155 without additional input from the content provider device 110. For each selected topic term, the audience generator 165 may access the database 175 to obtain, fetch, or identify an association between the topic term and an identifier of the end user device 125.
[0042] In some embodiments, the audience generator 165 may identify or select a subset of end user devices 125 identified for an audience of a topic term. The audience generator 165 may access a database 175 to identify web pages associated with the topic term. For each end user device 125, the audience generator 165 may determine a topic score based on the number of times the end user device 125 accessed the web page associated with the topic term. The identification may be based on the use of cookies to track the end user devices 125 that access the web page. In some embodiments, the audience generator 165 may rank the end user devices 125 identified by topic score. From the ranking, the audience generator 165 may select or identify the number of end user devices 125 with the highest topic score, as defined by audience size.
[0043] In some embodiments, the audience generator 165 may compare the number of accesses to a threshold. The threshold may define a value of the number of times to select the end user device 125 for the audience. If the topic score meets the threshold, the audience generator 165 may select the end user device 125 for the audience. Alternatively, if the topic score does not meet the threshold, the audience generator 165 may exclude the end user device 125 from the audience. Upon identification, the audience generator 165 may assign the identifier of the end user device 125 to the audience of the topic term. Upon assignment, the audience generator 165 may provide, send, or transmit the audience identifying identifier of the end user device 125 to the content provider device 110. The audience generator 165 may store and maintain the audience definition on the database 175. In some embodiments, the audience generator 165 may transmit related information, such as the topic score, in combination with the identifier of the end user device 125.
[0044] The content provider device 110, in turn, may obtain, identify, or receive the audience definition from the placement server 105. The content provider device 110 may present or display information about the audience definition on a graphical user interface to define a content delivery campaign. For example, the content provider device 110 may present the number, topic score, device type, and location of end user devices 125 in the audience, among others. The content provider device 110 may detect, identify, or receive one or more inputs to adjust the audience definition and transmit the adjustments to the placement server 105 to repeat the above operations. The content provider device 110 may also detect or receive an interaction to initiate a content delivery campaign according to the audience definition. Upon receipt, the content provider device 110 may provide, send, or transmit an indication to initiate to the placement server 105. The placement server 105 may then receive the indication and initiate a content delivery campaign according to the audience definition.
[0045] One of the end user devices 125 may then access a web page hosted on the third party server 115. The web page may include an element (e.g., an inline frame) that inserts content (e.g., an online advertisement) from an entity associated with the content provider device 110. Upon reading the element, the end user device 125 may generate a request for a selection value for inserting content into the element of the web page. The selection value may be used by the content exchange server 120 to select content (e.g., an online advertisement) from the content provider device 110 for placement on the web page accessed by the end user device 125. The request may include, among other information, an identifier of the end user device 125. The identifier of the end user device 125 may correspond to one of the identifiers in the audience definition. In some embodiments, the end user device 125 may send the request to the content exchange server 120, which may then forward the request to the placement server 105.
[0046] The selection handler 170 executing on the placement server 105 may obtain, receive, or otherwise identify a request for a selection value from an end user device 125. The request may be part of a bid stream, which may be a data stream of published URLs available for bids to content generators interested in placing campaign content at those URLs. Upon receipt, the selection handler 170 may parse the request for a selection value to extract or identify an identifier for the end user device 125. Upon identification, the selection handler 170 may compare the identifier to identifiers within an audience defined for the content delivery campaign of the content provider device 110.
[0047] From the comparison, the selection handler 170 may determine whether the identifier is part of an audience of the content provider device 110's content delivery campaign. If the identifier is not part of the audience, the selection handler 170 may refrain from providing a selection value. In some embodiments, the selection handler 170 may request a selection value from the content provider device 110 with an indication that the end user device 125 is not part of the audience. Conversely, if the identifier is part of the audience, the selection handler 170 may obtain, receive, or identify a selection value of the content provider device 110 associated with the audience. In some embodiments, the selection handler 170 may request a selection value from the content provider device 110 with an indication that the end user device 125 is part of the audience. The selection handler 170 may receive a selection value generated by the content provider device 110 for placement of content on a web page accessed by the end user device 125. In some embodiments, the selection handler 170 may access a database 175 to fetch, obtain, or identify a selection value of the content provider device 110. The database 175 may store and maintain selection values previously received from the content provider device 110. Upon identification, the selection handler 170 may provide, send, or transmit the selection values to the content exchange server 120.
[0048] The content exchange server 120 may manage competition among the content provider devices 110 to compete for the opportunity to deliver content on various web pages, such as web pages accessed by the end user devices 125. The content exchange server 120 may be any third-party external web service that exposes information (e.g., API services) and instructions (e.g., API requests) for performing various tasks described herein. Using the content selection values of the content provider devices 110, the content exchange server 120 may perform a content selection process (e.g., a real-time bidding auction) to select one of the content provider devices 110. For example, the content exchange server 120 may select the content provider device 110 with the highest content selection value. The content exchange server 120 may send an indication of the selection to the selected content provider device 110. The content provider device 110 may then send, provide, or transmit content to elements in the web pages accessed by the end user devices 125.
[0049] 2 depicts a flowchart of a method 200 for determining a targeted end-user audience for content delivery. For ease of explanation, a server performs various operations described for method 200, although in other embodiments, method 200 or particular aspects of method 200 may be performed by any number and / or types of computing devices. In some embodiments, for example, method 200 may be performed by one or more components described with respect to system 100 of FIG.
[0050] In step 205, the server aggregates web pages previously accessed by the user and possibly user data (e.g., end user device IP address, end user device MAC address) of end users who accessed these past web pages. The server stores the past web pages and user data in one or more databases, such as a corpus database or an end user database. Each web page includes an online computing file including machine executable code in a markup language hosted on a third party server. The server may extract various types of data from the web pages, including text, terms, or phrases. In step 210, the server may generate topic terms from the web pages in the corpus database. Each topic term may define or identify the semantic meaning or subject of the content in the web page. The server may apply natural language processing algorithms to the web pages to derive the topic terms. In step 215, the server may associate the topic terms with the users. The server may identify users who accessed the web pages from which the topic terms are derived. Upon identification, the server may associate the topic terms with the users.
[0051] In step 220, the server may receive the in-context terms and out-of-context terms. The in-context terms may identify or include words or phrases in which additional contextual terms are found or identified. The out-of-context terms may identify or include words or phrases used to filter or remove one or more of the additional contextual terms found using the in-context terms. The context terms may be entered via a graphical user interface on the client. In step 225, the server may generate additional context terms, thereby generating a set of campaign context terms including the in-context terms (entered by the user) and the additional context terms (dynamically generated by the server). The server may apply a machine learning architecture to the in-context terms and out-of-context terms to generate the additional context terms. The machine learning architecture may be trained and established using a set of multiple corpus terms. The server may select a subset from the set of corpus terms related to the in-context terms. The server may then use the out-of-context terms to filter the initial subset.
[0052] In step 230, the server may calculate suggestive scores for the topic terms. For each topic term, the server may determine an suggestive score based on a statistical relationship between the topic term and a set of campaign context terms. The statistical relationship may be a co-occurrence or a conditional probability of the context terms occurring when the topic term is present on the web page from which the topic term is derived. In step 235, the server selects one or more specific topic terms of interest that are relevant to the campaign based on the suggestive scores. The server may select a subset of these specific topic terms that have suggestive scores above a threshold.
[0053] In step 240, the server may determine an audience for the selected topic term. The audience represents past end users who accessed past web pages that contain a particular subset of topic terms. The server may identify the web pages from which the topic terms are derived and the users who accessed the web pages. The server may rank the users of the audience, for example, by the number of times the users accessed web pages associated with the topic terms. The server may select the top number of users to assign to the audience. In step 245, the server may transmit campaign data generated by the server regarding the audience of the user to the content provider's computing device. Non-limiting examples of the campaign data include the audience of the user, the user data of the users in the audience, in-context terms, out-of-context terms, the set of campaign context terms, and past web pages that contain the particular topic term with an implication score that exceeds a threshold. Additionally or alternatively, the server stores the campaign data in a database (e.g., campaign database, content provider database) on behalf of the content provider for later retrieval, review, update, or adjustment. The content provider may choose to use the audience definition to execute a content distribution campaign. In step 250, the server may use the audience in sending content selection values for placement on web pages of interest that contain a particular subset of topic terms. For example, the server may transmit a bid to an RTB server, which may identify bid requests for web pages of interest that are being accessed by users of the audience and place bids with the RTB server. From the requests, the server may identify users who are visiting the web pages of interest as part of the audience. The server may request bid values from content providers and send the bid values to the RTB server (or other content exchange server) for the real-time content selection process.
[0054] FIG. 3 depicts an exemplary web page 300 that allows a user to determine an audience for content delivery. As shown, the web page 300 may include a graphical user interface (GUI) element for inputting in-context terms (e.g., “Thor,” “Mythology,” and “Norse Mythology”) and out-of-context terms (e.g., “Marvel”) to define an audience. The web page 300 may include another GUI graphical user element for displaying relevant contextual terms (e.g., “Odin,” “Aesir,” and “Ragnar,” among others) in the form of a word cloud. The relevant contextual terms may include terms related to the in-context terms and terms filtered using the out-of-context terms. The web page 300 may include another GUI element for submitting contextually relevant terms for audience definition.
[0055] FIG. 4 depicts an exemplary web page 400 that allows a user to determine an audience for content delivery. As shown, the web page 400 may include a GUI element for displaying relevant contextual terms (e.g., “Odin,” “Aas,” and “Ragnar,” among others) generated from in-context and out-of-context terms in the form of a word cloud. The web page 400 may include a set of generally content summary phrases under the word cloud that includes the contextually relevant terms. The web page 400 may include another GUI element for defining audience size in the form of a slide with ranges of “small audience / high interest” and “large audience / low interest.” The web page 400 may include another set of GUI elements for specifying audience availability to various content providers (e.g., the content provider device 110).
[0056] In some embodiments, a computer-implemented method for determining a user based on a database of a web page corpus for dynamically placing content on available web pages hosted by a third-party web server. The method includes: receiving, by a server, a context term from the client device, the context term identifying the additional term; generating, by the server, a set of context terms including the context term by selecting, by a corpus database, one or more of a plurality of corpus terms having a relationship with the context term, where the corpus database stores text extracted from a plurality of past web pages; calculating, by the server, an implication score for a plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of past web pages associated with the topic term; selecting, by the server, a particular topic term from the plurality of topic terms based on the implication score of the particular topic term; determining, by the server, a viewer representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and storing, by the server, campaign data including the set of one or more users, the set of context terms, and the viewer representing the particular topic term in a campaign database, where the campaign data is configured to perform a real-time bid selection operation on available web pages being accessed by users of the viewer hosted by one or more third party servers during the real-time bid selection operation.
[0057] In some implementations, the method further includes determining, by the server, a plurality of topic terms from a plurality of terms on a plurality of past web pages accessed by the plurality of users.
[0058] In some implementations, the method further includes maintaining, by the server, an association between each user of the plurality of users based on the users having accessed at least one of the plurality of past web pages associated with the topic term. Determining the audience further includes identifying a set of one or more users of the audience using the association between each user of the plurality of users and the particular topic term.
[0059] In some implementations, the method further includes ranking, by the server, the plurality of users for the topic term based on a number of times each user of the plurality of users accessed at least one of a plurality of past web pages associated with the topic term.
[0060] In some implementations, the method further includes identifying, by the server, a user identifier from a set of one or more users of the audience from the request for the selection value, and transmitting, by the server, the content provider's selection value to a content exchange server in response to identifying, by the server, the user identifier.
[0061] In some implementations, the method further includes receiving, by the server, from the client device, an audience size defining a number of users to be selected from the plurality of users for the audience. Determining the audience includes identifying a set of one or more users of the audience from the plurality of users based on the audience size.
[0062] In some implementations, the method further includes generating, by the server, a plurality of phrases of the topic term using the plurality of terms on the plurality of past web pages from which the topic term is determined.
[0063] In some implementations, calculating the suggestive score further includes calculating the suggestive score based on a number of occurrences of at least one of the set of context terms on at least one of the multiple past web pages associated with the topic term.
[0064] In some implementations, generating the set of context terms further includes (i) selecting a first subset of corpus terms from the plurality of corpus terms based on the context terms; and (ii) removing a second subset of corpus terms from the first subset of corpus terms using the out-of-context terms received from the client device.
[0065] In some implementations, the method further includes transmitting, by the server, the plurality of topic terms for display on a graphical user interface (GUI) of the client device.
[0066] In some embodiments, a system for determining a user based on a database of a web page corpus for dynamically placing content on available web pages hosted by a third-party web server, the system includes a non-transitory medium including one or more databases including a corpus database configured to store a plurality of past web pages and a campaign database configured to store campaign data, and a server having at least one processor coupled with a memory. The server is configured to receive from the client device a context term used to identify the additional terms; generate a set of context terms including the context term by selecting, from text in the corpus database, one or more of a plurality of corpus terms having a relationship with the context term; calculate an implication score for the plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of past web pages associated with the topic term; select a particular topic term from the plurality of topic terms based on the implication score of the particular topic term; determine an audience representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and store campaign data in a campaign database, the campaign data being configured to perform a real-time bid selection operation on available web pages being accessed by users of the audience hosted by one or more third-party servers during the real-time bid selection operation.
[0067] In some implementations, the server is further configured to determine topic terms from terms on a plurality of past web pages accessed by a plurality of users.
[0068] In some implementations, the server is further configured to maintain an association between each of the plurality of users based on the users having accessed at least one of the plurality of past web pages associated with the topic term, and to determine an audience using the association between each of the plurality of users and a particular topic term.
[0069] In some implementations, the server is further configured to rank the plurality of users for the topic term based on the number of times each user of the plurality of users accessed at least one of the plurality of past web pages associated with the topic term.
[0070] In some implementations, the server is further configured to identify a user identifier from a set of one or more users of the viewer from the request for the selection value, and in response to identifying the user identifier, transmit the content provider's selection value to the content exchange server.
[0071] In some implementations, the server is further configured to receive from the client device an audience size defining a number of users to be selected from the plurality of users for the audience. When determining the audience, the server is further configured to identify a set of one or more users of the audience from the plurality of users based on the audience size.
[0072] In some implementations, the server is further configured to transmit the plurality of topic terms for display on a graphical user interface (GUI) of the client device.
[0073] In some embodiments, a non-transitory computer readable medium comprising machine-executable program instructions. Execution of the program instructions by one or more processors of the computer system causes the one or more processors to perform the following steps: receiving a context term identifying the additional term from a client device; generating a set of context terms including the context term by selecting one or more of a plurality of corpus terms having a relationship with the context term from a corpus database, the corpus database storing text extracted from a plurality of past web pages; calculating an implication score for the plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of past web pages associated with the topic term; selecting a particular topic term from the plurality of topic terms based on the implication score of the particular topic term; determining an audience representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and storing campaign data including the set of one or more users, the set of context terms, and the audience representing the particular topic term in a campaign database, the campaign data being configured to perform a real-time bid selection operation on available web pages being accessed by users of the audience hosted by one or more third party servers during the real-time bid selection operation.
[0074] In some implementations, execution of the program instructions by one or more processors of the computer system further causes the one or more processors to maintain an association between each user of the plurality of users based on the users having accessed at least one of a plurality of past web pages associated with the topic term, and determine an audience using the association between each user of the plurality of users and a particular topic term.
[0075] In some implementations, execution of the program instructions by one or more processors of the computer system further causes the one or more processors to perform the steps of identifying a user identifier from a set of one or more users of the audience from the request for the selection value, and transmitting the content provider's selection value to a content exchange server in response to identifying the user identifier.
[0076] The foregoing method descriptions and process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. The steps in the foregoing embodiments may be performed in any order. Words such as "then," "next," and the like are not intended to limit the order of the steps. These words are merely used to guide the reader through the method description. Although the process flow diagrams may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, and the like. Where a process corresponds to a function, the end of the process may correspond to a return of the function to the calling function or to the main function.
[0077] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed by the overall system. Those skilled in the art may implement the described functions in different ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure or the claims.
[0078] Computer software implemented embodiments may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
[0079] The actual software code or specialized control hardware used to implement these systems and methods is not a limitation of the claimed features or the present disclosure. Thus, the operation and behavior of the systems and methods have been described without reference to specific software code, with the understanding that software and control hardware may be designed to implement the systems and methods based on the description herein.
[0080] If implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module that may reside on a computer-readable or processor-readable storage medium. Non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. Non-transitory processor-readable storage media may be any available medium that can be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer or processor. Disk and disc, as used herein, include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while discs reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media. Furthermore, operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable medium and / or computer-readable medium, which may be embodied in a computer program product.
[0081] The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.
[0082] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for purposes of illustration and are not intended to be limiting, with a true scope and spirit being indicated by the following claims.
Claims
1. 1. A computer-implemented method for determining a user based on a database of a web page corpus for dynamically placing content on available web pages hosted by a third-party web server, the method comprising: receiving, by the server, from the client device, a context term identifying an additional term; generating, by the server, a set of context terms including the context term by selecting one or more of a plurality of corpus terms having a relationship with the context term from a corpus database, the corpus database storing text extracted from a plurality of past web pages; calculating, by the server, an implicit score for a plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of historical web pages associated with the topic term; selecting, by the server, a particular topic term from the plurality of topic terms based on the suggestion score of the particular topic term; determining, by the server, an audience representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and storing, by the server, in a campaign database, campaign data including the set of one or more users, the set of context terms, and the audience representing the particular topic terms, the campaign data being configured to perform a real-time bid selection operation during the real-time bid selection operation on available web pages hosted by one or more third-party servers and accessed by users of the audience.
2. The method of claim 1 , further comprising determining, by the server, the plurality of topic terms from a plurality of terms on the plurality of past web pages accessed by a plurality of users.
3. maintaining, by the server, an association between each of a plurality of users based on the users having accessed at least one of the plurality of past web pages associated with the topic term; 2. The method of claim 1, wherein determining the audience further comprises: identifying the set of one or more users of the audience using the associations between each user of the plurality of users and the particular topic terms.
4. 2. The method of claim 1, further comprising: ranking, by the server, a plurality of users for the topic term based on a number of times each user of the plurality of users accessed at least one of the plurality of past web pages associated with the topic term.
5. identifying, by the server, an identifier of the user from the set of one or more users of the audience from a request for a selection value; 10. The method of claim 1, further comprising transmitting, by the server, the content provider selection to a content exchange server in response to identifying the identifier of the user.
6. receiving, by the server, from the client device, an audience size defining a number of users to be selected from the plurality of users for the audience; The method of claim 1 , wherein determining the audience comprises identifying a set of the one or more users of the audience from a plurality of users based on the audience size.
7. The method of claim 1 , further comprising generating, by the server, a plurality of phrases of the topic term using a plurality of terms on the plurality of past web pages from which the topic term is determined.
8. 2. The method of claim 1 , wherein calculating the suggestive score further comprises calculating the suggestive score based on a number of occurrences of at least one of the set of context terms on at least one of the plurality of past web pages associated with the topic term.
9. 2. The method of claim 1 , wherein generating the set of context terms further comprises: (i) selecting a first subset of corpus terms from the plurality of corpus terms based on the context terms; and (ii) removing a second subset of corpus terms from the first subset of corpus terms using out-of-context terms received from the client device.
10. The method of claim 1 , further comprising transmitting, by the server, the plurality of topic terms for display on a graphical user interface (GUI) of the client device.
11. 1. A system for determining a user based on a database of a web page corpus for dynamically placing content on available web pages hosted by a third party web server, the system comprising: a non-transitory medium including one or more databases, including a corpus database configured to store a plurality of historical web pages, and a campaign database configured to store campaign data; 1. A server having at least one processor coupled to a memory, the processor comprising: receiving, from a client device, a context term used to identify an additional term; generating a set of context terms including the context term by selecting, from text in the corpus database, one or more of a plurality of corpus terms having a relationship with the context term; calculating an implicit score for a plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of historical web pages associated with the topic term; selecting a particular topic term from the plurality of topic terms based on the suggestive score of the particular topic term; determining an audience representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and a server configured to store the campaign data in the campaign database, the campaign data being configured to perform a real-time bid selection operation on available web pages being accessed by users of the audience that are hosted by one or more third-party servers during the real-time bid selection operation.
12. The system of claim 11 , wherein the server is further configured to determine the plurality of topic terms from a plurality of terms on the plurality of past web pages accessed by a plurality of users.
13. The server, maintaining an association between each user of a plurality of users based on the user having accessed at least one of the plurality of past web pages associated with the topic term; The system of claim 11 , further configured to determine the audience using the associations between each user of the plurality of users and the particular topic terms.
14. 12. The system of claim 11, wherein the server is further configured to rank a plurality of users for the topic term based on a number of times each user of the plurality of users accessed at least one of the plurality of past web pages associated with the topic term.
15. The server, identifying an identifier of the user from the set of one or more users of the audience from the request for a selection value; The system of claim 11 , further configured to transmit the content provider preference to a content exchange server in response to identifying the identifier of the user.
16. the server is further configured to receive from the client device an audience size defining a number of users to be selected from the plurality of users for the audience; 12. The system of claim 11, wherein when determining the audience, the server is further configured to identify the set of one or more users of the audience from a plurality of users based on the audience size.
17. The system of claim 11 , wherein the server is further configured to transmit the plurality of topic terms for display on a graphical user interface (GUI) of the client device.
18. A non-transitory computer-readable medium containing machine-executable program instructions, the execution of which by one or more processors of a computer system causes the one or more processors to: receiving, from a client device, a context term identifying an additional term; generating a set of context terms including the context term by selecting one or more of a plurality of corpus terms having a relationship with the context term from a corpus database, the corpus database storing text extracted from a plurality of past web pages; calculating an implicit score for a plurality of topic terms based on a co-occurrence between each topic term and one or more of the set of context terms on a set of historical web pages associated with the topic term; selecting a particular topic term from the plurality of topic terms based on the implicit score of the particular topic term; determining an audience representing a set of one or more users who accessed at least one of the set of past web pages associated with the particular topic term; and storing in a campaign database campaign data including the set of one or more users, the set of context terms, and the audience representing the particular topic terms, wherein the campaign data is configured to perform a real-time bid selection operation during the real-time bid selection operation on available web pages hosted by one or more third-party servers and accessed by users of the audience.
19. The execution of the program instructions by the one or more processors of the computer system causes the one or more processors to: maintaining an association between each of a plurality of users based on the users having accessed at least one of the plurality of past web pages associated with the topic term; and determining the audience using the associations between each user of the plurality of users and the particular topic terms.
20. The execution of the program instructions by the one or more processors of the computer system causes the one or more processors to: identifying an identifier of said user from said set of one or more users of said audience from a request for a selection value; 20. The non-transitory computer-readable medium of claim 18, further comprising: transmitting the content provider selection to a content exchange server in response to identifying the identifier of the user.
