Neural network processing of return path data to estimate the demographics of household members and visitors

Abstract

Demographic estimation system, comprising: a feature generator for generating features from return path data (RPD) reported by set-top boxes associated with return path data households; a neural network to process the features generated from the return path data and predict demographic classification probabilities for the return path data households, wherein the neural network is to be trained based on forum data reported by measurement devices that monitor media devices associated with the participant households; a demographic assignment engine for assigning one or more demographic categories to corresponding ones of the return path data households based on the predicted demographic classification probabilities; and a visitor assignment engine for assigning virtual visitors to at least a subset of the corresponding ones of the return path data households based on the one or more demographic categories assigned to the corresponding ones of the return path data households.

Description

RELATED APPLICATION(S)

[0001] According to the provisions of the Utility Model Act, only devices (devices, systems) as defined in the appended claims are eligible for protection and are the subject of the utility model, not methods. Where reference is made to methods in the following description, these references serve only as an example to explain the device(s) protected in the appended claims. This utility model is a continuation-in-part of U.S. patent application No. 16 / 230,620, entitled "NEURAL NETWORK PROCESSING OF RETURN PATH DATA TO ESTIMATE HOUSEHOLD DEMOGRAPHICS," filed December 21, 2018, which claims priority to U.S. provisional patent application No. 62 / 743,925, entitled "NEURAL NETWORK PROCESSING OF SET-TOP BOX RETURN PATH DATA TO ESTIMATE HOUSEHOLD DEMOGRAPHICS," filed October 10, 2018.This utility model also claims priority to U.S. Provisional Patent Application Serial No. 62 / 841,641, entitled “NEURAL NETWORK PROCESSING OF RETURN PATH DATA TO ESTIMATE HOUSEHOLD MEMBER AND VISITOR DEMOGRAPHICS,” filed May 1, 2019. Priority is claimed to U.S. Provisional Patent Application Serial No. 16 / 230,620, U.S. Provisional Patent Application Serial No. 62 / 743,925, and U.S. Provisional Patent Application Serial No. 62 / 841,641. U.S. Patent Application Serial No. 16 / 230,620, U.S. Provisional Patent Application Serial No. 62 / 743,925, and U.S. Provisional Patent Application Serial No. 62 / 841,641 are incorporated by reference into this application. AREA OF REVELATION

[0002] This disclosure relates generally to neural networks and, more particularly, to neural network processing of back-path data for estimating the demographics of household members and visitors. BACKGROUND

[0003] Audience Measurement Entities (AMEs), such as The Nielsen Company (US), LLC, may extrapolate ratings metrics and / or other audience measurement data relative to a total television viewership from a relatively small sample of forum households. The forum households may be well-studied and typically selected to be representative of a target audience as a whole. Further, to supplement the forum data, an AME, such as The Nielsen Company (US), LLC, may enter into arrangements with pay-TV providers to obtain television channel selection information, referred to herein and in the industry as return path data, from set-top boxes and / or other devices / software. BRIEF DESCRIPTION OF THE DRAWINGS Fig.1 is a block diagram of an exemplary processing flow for estimating demographic classification probabilities from set-top box return path data using a neural network in accordance with the teachings of this disclosure. Fig. 2 is a block diagram of an example processing flow for using the demographic classification probabilities obtained by the example processing flow of Fig. 1 were estimated to assign demographics to households in accordance with the teachings of this revelation. Fig. Figure 3 is a block diagram of an exemplary neural network-based demographic estimation system structured to support the processing flows of Fig. 1 and Fig. 2 to estimate household demographics from set-top box return path data in accordance with the teachings of this disclosure. Fig.4A-B illustrate example features generated by the example feature generator used in the example neural network-based demographic estimation system of Fig. 3 is included. Fig. 5 is a block diagram of an exemplary implementation of the exemplary neural network for demographic prediction used in the exemplary neural network-based demographic estimation system of Fig. 3 is included. Fig. 6A-C illustrate an example operation of the neural network for demographic prediction of Fig. 3 to estimate demographic classification probabilities from set-top box return path data in accordance with the teachings of this disclosure. Fig. Figure 7 illustrates exemplary pseudocode for implementing the exemplary household demographic assignment engine used in the exemplary neural network-based demographic estimation system of Fig.3 is included. Fig. 8A-E illustrate an example operation of the household demographic assignment engine of Fig. 3 to allocate demographics to households in accordance with the teachings of this revelation. Fig. 9A-C illustrate example simulated annealing operations performed by the household demographic allocation engine of Fig. 3 can be executed. Fig. 10 is a block diagram of a second exemplary neural network-based demographics estimation system structured to estimate household demographics for primary household members and long-term visitors from the set-top box return path data in accordance with the teachings of this disclosure. Fig. 11 is a block diagram of an example visitor assignment engine used in the example neural network-based demographic estimation system of Fig.10 to assign virtual visitors to return path data households in accordance with the teachings of this disclosure. Fig. 12 is a flowchart representative of exemplary computer-readable instructions that may be executed to implement the neural network-based demographic estimation system of Fig. 3 to be implemented. Fig. 13 is a flowchart representative of exemplary computer-readable instructions that may be executed to implement the second neural network-based demographic estimation system of Fig. 10 to be implemented. Fig. 14-15 are flowcharts representative of exemplary computer-readable instructions that may be executed to implement the exemplary demographic target adjustment apparatus used in the neural network-based demographic estimation system of Fig. 10 is included. Fig.16-18 are flowcharts representing example computer-readable instructions that may be executed to configure the visitor allocation engine of Fig. 10 to be implemented. Fig. 19 is a flowchart representative of exemplary computer-readable instructions that may be executed to implement the visitor vector allocation apparatus of Fig. 11 subject to the restriction that a visitor must be assigned to a specific household. Fig. 20 is a block diagram of an example processor platform structured to execute the example machine-readable instructions of Fig. 12 to implement the exemplary neural network-based demographic estimation system of Fig. 3 to be implemented. Fig. 21 is a block diagram of an example processor platform structured to execute the example machine-readable instructions of the Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and / or 19 to implement the exemplary neural network-based demographic estimation system of Fig. 10 to be implemented. The figures are not to scale. In general, the same reference numerals are used throughout the drawing(s) and the accompanying description to refer to the same or similar parts, elements, etc. DETAILED DESCRIPTION

[0004] Descriptors such as "first," "second," "third," etc., are used herein to identify multiple elements or components that may be referred to separately. Unless otherwise indicated or understood based on their context of use, such descriptors are not intended to confer any significance regarding priority or chronological order, but merely serve as labels to refer to multiple elements or components separately and to facilitate understanding of the disclosed examples. In some examples, the descriptor "first" may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as "second" or "third." In such cases, it is understood that such descriptors are used merely to conveniently refer to multiple elements or components.

[0005] Example methods, apparatus, systems, and articles of manufacture (e.g., physical storage media) for implementing neural network processing of backpath data to estimate household demographics are disclosed herein. Examples of such demographic estimation systems disclosed herein include a feature generator to generate features from backpath data reported by set-top boxes associated with backpath data households. Example demographic estimation systems disclosed herein also include a neural network to process the features generated from the backpath data to predict demographic classification probabilities for the backpath data households.Example demographic estimation systems disclosed herein further include a demographic assignment engine to assign one or more demographic categories to corresponding ones of the back-path data households based on the predicted demographic classification probabilities.

[0006] These and other exemplary methods, apparatus, systems, and articles of manufacture (e.g., physical storage media) for implementing neural network processing of back-path data to estimate household demographics are described in more detail below.

[0007] As mentioned above, AMEs extrapolate ratings metrics and / or other audience measurement data for an entire television viewing audience from a relatively small sample of participant households, also referred to herein as forum households. Forum households can be well-studied and typically selected to be representative of a target audience as a whole. However, accurately representing the geographic distribution and demographic diversity that exists in the overall audience population with a small sample of forum households remains challenging. Incorporating additional information streams about media exposure to the overall audience population can fill in gaps or biases inherent in any statistical sample.

[0008] An AME, such as The Nielsen Company (US), LLC, may, to supplement the Forum Data, enter into arrangements with pay-TV providers to obtain television channel selection information, referred to herein and in the industry as Return Path Data (RPD), from set-top boxes and / or other devices / software. Set-top box (STB) data includes all data collected by the set-top box. STB data may include, for example, channel selection events and / or commands received by the STB (e.g., power on, power off, channel change, input source change, beginning a media presentation, pausing a media presentation, recording a media presentation, volume up / down, etc.). STB data may additionally or alternatively include commands sent from the STB to a content provider (e.g.,Changing input sources, recording a media presentation, deleting a recorded media presentation, the time / date a media presentation was started, the time a media presentation was completed, etc.), heartbeat signals, or the like. The set-top box data may additionally or alternatively include a household identifier (e.g., a household ID) and / or an STB identifier (e.g., an STB ID).

[0009] Return path data includes any data that can be received at a media service provider (e.g., a cable television service provider, a satellite television service provider, a streaming media service provider, a content provider, etc.) via a return path from a media consumer site. As such, return path data includes at least a portion of the set-top box data. Return path data may additionally or alternatively include data from any other consumer device with network access capabilities (e.g., via a cellular network, the Internet, other public or private networks, etc.). For example, return path data may include any or all real-time linear data from an STB, television program user data from a television program broadcaster, click stream data, key stream data (e.g., any remote control clicks—volume, mute, etc.), interactive activity (such as video on demand), and any other data (e.g., data from middleware).RPD data can additionally or alternatively come from the network (e.g. via switched digital software) and / or from any cloud-based data (such as a remote server DVR).

[0010] RPDs can provide information about media exposure associated with a larger segment of the audience population. This is because RPDs typically provide a rich stream of television viewing information for a much larger number of households than are contained in an AME's forum households. However, unlike the well-studied AME forum households, the demographic details of pay-TV subscribers are typically unknown. These missing demographic details in the RPDs can result in technical issues that prevent or at least limit the effective use of the RPDs to supplement the AME's forum data, as monitoring the behavioral profiles of different audience demographics requires knowledge of the demographic composition of the subscriber households providing the RPDs.

[0011] The neural network processing of set-top box RPD to estimate household demographics disclosed herein provides a technical solution to the technical problem of combining RPD with forum data for audience measurement. As further described below, exemplary neural network-based demographic estimation systems implemented according to the teachings of this disclosure use forum data collected from monitored AME forum households as a training set to train a neural network (e.g., a recurrent neural network) to predict, from RPD tuning data describing historical television tuning behavior, probabilities of various household demographic characteristics associated with corresponding ones of the RPD households reporting the RPD data.The disclosed exemplary neural network-based demographic estimation system then predicts the use of the predicted probabilities of various household demographic characteristics to assign demographic compositions. In this manner, exemplary neural network-based demographic estimation systems assign demographic compositions to the subscriber households providing the RPD, thereby enabling the RPD to be combined with the forum data or otherwise enhance the forum data driving an audience measurement system of the AME. Such exemplary neural network-based demographic estimation systems are also referred to as implementing an exemplary household demographic assignment model (HDAM) to assign demographic compositions to households.

[0012] In some disclosed examples, the HDAM implemented by the neural network-based demographic estimation system predicts household-level demographic assignments based on television viewing data, but predicts household-level demographic characteristics for primary household members and not for long-term visitors. As used herein, a long-term visitor is an individual who visits the household more than once and / or for an extended period of time, such as at least once every two weeks or stays in the household for at least one month in a calendar year. A long-term visitor has a primary residence elsewhere, is not a household member, and typically watches and / or listens to television in the household during at least a portion of their visits or extended stay.However, other criteria may be used to classify individuals as long-term attendees by considering factors such as improving household compliance. Some of the examples disclosed herein modify the HDAM model to predict long-term attendee demographics while consistent with long-term attendee distributions obtained from forum data obtained by the AME through monitoring its participants, such as the Nielsen People Monitoring forum data (NPM Forum Data) generated by The Nielsen Company (US), LLC.

[0013] In some examples disclosed herein, the modification of the HDAM model to predict demographics of long-term visitors uses long-term visitor data available from the AME forum data. Using this forum data, the percentages of all individuals in each age-sex bin who are long-term visitors can be determined. Using the estimates for the total number of individuals in each age-sex bin, these estimated totals can be modified to include visitors based on the forum visitor percentages. By modifying the population targets applied to the HDAM model, the HDAM model can be modified to predict household compositions that include both long-term visitors and primary household members.

[0014] Examples disclosed herein distinguish long-term visitors from primary household members. In some examples, the expected total number of long-term visitors and the expected age-sex distributions are known from the AME forum data. The output of the HDAM model can be viewed as a vector containing the total number of long-term visitors and primary household members in each of the various demographic categories (e.g., age-sex bins). Knowing the percentage of long-term visitors in each demographic category (e.g., each age-sex group), a population of long-term visitors can be assigned to households to satisfy the expected distribution across demographic categories. In the examples disclosed herein, a set of visitor vectors is generated that satisfy forum-related consistency requirements.After generating the visitor vectors, households are identified into which these visitor vectors can be placed. For example, the set of possible (candidate) households (from the HDAM predictions) to which each visitor vector could be assigned is determined. Using the probabilities that corresponding one of the individual households has a long-term visitor, the visitor vectors are placed into the households, prioritizing those households that are more likely to have long-term visitors. Examples disclosed herein generate long-term visitor assignments to the predicted households that automatically satisfy forum-related consistency requirements without disturbing / affecting the HDAM predictions for primary household members. Thus, primary household member assignments and long-term visitor assignments can be achieved by the examples disclosed herein.

[0015] Referring to the figures, a block diagram of an exemplary processing flow 100 for estimating demographic classification probabilities from set-top box RPD using a neural network according to the teachings of this disclosure is shown in Fig. 1. The example processing flow 100 includes an example data collection phase 105, an example feature generation phase 110, and an example neural network demographic probability prediction phase 115. The example processing flow 100 is further divided into an example neural network training branch 120 and an example neural network application branch 125.

[0016] In the data collection phase 105 of the neural network training branch 120, exemplary participant preference data 130 is collected from measurement devices that monitor media exposure in forum households recruited by an AME. The participant preference data 130 may include any data that can be collected by the measurement devices, such as, without limitation, data identifying media presented by media devices in the forum households, demographic data identifying characteristics of the participants in the forum households, etc. In the feature generation phase 110 of the neural network training branch 120, exemplary features 135 are generated from the collected participant preference data 130 and arranged to form feature vectors, as described in more detail below.In the neural network demographic probability prediction phase 115 of the neural network training branch 120, a neural network 140 is trained to predict probabilities of different household demographic characteristics associated with the various forum households from the features 135 generated from the collected participant preference data 130, as described in more detail below.

[0017] In the data collection phase 105 of the neural network application branch 125, exemplary RPD setting data 145 is collected from set-top boxes of one or more pay-TV providers (e.g., cable TV service providers, satellite TV service providers, streaming media service providers, content providers, etc.). A set-top box may also refer to any decoder, receiver, integrated receiver decoder (IRD), media device, etc., from which the RPD setting data 145 may be collected. In the feature generation phase 110 of the neural network application branch 125, exemplary features 150 are generated from the collected RPD setting data 145 and arranged to form feature vectors, as described in more detail below.In the neural network demographic probability prediction phase 115 of the neural network application branch 125, the trained neural network 155 is applied to the features 150 generated from the collected RPD setting data 145 to predict example estimated probabilities 160 of various household demographic characteristics associated with the various RPD subscriber households that reported the RPD setting data 145, as described in more detail below.

[0018] A block diagram of an example processing flow 200 for using the estimated demographic classification probabilities 160 generated by the example processing flow 100 of Fig. 1 to assign demographics to households according to the teachings of this revelation is in Fig.2. As described in more detail below, the processing flow 200 uses an example mixed integer programming solution 205 that solves a constrained optimization problem based on the estimated demographic classification probabilities 160 predicted by the example processing flow 100 to assign example estimated demographic compositions 210 to the subscriber households providing the RPD setting data 145.

[0019] A block diagram of an exemplary neural network-based demographic estimation system 300 structured to include the processing streams 100 and 200 of the Fig. 1 and Fig. 2 to estimate household demographics for primary household members from the set-top box RPD in accordance with the teachings of this disclosure is in Fig.3. The example neural network-based demographics estimation system 300 includes an example network interface 305, an example forum setting data collector 310, an example participant database 315, an example RPD data collector 320, an example RPD database 325, an example feature generator 330, an example demographics prediction neural network 335, an example household demographics assignment engine 340, an example constraint database 345, and an example scoring calculator 350.

[0020] In the illustrated example, the forum preferences data collector 310, in communication with one or more example networks 355 via the network interface 305, collects the subscriber preferences data 130 from example measurement devices 360A-B that monitor media exposure associated with the example media devices 365A-B (e.g., televisions, radios, computers, tablets, smartphones, etc.) in forum households recruited by an AME. The forum setting data collector 310 stores the collected subscriber setting data 130 in the subscriber database 315. In the illustrated example, the RPD data collector 320, via the network interface 305 in communication with the one or more networks 355, collects the RPD setting data 145 from one or more example service providers 370, which collect the RPD setting data 145 from example individual STBs 375 in the subscriber homes.Additionally or alternatively, in some examples, the RPD data collector 320 collects the RPD setting data 145 from one or more of the individual STBs 375 in the subscriber homes directly via the network interface 305 in communication with the one or more networks 355. The RPD data collector 320 stores the collected RPD setting data 145 in the RPD database 325.

[0021] The feature generator 330 of the illustrated example generates the features and feature vectors used by the example demographic prediction neural network 335. In some examples, RPD tuning data consists of sequential logs of when the corresponding set-top boxes were tuned to various channels. Individuals (e.g., audience members) switch between multiple networks over the course of a contiguous television viewing session, and this pattern of activity can provide additional information about the household in isolation beyond the tuning record. To capture this behavior, the feature generator 330 compiles the STB television channel selection records into "viewing chunks" that aggregate the viewing behavior of one or more unknown viewers into a fixed number of features that summarize each contiguous viewing session.In some examples, viewing block time periods are limited to one hour or another duration to accommodate situations where multiple viewers can take control of a television without necessarily turning off the television between sessions. In the illustrated example, each viewing block F includes features that record information about the start time of the viewing block, the channel click-through rate, the duration of the viewing sessions, and a listing of the television channels visited during the session.

[0022] The Fig. 4A-B illustrate an exemplary operation of the feature generator 330 to combine exemplary RPD setting data sets 405 from the RPD setting data 145 into corresponding exemplary view blocks 410 and 415. In the illustrated example of Fig.4A, corresponding ones of the data records 405 record tuning events reported by the STBs 375. A given data record 405 specifies an STB identifier (STB ID) 420 that identifies the STB according to the event log, start and end times 425 and 430 that correspond to the tuning event represented by the event log, a source identifier (SID) 435 that identifies the media source (e.g., channel number, station ID, etc.) associated with the tuning event, and a broadcast time 440 that identifies when the media associated with the tuning event was originally broadcast (e.g., to distinguish between live and time-delayed tuning events). In the illustrated example of Fig.4B, the consideration block 410 aggregates the setting events recorded in the data records 405 for a particular household that occur in the hourly interval beginning at 8:23 a.m. on November 5, 2016. In the illustrated example of Fig. 4B, the observation block 415 aggregates the setting events recorded in the data records 405 for a particular household that occur in the hourly interval beginning at 6:04 PM on November 6, 2016.

[0023] The feature generator 330 of the illustrated example groups viewing blocks by household, and a group of N viewing blocks is combined into a two-dimensional (NxF) matrix containing a dataset of the viewing blocks generated by a household over a given observation period. In some examples, the feature generator 330 aggregates relevant household-level characteristics, including the number of television tuners and television viewing time, with the viewing block data into an H-dimensional (1xH) additional feature vector for each household.

[0024] In some examples, each viewing block is a (1x173) feature vector describing a corresponding television viewing session. The corresponding (NxF) matrix therefore has an F dimension of 173 for these examples. Table 1 illustrates the contents of an exemplary viewing block represented as a (1x173) feature vector. Table 1 index Short description Valid range 0 weekday 0-6 1 anniversary 0-364 2 quarter hour of day 0-95 3 Channel change rate 0-Inf 4-173 Minutes each network was viewed 0-60

[0025] The first three features in Table 1 are self-explanatory. The "Channel Switch Rate" feature in Table 1 is the ratio of the number of channel switches during the viewing block to the duration of the viewing block in minutes. The "Minutes Each Network Watched" feature indicates the total number of minutes each television station was watched. In the example in Table 1, viewing blocks are limited to 60 minutes in duration, so the summation of these features across all networks must be <= 60.0 minutes. In some such examples, a viewing session may be associated with one or more viewing blocks. In the example in Table 1, each station is randomly assigned an index value between 4 and 173.

[0026] In some examples, viewing blocks (from forum households) that contain less than 5 minutes of television viewing behavior are not used to train the demographic prediction neural network 335. The viewing blocks for each household (e.g., forum households for neural network training and RPD households for neural network application) are then stacked into a two-dimensional matrix with, for example, 400 rows (e.g., N = 400). In some examples, households that have generated fewer than 400 unique viewing blocks are zero-padded by the feature generator 330 until they have 400 rows, while those with more than 400 are truncated to the first 400 rows by the feature generator 330.The two-dimensional arrays from each household are then stacked by the feature generator 330 to form a three-dimensional matrix that can be fed into the demographic prediction neural network 335.

[0027] In some examples, feature generator 330 augments viewing data with three household-level features H merged into demographic prediction neural network 335 after a recurrent layer, as described below. Table 2 illustrates an exemplary set of the three household-level features H corresponding to (i) a total tuning amount reported for the given household over the various time periods covered by the viewing blocks (e.g., a 24-hour period) (corresponding to index 0 in the table), (ii) a number of viewing blocks reported for the given household over the various time periods (corresponding to index 1 in the table), and (iii) a total number of tuners included in the first of the return-path data households (corresponding to index 2 in the table). Table 2 index Short description Valid Area 0 Total TV consumption (minutes) 1-Inf 1 Number of recorded viewing blocks 1-Inf 2 Number of TV tuners in the household 1-Inf

[0028] In the illustrated example, the demographic prediction neural network 335 is structured to predict 20 variables (e.g., a 1x20 vector) representing probabilities of various household-level demographics present in a household (although other numbers of variables representing other demographics could additionally or alternatively be predicted in other example implementations of the demographic prediction neural network 335). In the illustrated example, fourteen household demographic target variables predicted by the demographic prediction neural network 335 indicate the corresponding probabilities (e.g., probabilities) of 14 different age-sex combinations present in the household, examples of which are shown in Table 3. Table 3 index Age / Gender 0 0-12 1 13-17 2 18-24 Male 3 25-34 Male 4 35-44 Male 5 45-54 Male 6 55-64 Male 7 65+ Male 8 18-24 Female 9 25-34 Female 10 35-44 Female 11 45-54 Female 12 55-64 Female 13 65+ Female

[0029] In addition to the presence variables in Table 3, the demographic prediction neural network 335 predicts, in some examples, six additional target variables that describe the demographic profile of the Head of Household (HOH), examples of which are shown in Table 4. Table 4 index HOH properties 14 HOH age 15 HOH gender 16 Hispanic origin 17 European American 18 African Americans 19 Asian American

[0030] An example implementation of the neural network for demographic prediction 335 by Fig. 3 is in Fig.5. In some examples, the two-dimensional (NxF) feature vectors (e.g., 400 x 173 feature vectors) generated for corresponding households to be processed (e.g., Forum and / or RPD households) are typically sparse (e.g., many broadcast networks represented in the feature vectors are never visited during a particular viewing block). To condense this input into a smaller subset of features, the demographic prediction neural network 335 includes an example Time Distributed Dense Layer (TDDL) 505 that learns a single set of weights that map each viewing block to a condensed representation of the input (NxF', where F' << F). This condensed data is then fed into an example recurrent Long Short Term Memory (LSTM) neural network layer 510.The LSTM 510 examines each row of the viewing block matrix in order and uses this information to selectively update a single internal state vector that encodes information from each viewing session / block. The output of the LSTM 510 is a one-dimensional (1xF') feature vector that summarizes the evidence history observed for each household. The example demographics prediction neural network 335 of . Fig.5 includes an exemplary merging layer 515 for merging (linking) additional (1×H) household-level features with the one-dimensional representation of the viewing data output from the LSTM 510. The additional (1×H) household-level features include information about the total number of devices in the household, the total minutes viewed over the observation window, and the total number of viewing blocks recorded for the corresponding household over the observation window, as described above.

[0031] In the example neural network for demographic prediction 335 of Fig.5, the augmented feature vector output from the merging layer 515 is passed to one or more additional example hidden layers 520 before being output by an example output layer 525 as a (1 x C) probability vector representing the corresponding predicted probabilities of the C possible demographic categories present in the household. The C demographic classes modeled by the demographic prediction neural network 335 need not be mutually exclusive (e.g., households may contain multiple individuals of different ages / genders), so the output vector encodes the relative probability with which each modeled household-level demographic is present in the unknown household.

[0032] Table 5 lists example dimensions of the data in each stage of the example neural network for demographic prediction 335 of Fig.5. In Table 5, N is the total number of observation blocks per household, F is the number of features in each observation block, F' is the number of density features generated by the TDDL 505, and H is the number of additional household-specific features. Table 5 dimension Number of nodes H 3 N 400 F 173 F' 30 C 20

[0033] In some examples, the feature generator 330 shuffles the order of the blocks fed to the demographic prediction neural network 335 during each training period to prevent the demographic prediction neural network 335 from becoming overfit and to enable it to generalize better.

[0034] The Fig.6A-C illustrate an example operation of the demographic prediction neural network 335 to predict the demographic target variables 605, 610, 615, and 620 generated from the RPD setting data 145 as feature vectors 625, 630, and 635 that are applied to the demographic prediction neural network 335 after the demographic prediction neural network 335 is trained with feature vectors generated from the forum data 130.In the illustrated example, the demographic prediction neural network 335 is trained by (i) generating view blocks from the subscriber data 130 reported for the subscriber household, (ii) generating the features for corresponding ones of the subscriber households from the view blocks created for the corresponding subscriber households, as described above, and (iii) applying the features for the corresponding ones of the subscriber households to the neural network 335 according to a training procedure that adjusts the internal parameters of the neural network to reduce an error between the predicted demographic classification probabilities 160 output by the neural network 335 and the actual demographics known for the subscriber households. As in the example of FIG. Fig.6A-C, the output of the network 335 will converge to predict demographic classification probabilities 160 in accordance with the actual demographics known for the participant households as more viewing blocks are applied to train the neural network 335.

[0035] With reference to Fig. 3, the example household demographic assignment engine 340 of the example neural network-based demographic estimation system 300 uses the estimated demographic classification probabilities (also referred to above as the predicted demographic target variables) the output from the demographic prediction neural network 335 to assign demographics to RPD households in accordance with the teachings of this disclosure. Fig.7 illustrates the example pseudocode 700 for implementing the household demographic allocation engine 340. The example pseudocode 700 also corresponds to an example of the mixed-integer programming solution 205 of Fig. 2. In the illustrated example of Fig. 7, the pseudocode 700 for implementing the household demographic assignment engine 340 assigns demographics to households by solving an objective function to determine a matrix x0, which is a Boolean matrix representing the demographic categories assigned to different RPD households, given a cost matrix CO representing the cost of assigning different demographic categories to the RPD households subject to a set of constraints having values stored in the example constraint database 345. In the example of Fig.7, the matrix x0 is a matrix with a number of rows equal to the number of RPD households and a number of columns equal to the number of different possible demographic categories that can be assigned to a household. Furthermore, in the illustrated example, for a given row of x0 representing a given RPD household, the elements of the row contain binary (Boolean) variables representing the different possible demographic categories, where the given binary variable representing a given possible demographic category is assigned a value of 1 by pseudocode 700 if that demographic category is assigned to that RPD household, or a value of 0 by pseudocode 700 if that demographic category is not assigned to that RPD household. In the example of Fig.7, the matrix CO is also a matrix with a number of rows equal to the number of RPD households and a number of columns equal to the number of different possible demographic categories that can be assigned to a household. Furthermore, in the illustrated example, for a particular series of COs representing a particular RPD household, the elements of the series include cost variables representing the corresponding costs of assigning the various possible demographic categories to the particular RPD household. In some examples, the cost variables in the CO are determined by the household demographic assignment engine 340 based on the estimated demographic classification probabilities (also referred to above as the predicted demographic target variables) output by the demographic prediction neural network 335.For example, the cost variable for assigning a given possible demographic category to the given RPD household may be determined by the household demographic assignment engine 340 as the inverse function (or other function) of the demographic classification probability for that demographic category and the RPD household, as determined by the demographic prediction neural network 335.

[0036] As in the example of Fig. 7, the pseudocode 700 uses mixed-integer programming or a similar technique to determine the demographic assignment matrix x0 by solving the objective function: ∑i,jC0(i,j)⋅x0(i,j) subject to a set of limitations. The exemplary limitations of Fig.7 are based on a matrix x1, which is a Boolean matrix representing the different possible household sizes that can be assigned to the different RPD households, and a size matrix S1, which represents the values of the different possible household sizes.

[0037] The Fig. 8A-E illustrate an exemplary operation of the household demographic assignment engine 340 implemented by the pseudocode 700 of Fig. 7 is implemented to assign demographic categories to RPD households by using the above expression, taking into account the exemplary restrictions of Fig. 7 is solved. Fig.Figure 8A illustrates an exemplary CO cost matrix 805 with 5 rows representing 5 RPD households for which demographic categories are to be assigned, and 4 columns representing 4 possible demographic categories that could be assigned to corresponding RPD households. The cost values for the various possible demographic categories are shown in Fig. 8A by dollar signs ($), where more dollar signs represent higher costs. In the illustrated example, the costs contained in the CO cost matrix 805 are inversely proportional to the corresponding estimated demographic classification probabilities (also referred to above as the predicted demographic targets) output by the output layer 525 of the demographic prediction neural network 335 for the given combinations of household and demographic categories.

[0038] With reference to the Fig. 7 and Fig.8A-E include the exemplary limitations of Fig. 7, a first constraint 705, which specifies that the totals of the different demographic categories assigned to all RPD households must equal the known universe estimates (UEs) for the corresponding different demographic categories (e.g., within a tolerance level represented by the variable "slack"). An example of the first constraint 705 is shown in Fig. 8B, in which the sums of the corresponding demographic categories assigned across the 5 households must correspond to the corresponding exemplary UEs 810 for the various demographic categories (which can be obtained, for example, from the service providers) that provide the RPD and that are stored in the constraint database 345. For example, specified in Fig.8B, the first constraint 705 that the number of households to be assigned the demographic category of “man” must equal the UE of 2 for that demographic category, the number of households to be assigned the demographic category of “woman” must equal the UE of 4 for that demographic category, the number of households to be assigned the demographic category of “girl” must equal the UE of 3 for that demographic category, and the number of households to be assigned the demographic category of “man” must equal the UE of 2 for that demographic category.

[0039] The exemplary restrictions of Fig. 7 include a second constraint 710 that specifies that there must be at least one adult demographic category assigned to each RPD household. An example of the second constraint 710 is shown in Fig.8B, in which each RPD household is constrained to include the male demographic category and / or the female demographic category (represented by reference numeral 815).

[0040] The exemplary restrictions of Fig. 7 include a third constraint 715, which specifies that the total number of different possible household sizes assigned to all RPD households must equal the known universe estimates (UEs) for the different possible household sizes (e.g., within a tolerance level represented by the variable "slack"). An example of the third constraint 715 is shown in Fig.8D, in which the numbers of corresponding possible household sizes allocated across the 5 households must correspond to the corresponding exemplary UEs 820 for the various possible household sizes (which can be obtained, for example, from the service provider(s) providing the RPD and stored in the constraint database 345). In Fig. For example, 8D specifies the third constraint 715 that the number of households containing two persons must equal the UE of 3 for that household size, the number of households containing three persons must equal the UE of 1 for that household size, and the number of households containing four persons must equal the UE of 1 for that household size.

[0041] The exemplary restrictions of Fig.7 include a fourth constraint 720 specifying that each RPD house shall be assigned only one of the possible household sizes, and a fifth constraint 725 specifying that the number of different demographic categories assigned to a given RPD household must equal the household size assigned to that household. Fig. 8E illustrates the resulting exemplary demographic category assignments 825 determined by the household demographic assignment engine 340 implemented with the pseudocode 700 of Fig. 7 and the given exemplary limitations 705 to 725, as set out in the Fig. 8A-D. In the example of Fig. 8E, the household demographic allocation engine 340 implemented with the pseudocode 700 solves the above (and in Fig.7) provided expression, subject to the limitations mentioned above, to assign: (I) the demographic categories “Woman” and “Boy” to the first RPD household, (2) the demographic categories “Woman” and “Boy” to the second RPD household, (3) the demographic categories “Man” and “Girl” to the third RPD household, (4) the demographic categories “Woman”, “Girl”, and “Boy” to the fourth RPD household, and (5) the demographic categories “Man”, “Woman”, “Girl”, and “Boy” to the fifth RPD household. As in the examples of Fig. As can be seen from Figures 8A-E, the 825 demographic category assignments meet the specified restrictions.

[0042] In some examples, the household demographic assignment engine 340 implements simulated annealing to further adjust the demographic category assignments made for the RPD households. An example operation of the household demographic assignment engine 340 for performing simulated annealing is shown in Fig. 9A-C. With reference to Fig. 9A, in the illustrated example, the household demographic assignment engine 340 has performed an initial household demographic assignment in which demographic category assignments containing both "boy" and "girl" are overrepresented by five households with respect to UE restrictions for that combination of demographic categories, and demographic category assignments containing both "man" and "girl" are underrepresented by five households with respect to UE restrictions for that combination of demographic categories. As shown in the Fig. 9B-C, the household demographic assignment engine 340 may perform simulated annealing to identify five households that meet the overrepresented demographic category assignment of “boy” and “girl” (see Fig. 9B) and shift the demographic category assignments of “girl” from those households to five households that do not have a demographic category assignment of both “boy” and “girl” (see Fig. 9C). The result is revised demographic category assignments, which reflect the Fig. Correct the over- and underrepresentation illustrated in Figure 9A.

[0043] In some examples, the household demographic allocation engine 340 decomposes the Fig. 7 into several smaller batches to reduce processing and storage requirements. For example, if a market contains 100,000 RPD households for which demographic categories are to be assigned, the household demographic assignment engine 340 may decompose the assignment problem into 100 groups of 1,000 households, or 1,000 groups of 100 households, etc. In such examples, the pseudocode 700 of Fig. 7 is adjusted such that the constraints associated with universe estimates (UEs) are scaled down by the ratio of the number of RPD households contained in the batch groups to the total number of RPD households, and pseudocode 700 is applied to perform demographic category assignment for each batch group. However, since such simple scaling may not result in solvable constraints for all batch groups, the tolerance levels (e.g., represented by “slack” in Fig. 7) to increase the probability that each batch group has a solvable demographic assignment.

[0044] With further reference to Fig. 3, the neural network-based demographic estimation system 300 includes the rating calculator 350 to determine rating data and / or other audience metrics by using the household demographic assignments determined for the RPD households by the household demographic assignment engine 340 to augment / combine the forum setting data from the participant database 315, which already has assigned demographic data, with the RPD setting data from the RPD database 325.

[0045] Examples mentioned above in connection with the Fig. 3, 4A-B, 5, 6A-C, 7, 8A-E, and 9A-C implement neural network processing of set-top box return path data to assign one or more demographic categories to RPD households. In other words, such disclosed examples assign demographic presence to households to satisfy known or estimated UEs for corresponding demographic categories (e.g., from forum data and / or other audience measurement techniques). Some examples disclosed above also combine the demographic presence categories assigned to the RPD households with known or estimated UEs of numbers of individual household members associated with the RPD households (e.g., from forum data and / or other audience measurement techniques) to assign individual household members (e.g., virtual household members) to corresponding ones of the RPD households and satisfy the UE objectives.

[0046] Further examples disclosed herein assign long-term visitors (e.g., virtual long-term visitors) as well as household members (also referred to as primary household members, such as virtual members for whom the household is their residence) to the RPD households to satisfy the demographic presence categories assigned to the households and known or estimated UEs of the numbers of long-term visitors associated with the RPD households (e.g., from forum data and / or other audience measurement techniques). At a higher level, some such disclosed visitor assignment techniques modify the targets (e.g., UEs) used by the exemplary demographic category assignment techniques disclosed above to incorporate visitors into the counts for the numbers of people in the various demographic classifications used.By modifying the input targets, the above-disclosed example demographic category assignment techniques (also referred to as household demographic assignment model (HDAM) techniques) can predict total household compositions that include both household members and long-term visitors (but do not distinguish one from the other at this point). Disclosed example visitor assignment techniques also include generating visitor vectors consistent with provided visitor distribution targets (e.g., from forum data and / or other audience measurement techniques) and then assigning the visitor vectors to RPD households that have predicted total household compositions capable of supporting the sizes and demographic compositions of the corresponding visitor vectors.

[0047] A block diagram of an exemplary neural network-based demographic estimation system structured to simplify the processing of Fig. 1 and Fig. 2 to estimate household demographics for primary household members and long-term visitors from set-top box return path data in accordance with the teachings of this disclosure is in Fig. 10. The exemplary neural network-based demographic estimation system 1000 includes the exemplary network interface 305, the exemplary forum setting data collector 310, the exemplary participant database 315, the exemplary RPD data collector 320, the exemplary RPD database 325, the exemplary feature generator 330, the exemplary neural network for demographic prediction 335, the exemplary household demographic assignment engine 340, the exemplary constraint database 345, and the exemplary score calculator 350 from the exemplary neural network-based demographic estimation system 300, which are described above in connection with Fig. 3. The exemplary neural network-based demographic estimation system 1000 also includes an exemplary demographic target adjuster 1050 and an exemplary visitor assignment engine 1055.

[0048] In the illustrated example of Fig. 10, the demographic target adjustment device 1050 adjusts the UE demographic targets to account for long-term visitor data. As disclosed above in connection with the neural network-based demographic estimation system 300, the HDAM technique of the neural network-based demographic estimation system 300 uses UEs of the corresponding various demographic categories, also referred to as demographic presence UEs, which specify the total number of RPD households to be assigned to corresponding ones of the various demographic categories. In some examples, the demographic presence UEs are determined based on (i) a total target UE for the number of RPD household members (e.g., determined or estimated from forum data, vendor data, and / or other audience measurement techniques) and (ii) demographic distribution data (e.g.,from forum data such as Nielsen People Monitoring (NPM) data, which are used to determine UEs for total numbers of RPD household members in each demographic category. The UEs for the total numbers of RPD household members in each demographic category are then used with demographic presence UEs obtained from the forum data (e.g., NPM data) to determine the demographic presence UEs for the HDAM technique. However, such unmodified UE targets do not account for the presence of long-term visitors. However, forum data (e.g., NPM data) can also provide information about the demographics (e.g., age and gender) of these tagged long-term visitors.This information is used by the example demographic target adjuster 1050 to calculate a scale vector comprising elements called scale factors to be applied to the UEs for the number of RPD household members in the various demographic categories. Each scale factor corresponds to a corresponding demographic category (e.g., age-sex bin). Using the index i to denote the i. -ten For each demographic category, the scale factors for the scale vector are determined based on the proportion of total individuals in the demographic category (e.g., age-sex bin) i who are long-term visitors, which can be seen in Equation 1 below. fi=Nv,i / (Np,i+Nv,i)

[0049] In the above equation 1, N v,i the number of visitors in the demographic category (e.g., age-gender bin) i (e.g., obtained from forum / NPM data) and Np,i is the number of primary household members in demographic category (e.g., age-sex bin) i (e.g., obtained from Forum / NPM data). The scale factor for demographic category i is: Si=1−fi=Np,i / (Np,i+Nv,i)

[0050] The scale factor S i In Equation 2 above, is used by the example demographic target adjuster 1050 to adjust (e.g., divide) the UE for the total number of RPD household members in demographic category i, which is used as the demographic presence UE for demographic category i for the HDAM technique, as described above in connection with the neural network-based demographic estimation system 300. In this manner, the example demographic target adjuster 1050 adjusts the presence UEs used by the HDAM technique to account for the presence of long-term visitors.

[0051] In some examples, the demographic target adjuster 1050 may adjust the demographic targets using target rates instead of the scale factors described above. In some such examples, the HDAM technique disclosed in the neural network-based demographic estimation system 300 technique is further modified to assign individual household members to RPD households based on the demographic categories (e.g., demographic presence) to which RPD households have been assigned, as disclosed above. The demographic target adjuster 1050 determines the target rates of occurrence of different numbers of people in the various demographic categories (e.g., as determined from forum data and / or other audience measurement techniques). The exemplary HDAM technique disclosed above may be further modified to assign the demographic categories (e.g.,Demographic presence) assigned to RPD households based on the target rates found by the example demographic target adjuster 1050 to assign individual members to RPD households according to those targets. If the target rates of occurrence of a first demographic category are 80% such that there is one person in a household in that category, 15% such that there are two people in a household in that category, and 5% such that there are three people in a household in that category, then a modified HDAM technique can select RPD households assigned the first demographic category such that 80% of those households are assigned to one person in that category, 15% of those households are assigned to two people in that category, and 5% of those households are assigned to three people in that category.

[0052] The visitor assignment engine 1055 of the illustrated example of Fig. 10 serves to distinguish visitors from primary household members in the overall household compositions assigned to RPD households based on the updated UE targets. In some examples, the target aggregate-level visitor population and demographic (e.g., age-gender) distributions are obtained from forum data in subscriber database 315. The example visitor assignment engine 1055 receives the output of the household demographic assignment engine 340 with the updated UE targets from the example demographic target adjuster 1050 for a given RPD household, which output can be viewed as a vector containing the total numbers of visitors and primary household members assigned to the RPD household for each possible demographic category.The example visitor assignment engine 1055 constructs a target population of visitors having the target demographics for assignment to the RPD households. The example visitor assignment engine 1055 is described below in connection with. Fig. 11 described in more detail.

[0053] Fig. 11 is a block diagram of the exemplary visitor assignment engine 1050 of the exemplary neural network-based demographics estimation system of Fig. 10. The example visitor assignment engine 1050 includes an example visitor demographic distribution calculator 1105, an example visitor household distribution calculator 1110, an example visitor vector generator 1120, and an example visitor vector assigner 1125.

[0054] In the illustrated example of Fig. 11, the visitor vector generator 1120 generates a set of visitor vectors that satisfy the set of visitor demographic targets. In examples disclosed herein, the example visitor vector generator 1120 iteratively generates visitor vectors representative of the visitor demographic compositions to be assigned to the RPD households. The example visitor vector generator 1120 generates the visitor vectors based on inputs that include percentages of long-term visitors in each demographic category and the percentages of households with correspondingly different possible numbers of visitors. In the illustrated example of Fig. 11, the visitor demographic distribution calculator 1105 determines the corresponding percentages of long-term visitors in one of the one or more demographic categories based on the forum data from the participant database 315. The visitor household distribution calculator 1110 of the illustrated example of Fig. 11 determines corresponding percentages of RPD households with corresponding numbers of long-term visitors based on the forum data from the participant database 315.

[0055] The visitor vector assignor 1125 of the illustrated example of Fig. 11 assigns the visitor vectors to RPD households based on corresponding probabilities that corresponding ones of the RPD households contain at least one visitor. The example visitor vector assigner 1125 iteratively assigns the visitor vectors generated by the visitor vector generator 1120 to valid RPD households. The example visitor vector assigner 1125 assigns the visitor vectors based on the household demographic assignments made by the example household demographic assignment engine 340 using the updated demographic targets from the example demographic target adjuster 1050. The example visitor vector assigner 1125 is also provided to calculate the probabilities that the RPD households have at least one visitor.The exemplary visitor vector assignor 1125 determines which RPD households are valid households for an exemplary visitor vector and creates a set of possible RPD households. In the illustrated example, the visitor vector assignor 1125 selects the RPD household that has the highest probability that the corresponding RPD household has at least one visitor from the set of possible RPD households, and the visitor vector assignor 1125 assigns the visitor vector to the selected RPD household. In the illustrated example, once a visitor vector has been assigned to the selected RPD household, the visitor vector assignor 1125 removes the selected RPD household from the set of available RPD households, so that it is no longer considered for further visitor vector assignments.In the illustrated example, RPD households are limited to having only one visitor vector assignment. However, in some examples, the RPD household may be allowed to have multiple visitor vector assignments. In some examples, the visitor vector assignment device 1125 may not remove the selected RPD household from the set of available RPD households to allow the RPD household to be assigned multiple visitor vectors. The example visitor vector assignment device 1125 iteratively assigns visitor vectors to RPD households until no visitor vectors remain. In some examples, if not all visitor vectors can be assigned to RPD households (e.g.,because at some point no remaining RPD budgets are valid for a given visitor vector), the example visitor vector generator 1120 collects the remaining unassigned visitor vectors, creates a new visitor pool from the remaining visitors represented in the unassigned visitor vectors, to generate new visitor vectors for the example visitor vector allocator 1125.

[0056] The output of the example visitor vector assignment engine 1125 includes demographic assignments for primary household members and long-term visitors for the various RPD households. This output is provided by the example visitor assignment engine 1055 to the example rating calculator 350, with the rating calculator 350 performing calculations based on the output, which includes both primary household members and long-term visitors.

[0057] In some examples, the visitor vector assignor 1125 may be simplified under a constraint that only one visitor is to be assigned to a particular RPD household. Such a simplification further requires that the number of visitors be less than or equal to the number of RPD households. In such an example, the example visitor vector assignor selects one of the demographic categories (e.g., one of the age-gender bins) to be used in assigning the visitor vectors. The example visitor vector assignor 1125 identifies the set of RPD households that are candidates for a visitor in the selected demographic category (e.g., selected age-gender bin) and ranks that set of RPD households based on the probability of having a visitor in that demographic category (e.g., age-gender bin).The exemplary visitor vector assigner 1125 selects the RPD household with the highest visitor probability and designates one of the individuals assigned to those RPD households in that demographic category (e.g., age-gender bin) as a long-term visitor. In such an example, the exemplary visitor vector assigner iterates until all visitors in the selected demographic category (e.g., age-gender bin) have been placed into RPD households and across the various demographic categories until the target number of visitors has been assigned to the RPD households.

[0058] While an exemplary implementation of the neural network-based demographic estimation system 300 in Fig. 3, one or more of the Fig. 3 may be combined, divided, rearranged, omitted, eliminated, and / or otherwise implemented. Furthermore, the example network interface 305, the example forum setting data collector 310, the participant database 315, the example RPD data collector 320, the example RPD database 325, the example feature generator 330, the example neural network for demographic prediction 335, the example household demographic assignment engine 340, the example constraint database 345, the example rating calculator 350, and / or, more generally, the example neural network-based demographic estimation system 300 of Fig. 3 be implemented by hardware, software, firmware and / or any combination of hardware, software and / or firmware. Therefore, for example, each of the example network interface 305, the example forum settings data collector 310, the participant database 315, the example RPD data collector 320, the example RPD database 325, the example feature generator 330, the example neural network for demographic prediction 335, the example household demographic assignment engine 340, the example constraint database 345, the example rating calculator 350, and / or, more generally, the example neural network-based demographic estimation system 300 could be implemented by one or more analog or digital circuits, logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)),programmable logic device(s) (PLD(s)), field-programmable gate arrays (FPGAs), and / or field-programmable logic device(s) (FPLD(s)). Upon reading any of the apparatus or system claims of this patent to cover a purely software and / or firmware implementation, at least one of the exemplary neural network-based demographics estimation system 300, the exemplary network interface 305, the exemplary forum setting data collector 310, the participant database 315, the exemplary RPD data collector 320, the exemplary RPD database 325, the exemplary feature generator 330, the exemplary demographics prediction neural network 335, the exemplary household demographics assignment engine 340, the exemplary constraint database 345, and / or the exemplary scoring calculator 350 is hereby expressly defined asa non-transitory computer-readable storage device or a storage disk, such as a memory, a digital versatile disk (DVD), a compact disc (CD), a Blu-ray disk, etc., including the software and / or firmware. Further, the exemplary neural network-based demographic estimation system 300 may comprise, in addition to or instead of the software and / or firmware described in FIG. Fig. 3 may include one or more elements, processes, and / or devices and / or include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the term "in communication," including variations thereof, includes direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0059] While an exemplary implementation of the neural network-based demographic estimation system 1000 in the Fig. 10 and Fig. 11, one or more of the elements, processes and / or devices shown in the Fig. 10 and Fig. 11 may be combined, split, rearranged, omitted, eliminated, and / or implemented in other ways. Furthermore, the example network interface 305, the example forum setting data collector 310, the participant database 315, the example RPD data collector 320, the example RPD database 325, the example feature generator 330, the example neural network for demographic prediction 335, the example household demographic assignment engine 340, the example constraint database 345, the example demographic target adjustment device 1050, the example visitor assignment engine 1055, the example rating calculator 350, and / or, more generally, the example neural network-based demographic estimation system 1000 of Fig. 10 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Therefore, for example, each of the example network interface 305, the example forum setting data collector 310, the participant database 315, the example RPD data collector 320, the example RPD database 325, the example feature generator 330, the example neural network for demographic prediction 335, the example household demographic assignment engine 340, the example constraint database 345, the example demographic target adjustment device 1050, the example visitor assignment engine 1055, the example rating calculator 350, and / or, more generally, the example neural network-based demographic estimation system 1000 could be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers,Graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)), field programmable gate arrays (FPGAs) and / or field programmable logic device(s) (FPLD(s)). Upon reading any of the apparatus or system claims of this patent to cover a pure software and / or firmware implementation, at least one of the example neural network-based demographics estimation system 1000, the example network interface 305, the example forum setting data collector 310, the participant database 315, the example RPD data collector 320, the example RPD database 325, the example feature generator 330, the example neural network for demographics prediction 335, the example household demographics assignment engine 340, the example constraint database 345,the exemplary demographic target adjustment device 1050, the exemplary visitor assignment engine 1055, and / or the exemplary scoring calculator 350 is hereby expressly defined to include a non-transitory computer-readable storage device or a storage disk, such as a memory, a digital versatile disk (DVD), a compact disc (CD), a Blu-ray disk, etc., including the software and / or firmware. Further, the exemplary neural network-based demographic estimation system 1000 may, in addition to or instead of the devices described in FIGS. Fig. 10 and Fig. 11 may include one or more elements, processes, and / or devices, and / or include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the term "in communication," including variations thereof, includes direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0060] In examples disclosed herein, the example feature generator 330 implements means for generating features from reverse path data. The example neural network 335 implements means for processing the features generated from the reverse path data to predict demographic classification probabilities for reverse path data households. The example demographic assignment engine 340 implements means for assigning one or more demographic categories to corresponding ones of the reverse path data households. The example visitor assignment engine 1055 implements means for assigning virtual visitors to at least a subset of the corresponding ones of the reverse path data households. The example demographic target adaptor 1050 implements means for updating demographic targets to account for visitor presence.The example visitor vector generator 1120 implements means for generating a visitor vector containing a first number of visitors. The example visitor vector assigner 1125 implements means for assigning the visitor vector to a first of the return path data households. The example visitor demographics distribution calculator 1105 implements means for determining corresponding percentages of visitors in one of the one or more demographic categories. The example visitor household distribution calculator 1110 implements means for determining corresponding percentages of the return path data households with corresponding visitor numbers.

[0061] A flowchart illustrating example hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the example neural network-based demographic estimation system 300 is shown in Fig. 12. In this example, the machine-readable instructions may be one or more executable programs or portions thereof for execution by a computer processor, such as the processor 2012 shown in the example processor platform 2000 described below in connection with Fig. 20. The one or more programs, or the portion(s) thereof, may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, a floppy disk, a hard disk, a DVD, a Blu-ray Disk™, or a memory connected to the processor 2012, but the entire program(s) and / or portions thereof may alternatively be executed by a device other than the processor 2012 and / or be embodied in firmware or dedicated hardware. Furthermore, although the example program(s) are described with reference to the Fig. 12. With reference to the flowchart illustrated in Fig. For example, in the flowchart illustrated in Figure 12, the order of execution of the blocks may be changed and / or some of the described blocks may be changed, eliminated, combined, and / or divided into multiple blocks. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, a comparator, an operational amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

[0062] Flowcharts illustrating exemplary hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the exemplary neural network-based demographic estimation system 1000 are shown in the Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and Fig. 19. In this example, the machine-readable instructions may be one or more executable programs or portions thereof for execution by a computer processor, such as the processor 2012 shown in the example processor platform 2100 described below in connection with Fig. 21. The one or more programs, or the portion(s) thereof, may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, a floppy disk, a hard disk, a DVD, a Blu-ray Disk™, or a memory connected to the processor 2100, but the entire program(s) and / or portions thereof could alternatively be executed by a device other than the processor 2100 and / or be embodied in firmware or dedicated hardware. Furthermore, many other methods for implementing the example neural network-based demographic estimation system 1000 may alternatively be used, although the example program(s) are described with reference to the Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and Fig. 19. With reference to the flow charts shown in the Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and Fig. For example, in the flowcharts illustrated in Figure 19, the order of execution of the blocks may be changed and / or some of the described blocks may be changed, eliminated, combined, and / or divided into multiple blocks. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, a comparator, an operational amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

[0063] The machine-readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a packaged format, etc. Machine-readable instructions as described herein may be stored as data (e.g., portions of instructions, code, code representations, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored on one or more storage devices and / or computing devices (e.g., servers). The machine-readable instructions may include one or more of installation, modification, adaptation, updating, combination, addition, configuration, decryption, decompression, unpacking, distribution, reassignment, etc.require to make them directly readable and / or executable by a computing device and / or other machine. For example, the machine-readable instructions may be stored in multiple parts that are individually compressed, encrypted, and stored on separate computing devices, which parts, when decrypted, decompressed, and combined, form a set of executable instructions that implement a program such as the one described herein. In another example, the machine-readable instructions may be stored in a state in which they can be read by a computer, but require the addition of a library (e.g., a dynamic linking library), a software development kit (SDK), an application programming interface (API), etc., to execute the instructions on a particular computing device or other device.In another example, the machine-readable instructions may need to be configured (e.g., settings saved, data entered, network addresses recorded, etc.) before the machine-readable instructions and / or corresponding program(s) can be executed in whole or in part. Therefore, the disclosed machine-readable instructions and / or corresponding program(s) are intended to encompass such machine-readable instructions and / or program(s) regardless of the respective format or state of the machine-readable instructions and / or program(s) when stored or otherwise at rest or in transit.

[0064] As mentioned above, the exemplary processes of Fig. 12, Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and / or 19 may be implemented using executable instructions (e.g., computer- and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium, such as a hard disk drive, flash memory, read-only memory, compact disc, digital versatile disk, cache, random access memory, and / or other storage device or disk in which information is stored for any duration (e.g., for extended periods of time, permanently, for short periods of time, for temporarily buffering and / or caching the information). As used herein, the term non-transitory computer-readable medium is expressly defined to include any type of computer-readable storage device and / or disk, and to exclude propagating signals and to exclude transmission media.Unless otherwise specified, the terms “computer readable” and “machine readable” are considered equivalent herein.

[0065] "Comprise" and "comprise" (and all forms and tenses thereof) are used herein as open-ended terms. Therefore, when a claim uses any form of "comprise" or "comprise" (e.g., comprises, has, comprising, having, etc.) as a generic term or within claim language of any kind, it is understood that additional elements, terms, etc. may be present without being outside the scope of the corresponding claim or language. When the term "at least" is used as a transitional term in, for example, a generic term of a claim, it is open-ended in the same way that the terms "comprise" and "comprise" are open-ended.The term “and / or,” when used, for example, in a form such as A, B, and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, and (7) A with B and with C. As used herein in the context of describing structures, components, elements, objects, and / or things, the phrase “at least one of A and B” is intended to refer to implementations that include (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, the phrase “at least one of A or B,” as used herein in the context of describing structures, components, elements, objects, and / or things, is intended to refer to implementations that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.As used herein in connection with describing the performance or execution of processes, instructions, actions, activities, and / or steps, the term “at least one of A and B” shall refer to implementations that include any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly as used herein in connection with describing the performance or execution of processes, instructions, actions, activities, and / or steps, the term “at least one of A or B” shall refer to implementations that include any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0066] An example program 1200 that can be executed to implement the example neural network-based demographic estimation system 300 of Fig. 3 is to be implemented by the Fig. 12. With reference to the preceding figures and the associated written descriptions, the example program 1000 of Fig. 12 at block 1205, the exemplary forum preference data collector 310 of the neural network-based demographic estimation system 300 collects the participant preference data, as described above. At block 1210, the exemplary feature generator 330 of the neural network-based demographic estimation system 300 generates feature vectors (such as the vectors described in Table 1 above) for the participant households based on the collected participant data, as described above. At block 1215, the feature generator 330 applies the participant feature vectors generated at block 1210 to the exemplary demographic prediction neural network 335 of the neural network-based demographic estimation system 300 to train the demographic prediction neural network 335 and predict demographic classification probabilities for the corresponding participant households, as described above.

[0067] At block 1220, the example RPD data collector 320 of the neural network-based demographic estimation system 300 collects RPD setting data, as described above. At block 1225, the example feature generator 330 generates feature vectors (such as the vectors described in Table 1 above) for the RPD households based on the collected RPD setting data, as described above. At block 1230, the feature generator 330 applies the RPD feature vectors generated at block 1225 to the trained neural network for demographic prediction 335 of the neural network-based demographic estimation system 300 to predict demographic classification probabilities for the corresponding RPD households, as described above.At block 1235, the example demographic assignment engine 340 of the neural network-based demographic estimation system 300 obtains the demographic assignment constraints from the example constraint database 345, as described above. At block 1240, the example demographic assignment engine 340 of the neural network-based demographic estimation system 300 uses the demographic classification probabilities determined at block 1230 to assign demographic categories to corresponding ones of the RPD households, as described above.At block 1245, the example scoring calculator 350 of the neural network-based demographic estimation system 300 augments / combines the forum setting data collected at block 1205, which already has associated demographic data, with the RPD voting data collected at block 1220 based on the demographic categories assigned to the corresponding one of the RPD households at block 1245, as described above.

[0068] An example program 1300 that can be executed to implement the example neural network-based demographic estimation system 1000 of Fig. 10 is to be implemented by the Fig. 13. With reference to the preceding figures and the associated written descriptions, the example program 1300 of Fig. 13 at block 1305, the exemplary forum preference data collector 310 of the neural network-based demographic estimation system 1000 collects the participant preference data, as described above. At block 1310, the exemplary feature generator 330 of the neural network-based demographic estimation system 1000 generates feature vectors (such as the vectors described in Table 1 above) for the participant household based on the collected participant data, as described above. At block 1315, the feature generator 330 applies the participant feature vectors generated at block 1310 to the exemplary demographic prediction neural network 335 of the neural network-based demographic estimation system 1000 to train the demographic prediction neural network 335 and predict demographic classification probabilities for the corresponding participant households, as described above.

[0069] At block 1320, the example RPD data collector 320 of the neural network-based demographic estimation system 1000 collects RPD setting data, as described above. At block 1325, the example feature generator 330 generates feature vectors (such as the vectors described in Table 1 above) for the RPD households based on the collected RPD setting data, as described above. At block 1330, the feature generator 330 applies the RPD feature vectors generated at block 1325 to the trained neural network for demographic prediction 335 of the neural network-based demographic estimation system 1000 to predict demographic classification probabilities for the corresponding RPD households, as described above. At block 1335, the demographic target adjuster 1050 updates the demographic targets for the household demographic assignment engine 340.As described in more detail below, the example flowcharts of the . Fig. 14 and Fig. 15 sample instructions that can be implemented to update demographic targets and accommodate long-term visitors.

[0070] At block 1340, the example demographic assignment engine 340 obtains the demographic assignment constraints from the example constraint database 345, as described above in connection with the Fig. 7 and Fig. 8A-E. At block 1345, the example demographic assignment engine 340 of the neural network-based demographic estimation system 1000 uses the demographic classification probabilities determined at block 1330, the demographic assignment constraints obtained at block 1345, and the demographic targets updated at block 1335 to assign demographic categories to corresponding ones of the RPD households, as described above.

[0071] At block 1350, the example visitor assignment engine 1055 assigns the visitors to households. The example visitor assignment engine 1055 uses the demographic category assignments of the corresponding RPD households determined by the example household demographic assignment engine 340 at block 1345 to assign the visitors to the corresponding RPD households. As described in more detail below, the example flowchart of Fig. 16 example instructions that can be implemented to assign visitors to households.

[0072] At block 1355, the example rating calculator 350 of the neural network-based demographic estimation system 1000 augments / combines the forum setting data collected at block 1305, which already has associated demographic data, with the RPD setting data collected at block 1320 based on the demographic categories assigned to the corresponding one of the RPD households at block 1345 and the visitors assigned to the corresponding one of the RPD households at block 1350, as described above.

[0073] A first example program 1340a may be executed to implement the example demographic target adjustment device 1040 of Fig. 10, and / or processing at block 1340 of Fig. 13 is through the Fig. 14. With reference to the preceding figures and the associated written descriptions, the example program 1340a of Fig. 14 illustrates execution at block 1405, in which the demographic target adjuster 1050 collects the demographic targets from the demographic prediction neural network 335. The exemplary demographic prediction neural network 335 provides predicted demographic target variables for primary household members. At block 1410, the exemplary demographic target adjuster 1050 calculates the scale factors for the demographic categories as described above in connection with Fig. 10. At block 1415, the example demographic target adjuster 1050 adjusts the demographic targets by the corresponding scale factors found at block 1410. The example demographic target adjuster 1050 adjusts the demographic targets to account for the presence of long-term visitors when used by the example household demographic assignment engine 340. The example program 1340a of Fig. 14 then ends and returns to the example program 1300 of Fig. 13 back.

[0074] A second exemplary alternative program 1340b that may be executed to implement the exemplary demographic target adjustment device 1040 of Fig. 10 and / or the processing at block 1340 of Fig. 13 is to be implemented by the Fig. 15. The example program 1340b of Fig. 15 assumes that the HDAM technique described above for assigning individual household members to RPD households based on the demographic categories assigned to the RPD households will be further modified, as described above in connection with Fig. 10. With reference to the preceding figures and the associated written descriptions, the example program 1340b of Fig. 15 illustrates execution at block 1505, where demographics target adjuster 1050 determines target visitor occurrence rates from the forum data in subscriber database 315. The example demographics target adjuster 1050 determines target visitor occurrence rates for different numbers of visitors in the various demographic categories (e.g., as determined from forum data and / or other audience measurement techniques). For example, the target occurrence rates of a first demographic category may be 80% that one person is in a household of that category, 15% that two people are in a household in that same category, and 5% that three people are in a household in that same category.

[0075] At block 1510, the example demographic assignment engine 340 assigns individuals to the households according to the target rates found at block 1505. Referring to the same example above for block 1505, the example household demographic assignment engine 340 selects RPD households that have been assigned the first demographic category such that 80% of these households are assigned to one individual in this category, 15% of these households are assigned to two individuals in this category, and 5% of these households are assigned to three individuals in this category. In the example program 1340b of Fig. 15, the program 1340b ends and returns to the example program 1300 at block 1350 of Fig. 13 back.

[0076] An example program 1350 may be executed to implement the example visitor assignment engine 1055 of Fig. 10, and / or processing at block 1350 of Fig. 13 is replaced by the Fig. 16. With reference to the preceding figures and the associated written descriptions, the example program 1350 of Fig. 16 illustrates execution at block 1605, where visitor assignment engine 1050 collects subscriber preference data for the identified long-term visitors from subscriber database 315. At block 1610, example visitor vector generator 1120 receives the assigned demographics for the households determined by household demographic assignment engine 1055, as described above in connection with Fig. 3 is described.

[0077] At block 1615, the example visitor demographic distribution calculator 1105 determines the percentages of long-term visitors in each of the demographic categories. The example visitor demographic distribution calculator 1105 uses the subscriber data from the subscriber database 315 to determine what percentage of people in each of the demographic categories (e.g., age-gender bins) are long-term visitors. At block 1620, the example visitor household distribution calculator 1110 determines the percentages of households with corresponding numbers of long-term visitors. The example visitor household distribution calculator 1110 uses the subscriber data from the subscriber database 315 to determine the percentage of households each having 1, 2, 3, 4, etc., visitors.For example, the example visitor household distribution calculator 1110 determines what percentage of households have one visitor and then what percentage of households have two visitors, and so on.

[0078] At block 1625, the example visitor vector generator 1120 generates visitor vectors. The example visitor vector generator 1120 uses the percentages determined by the example visitor demographic distribution calculator 1105 and the example visitor household distribution calculator 1110 to generate the visitor vectors. As described in more detail below, the example flowchart of Fig. 17 example instructions that can be implemented to generate the visitor vectors.

[0079] At block 1635, the exemplary visitor vector assigner 1125 assigns visitor vectors to the households. The exemplary visitor vector assigner 1125 uses the subscriber data from the subscriber database 315 when assigning the visitor vectors generated by the visitor vector generator at block 1625 to the RPD households. As described in more detail below, the exemplary flowcharts of the Fig. 18 and Fig. 19 illustrates exemplary instructions that can be implemented to assign the visitor vectors to the households. After execution of block 1635, the example program 1350 of Fig. 16 and returns to the example program 1300 of Fig. 13 back.

[0080] An example program 1625 may be executed to implement the example visitor vector generator 1120 of Fig. 11, and / or processing at block 1625 of Fig. 16 is through the Fig. 17. With reference to the preceding figures and associated written descriptions, the example program 1625 of Fig. 17 illustrates execution at block 1705, where the example visitor vector generator 1120 multiplies the total assigned demographics for each household by the visitor percentage for each demographic category. The example visitor vector generator 1120 multiplies the total number of people in each demographic category assigned to the RPD households, determined by the example household demographic assignment engine 340, by the percentage of long-term visitors in each demographic category, determined by the example visitor demographic distribution calculator 1105. The example visitor vector generator 1120 determines how many visitors to include in each demographic category by performing the aforementioned multiplication.

[0081] At block 1710, the example visitor vector generator 1120 creates a visitor pool. The example visitor vector generator 1120 creates a visitor pool with the total number of expected visitors in each demographic category (e.g., each age-gender bin). At block 1715, the example visitor vector generator 1120 creates a visitor vector of a selected size. In examples disclosed herein, the selected size is determined based on a random number generator, where the probability of a particular size being selected corresponds to the input percentage of households having the given number (e.g., 1, 2, 3, 4, etc.) of visitors, as determined by the example visitor household distribution calculator 1110. However, other selection methods may be used additionally or alternatively.

[0082] At block 1720, the example visitor vector generator 1120 selects a number of visitors from the visitor pool based on the selected visitor vector size and places the selected visitors into the generated visitor vector. The example visitor vector generator 1120 selects a number of visitors from the visitor pool according to the selected visitor vector size determined at block 1715. The example visitor vector generator 1120 then places the selected visitors into the visitor vector generated at block 1715. At block 1725, the example visitor vector generator 1120 determines whether any visitors remain in the visitor pool. If the example visitor vector generator 1120 determines that visitors remain in the visitor pool, the example program 1625 returns from Fig. 17 returns to block 1715, where another visitor vector of a selected size is generated. If the example visitor vector generator 1120 determines that no visitors remain in the visitor pool, the example program 1625 of Fig. 17 and returns to example program 1350 of Fig. 16 back.

[0083] A first example program 1635a may be executed to implement the example visitor vector allocation device 1125 of Fig. 11, and / or processing at block 1635 of Fig. 16 is through the Fig. 18. With reference to the preceding figures and the associated written descriptions, the example program 1635a of Fig. 18 illustrates execution at block 1805, where the example visitor vector assignor 1125 accesses the generated visitor vectors from the visitor vector generator 1120 and the individual member assignments to the RPD households (which include, but do not distinguish, primary household members and visitors) determined by the household demographic assignment engine 340 from the visitor vector generator 1120. The example visitor vector assignor 1125 receives the RPD household assignments including primary household members and long-term visitors based on the updated demographic targets provided by the demographic target adjuster 1050.

[0084] At block 1810, the example visitor vector assigner 1125 determines the probabilities that each household includes at least one visitor corresponding to the set of households. In some examples, the probabilities may be the same for all RPD households, so that each household has an equal probability of including a visitor. In some examples, the demographic prediction neural network 335 disclosed above may be adapted to output a probability that a particular RPD household has a visitor based on the subscriber preference data from the subscriber database 315.

[0085] At block 1815, the exemplary visitor vector assigner 1125 selects a visitor vector from the visitor vectors generated by the exemplary visitor vector generator 1120. At block 1820, the exemplary visitor vector assigner 1125 generates a list of valid households for placement of the selected visitor vector. An RPD household is valid if the RPD household is assigned (by the exemplary demographic assignment engine 340 with the modified demographic targets from the demographic target adjustment engine 1050) the same number or more people in each demographic category than are included in the selected visitor vector. The visitor vector assigner 1125 also ensures that there is at least one person in the adult demographic category assigned to the RPD household who is not a long-term visitor.

[0086] At block 1825, the exemplary visitor vector assignor 1125 selects a household from the list of valid households determined at block 1815 that has the highest probability of being a visitor determined at block 1810. At block 1830, the exemplary visitor vector assignor 1125 assigns the visitor vector to the selected RPD household. At block 1835, the exemplary visitor vector assignor 1125 removes the selected household from the remaining set of available households for visitor assignments.

[0087] At block 1840, the example visitor vector allocator 1125 determines whether any visitor vectors remain. If the example visitor vector allocator 1125 determines that any visitor vectors remain, the example program 1635a returns from Fig. 18 returns to block 1815, where the exemplary visitor vector allocator 1125 selects a visitor vector. If the exemplary visitor vector allocator 1125 determines that no visitor vectors remain, the example program 1635a of Fig. 18 and returns to program 1350 of Fig. 16 back.

[0088] A second alternative program 1635b may be executed to implement the exemplary visitor vector allocation device 1125 of Fig. 11 under the restriction that only one visitor is to be assigned to a given household, which is achieved by the Fig. 19. With reference to the preceding figures and the associated written descriptions, the example program 1635b of Fig. 19 illustrates execution at block 1900, where the exemplary visitor vector assignor 1125 receives the generated visitor vectors from the visitor vector generator 1120 and the individual member assignments to the RPD households (which include, but do not distinguish, primary household members and visitors) determined by the household demographic assignment engine 340 from the visitor vector generator 1120. The exemplary visitor vector assignor 1125 receives the RPD household assignments including primary household members and long-term visitors based on the updated demographic targets provided by the demographic target adjuster 1050.

[0089] At block 1905, the example visitor vector assigner 1125 determines the probabilities that each household includes at least one visitor corresponding to the set of households. In some examples, the probabilities may be the same for all RPD households, so that each household has an equal probability of including a visitor. In some examples, the demographic prediction neural network 335 disclosed above may be adapted to output a probability that a particular RPD household has a visitor based on the subscriber preference data from the subscriber database 315.

[0090] At block 1910, the example visitor vector allocator 1125 selects a demographic category. At block 1915, the example visitor vector allocator 1125 identifies a set of RPD households that may have a visitor in the selected demographic category. At block 1920, the example visitor vector allocator 1125 orders the set of RPD households based on the probability of having a visitor in the selected demographic category determined at block 1905. At block 1925, the example visitor vector allocator 1125 selects the household from the ordered set of households. In the examples disclosed herein, the example visitor vector allocator 1125 selects the RPD household from the ordered set that has the highest probability of having at least one visitor.At block 1930, the exemplary visitor vector assigner 1125 assigns the visitor of the selected demographic category to the selected RPD household. At block 1935, the exemplary visitor vector assigner 1125 removes the selected household from the ordered set.

[0091] At block 1940, the example visitor vector allocator 1125 determines whether there are any visitors remaining in the demographic category. If the example visitor vector allocator 1125 determines that there are any visitors remaining in the demographic category, the example program 1635b returns from Fig. 19 returns to block 1915, where the exemplary visitor vector allocator 1125 identifies a set of households that may have a visitor in the selected demographic category. If the exemplary visitor vector allocator 1125 determines that no visitors remain in the demographic category, the exemplary program 1635 continues from Fig. 19 continues with block 1945.

[0092] At block 1945, the example visitor vector allocator 1125 determines whether any demographic categories remain. If the example visitor vector allocator 1125 determines that any demographic categories remain, the example program 1635 returns from Fig. 19 returns to block 1910, where the exemplary visitor vector assignor 1125 selects a demographic category. If the exemplary visitor vector assignor 1125 determines that no demographic categories remain, the example program 1635 of Fig. 19 and returns to example program 1350 of Fig. 16 back.

[0093] Fig. 20 is a block diagram of an exemplary processor platform 2000 structured to execute the instructions of Fig. 12 to implement the exemplary neural network-based demographic estimation system 300 of Fig. 3. The processor platform 2000 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smartphone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet device, or any other type of computing device.

[0094] The processor platform 2000 of the illustrated example includes a processor 2012. The processor 2012 of the illustrated example is hardware. The processor 2012 may be implemented, for example, by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 2012 may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 2012 implements the example forum settings data collector 310, the example RPD data collector 320, the example feature generator 330, the example demographics assignment engine 340, and the example scoring calculator 350.

[0095] The processor 2012 of the illustrated example includes a local memory 2013 (e.g., a cache). The processor 2012 of the illustrated example is in communication with a main memory, which includes a volatile memory 2014 and a non-volatile memory 2016, via an interconnect 2018. The interconnect 2018 may be implemented by a bus, one or more point-to-point connections, etc., or a combination thereof. The volatile memory 2014 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of random access memory device. The non-volatile memory 2016 may be implemented by flash memory and / or any other type of memory device. Access to the main memory 2014, 2016 is controlled by a memory controller.

[0096] The processor platform 2000 of the illustrated example also includes an interface circuit 2020. The interface circuit 2020 may be implemented using any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near-field communication (NFC) interface, and / or a PCI Express interface. In this example, the interface circuit 2020 implements the network interface 305.

[0097] In the illustrated example, one or more input devices 2022 are connected to the interface circuitry 2020. The input device(s) 2022 allow a user to input data and / or commands into the processor 2012. The input device(s) may be implemented, for example, by an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, a trackbar (such as an isopoint), a speech recognition system, and / or any other human-machine interface. Also, many systems, such as the processor platform 2000, may allow the user to control the computer system and provide data to the computer using physical gestures, such as, without limitation, hand or body movements, facial expressions, and facial recognition.

[0098] One or more output devices 2024 are also connected to the interface circuitry 2020 of the illustrated example. The output devices 2024 may be implemented, for example, by display devices (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, a printer, and / or speakers. The interface circuitry 2020 of the illustrated example therefore typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0099] The interface circuit 2020 of the illustrated example also includes a communication device, such as a transmitter, a receiver, a transceiver, a modem, a local gateway, a wireless access point, and / or a network interface, to facilitate the exchange of data with external machines (e.g., computing devices of any type) over a network 2026. Communication may occur, for example, via an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a microwave system, a cellular phone system, etc.

[0100] The processor platform 2000 of the illustrated example also includes one or more mass storage devices 2028 for storing software and / or data. Examples of such mass storage devices 2028 include floppy disk drives, hard disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID), and digital versatile disk (DVD) drives. In some examples, the mass storage device(s) 2028 may implement the subscriber database 315, the RPD database 325, and / or the constraint database 345. Additionally or alternatively, in some examples, the volatile memory 2014 may implement the subscriber database 315, the RPD database 325, and / or the constraint database 345.

[0101] The machine-executable instructions 2032 corresponding to the instructions of Fig. 12 may be stored in the mass storage device 2028, in the volatile memory 2014, in the non-volatile memory 2016, in the local memory 2013, and / or on a removable non-volatile computer-readable storage medium, such as a CD or DVD 2036.

[0102] Fig. 21 is a block diagram of an exemplary processor platform 2000 structured to execute the instructions of Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and Fig. 19 to implement the exemplary neural network-based demographic estimation system 1000 of Fig. 10. The processor platform 2100 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smartphone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet device, or any other type of computing device.

[0103] The processor platform 2100 of the illustrated example includes a processor 2112. The processor 2112 of the illustrated example is hardware. The processor 2112 may be implemented, for example, by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 2112 may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 2112 implements the example forum settings data collector 310, the example RPD data collector 320, the example feature generator 330, the example household demographic assignment engine 340, the example demographic target adjustment device 1050, the example visitor assignment engine 1055, and the example scoring calculator 350.

[0104] The processor 2112 of the illustrated example includes a local memory 2113 (e.g., a cache). The processor 2113 of the illustrated example is in communication with a main memory, which includes a volatile memory 2113 and a non-volatile memory 2116, via an interconnect 2118. The interconnect 2118 may be implemented by a bus, one or more point-to-point connections, etc., or a combination thereof. The volatile memory 2114 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of random access memory device. The non-volatile memory 2116 may be implemented by flash memory and / or any other type of memory device. Access to the main memory 2114, 2116 is controlled by a memory controller.

[0105] The processor platform 2100 of the illustrated example also includes an interface circuit 2120. The interface circuit 2120 may be implemented using any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near-field communication (NFC) interface, and / or a PCI Express interface. In this example, the interface circuit 2120 implements the network interface 305.

[0106] In the illustrated example, one or more input devices 2120 are connected to the interface circuitry 2120. The input device(s) 2122 allow a user to input data and / or commands into the processor 2112. The input device(s) may be implemented, for example, by an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, a trackbar (such as an isopoint), a speech recognition system, and / or any other human-machine interface. Also, many systems, such as the processor platform 2100, may allow the user to control the computer system and provide data to the computer using physical gestures, such as, without limitation, hand or body movements, facial expressions, and facial recognition.

[0107] One or more output devices 2124 are also connected to the interface circuit 2120 of the illustrated example. The output devices 2124 may be implemented, for example, by display devices (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-place switching display (IPS), a touchscreen, etc.), a tactile output device, a printer, and / or speakers. The interface circuit 2120 of the illustrated example therefore typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0108] The interface circuit 2120 of the illustrated example also includes a communication device, such as a transmitter, a receiver, a transceiver, a modem, a local gateway, a wireless access point, and / or a network interface, to facilitate the exchange of data with external machines (e.g., computing devices of any type) over a network 2126. Communication may occur, for example, via an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a microwave system, a cellular phone system, etc.

[0109] The processor platform 2100 of the illustrated example also includes one or more mass storage devices 2128 for storing software and / or data. Examples of such mass storage devices 2128 include floppy disk drives, hard disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID), and digital versatile disk (DVD) drives. In some examples, the mass storage device(s) 2128 may implement the subscriber database 315, the RPD database 325, and / or the constraint database 345. Additionally or alternatively, in some examples, the volatile memory 2124 may implement the subscriber database 315, the RPD database 325, and / or the constraint database 345.

[0110] The machine-executable instructions 2132, which correspond to the instructions of the Fig. 13, Fig. 14, Fig. 15, Fig. 16, Fig. 17, Fig. 18 and Fig.19 may be stored in the mass storage device 2128, in the volatile memory 2114, in the non-volatile memory 2116, in the local memory 2113, and / or on a removable non-volatile computer-readable storage medium, such as a CD or DVD 2136.

[0111] From the foregoing, it can be seen that exemplary methods, apparatus, and articles of manufacture have been disclosed that implement neural network processing of set-top box return path data to estimate household demographics. An exemplary neural network-based demographics estimation system 1000 disclosed above uses a neural network with a TDDL followed by a recurrent LSTM network layer to predict demographic classifications of a household (e.g., forum household for training and RPD households after training) from viewing data (e.g., subscriber setting data for training and RPD setting data after training).The example neural network-based demographic estimation system 1000 groups the viewing data for a household into viewing blocks describing corresponding viewing sessions, where a viewing block indicates the day of the week, the day of the year, the quarter hour of the day, the channel change rate, and the minutes of each possible network. In some examples, the viewing blocks are limited to 60 minutes. In some examples, viewing blocks for a given household are combined and processed by the TDDL to generate a condensed feature set for the household's viewing sessions. The condensed feature set is then processed by the LSTM to generate a condensed summary feature vector summarizing the viewing history for the household.The condensed summary feature vector is merged with additional household characteristics, such as total television consumption, the number of recorded viewing blocks, and the number of television tuners in the household, to produce a merged summary feature vector for the household. The merged feature vector is then applied to one or more additional hidden layers, which output a classification vector indicating the probability that the household belongs to the various possible demographic classes. Mixed integer programming is then used to solve an objective function based on the demographic classification probabilities output by the neural network and subject it to a set of constraints to assign one or more demographic categories to corresponding RPD households that provide the RPD setting data.

[0112] The disclosed methods, apparatus, and products of manufacture improve the efficiency of using a computing device by enabling RPD setting data to be combined with participant setting data in an audience measurement processing system. Combining RPD setting data with available forum data can greatly increase the amount of data accessible to the audience measurement processing system to predict audience metrics (e.g., ratings). Such increased data can improve the statistical completeness of the input data and thereby reduce the associated statistical bias in the results generated by the audience measurement processing system. The disclosed methods, apparatus, and products of manufacture are accordingly directed to one or more improvements in the operation of a computer.

[0113] Although certain exemplary methods, devices, and articles of manufacture have been disclosed herein, the scope of this patent is not limited to these. Rather, this utility model covers all methods, devices, and articles of manufacture that most fully fall within the scope of the claims of this patent. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 16 / 230,620

[0001] US 62 / 743,925

[0001] US 62 / 841,641

[0001] US 16 / 230,620

[0001] US 62 / 743,925

[0001]

Claims

[1] Demographic estimation system, comprising: a feature generator for generating features from return path data (RPD) reported by set-top boxes associated with return path data households; a neural network to process the features generated from the return path data and predict demographic classification probabilities for the return path data households, wherein the neural network is to be trained based on forum data reported by measurement devices that monitor media devices associated with the participant households; a demographic assignment engine for assigning one or more demographic categories to corresponding ones of the return path data households based on the predicted demographic classification probabilities; and a visitor assignment engine for assigning virtual visitors to at least a subset of the corresponding ones of the return path data households based on the one or more demographic categories assigned to the corresponding ones of the return path data households. [2] The demographic estimation system of claim 1, further comprising a demographic target adjustment device for updating demographic targets to take into account the presence of visitors. [3] The demographic estimation system of claim 2, wherein the demographic target adjustment device applies scale factors to corresponding ones of demographic targets to update demographic targets and take into account the presence of visitors. [4] The demographic estimation system of claim 1, wherein the visitor assignment engine comprises: a visitor vector generator for generating a visitor vector containing a first number of visitors; and a visitor vector assigning device for assigning the visitor vector to a first of the return path data households based on respective probabilities that respective ones of the return path data households include at least one visitor, the probabilities being based on the forum data. [5] The demographic estimation system of claim 4, wherein the first number of visitors is selected based on a probability that a percentage of the return path data households have the first number of visitors, the probability that the percentage of the return path data households have the first number of visitors being based on forum data. [6] The demographic estimation system of claim 1, wherein the visitor assignment engine comprises a visitor demographic distribution calculator to determine corresponding percentages of visitors in one of the one or more demographic categories based on the forum data. [7] The demographic estimation system of claim 1, wherein the visitor assignment engine comprises a visitor household distribution calculator to determine corresponding percentages of the return path data households with corresponding numbers of visitors. [8] At least one non-transitory computer-readable medium comprising instructions that, when executed, cause the at least one processor to at least: Generating features from return path data (RPD) reported by set-top boxes linked to return path data households, Processing the features generated from the return path data to predict demographic classification probabilities for the return path data households, wherein the process to be trained is based on forum data reported by measurement devices that monitor media devices linked to participant households; Assigning one or more demographic categories to corresponding ones of the return path data households based on the predicted demographic classification probabilities; and Assigning virtual visitors to at least a subset of the corresponding one of the return path data households based on the one or more demographic categories assigned to the corresponding one of the return path data households. [9] At least one non-transitory computer-readable medium according to claim 8, wherein the instructions, when executed, cause at least one processor to update demographic targets to account for the presence of visitors. [10] At least one non-transitory computer-readable medium according to claim 9, wherein the instructions, when executed, cause the at least one processor to apply scale factors to corresponding demographic targets to update demographic targets and account for visitor presence. [11] At least one non-transitory computer-readable medium according to claim 8, wherein the instructions, when executed, cause at least one processor to: Generating a visitor vector containing a first number of visitors; and Assigning the visitor vector to a first one of the return path data households based on respective probabilities that respective ones of the return path data households include at least one visitor, the probabilities being based on the forum data. [12] At least one non-transitory computer-readable medium according to claim 11, wherein the first number of visitors is selected based on a probability that a percentage of the return path data households have the first number of visitors, wherein the probability that the percentage of the return path data households have the first number of visitors is based on forum data. [13] At least one non-transitory computer-readable medium according to claim 8, wherein the instruction, when executed, causes at least one processor to determine, based on the forum data, corresponding percentages of visitors in one of the one or more demographic categories. [14] At least one non-transitory computer-readable medium according to claim 8, wherein the instruction, when executed, causes at least one processor to determine corresponding percentages of the return path data households with corresponding numbers of visitors. [15] System for estimating demographics of households with visitors, the system comprising: Means for generating features from return path data (RPD) reported by set-top boxes associated with return path data households; means for processing the features generated from the return path data to predict demographic classification probabilities for the return path data households, wherein the means for processing is to be trained based on forum data reported by measurement devices that monitor media devices associated with subscriber households; first means for assigning one or more demographic categories to corresponding ones of the return path data households based on the predicted demographic classification probabilities; and second means for assigning virtual visitors to at least a subset of the corresponding ones of the return path data households based on the one or more demographic categories assigned to the corresponding ones of the return path data households. [16] The system of claim 15, further comprising means for updating demographic targets to account for visitor presence. [17] The system of claim 16, wherein the means for updating demographic targets to account for visitor presence comprises applying scale factors to corresponding ones of the demographic targets. [18] The system of claim 15, wherein the second means for assigning virtual visitors comprises: means for generating a visitor vector containing a first number of visitors; and Means for assigning the visitor vector to a first one of the return path data households based on respective probabilities that respective ones of the return path data households include at least one visitor, the probabilities being based on the forum data. [19] The system of claim 18, wherein the first number of visitors is selected based on a probability that a percentage of the return path data households have the first number of visitors, the probability that the percentage of the return path data households have the first number of visitors being based on forum data. [20] The system of claim 15, wherein the second means for assigning virtual visitors comprises means for determining, based on the forum data, corresponding percentages of visitors in one of the one or more demographic categories. [21] The system of claim 15, wherein the second means for assigning virtual visitors comprises means for determining corresponding percentages of the return path data budgets with corresponding numbers of visitors.

Images